Spam Fighting and Lessons Learned

re: Spam Fighting and Lessons Learned

mjswart — Wed, 16 Dec 2009 17:05:25 GMT

Very a propos. About an hour ago I wrote a post on the topic of learning from failures. http://michaeljswart.com/?p=340

re: Spam Fighting and Lessons Learned

AaronBertrand — Wed, 16 Dec 2009 17:10:19 GMT

Adam, do you know how to stop the topsy crap? I get a comment from topsy on every post, "this post was mentioned on twitter..."

re: Spam Fighting and Lessons Learned

Adam Machanic — Wed, 16 Dec 2009 17:40:15 GMT

Michael, glad you enjoyed the post. Yours is one of the accounts I killed :-)

Aaron, I've allowed that topsy stuff (in addition to ubervu) to come through since just before it started I'd had requests for "tweetback" support on the site and that's what it seems to provide. Do you consider it to be spam? I can certainly reevaluate...

re: Spam Fighting and Lessons Learned

AaronBertrand — Wed, 16 Dec 2009 18:29:38 GMT

I can see how some would see value in it, but I don't really.

re: Spam Fighting and Lessons Learned

Michael K. Campbell — Wed, 16 Dec 2009 19:31:00 GMT

You guys need to give ReverseDOS a try.

It's a simple HTTP Module for ASP.NET sites and uses filters to let you flag/trap spammy crap (patterns via plain text or regex). When it finds a matching result (or spammy comment) it lies to the bot pushing the spam up and tells it that it's 'processing' the request... for about a minute or so (you can configure the time as needed) then it throws a 503 - Server Busy. Other users on your site won't see anything out of the ordinary. But you'll end up:

a) stopping the spam from going into your DB

b) slowing the BOT down.

I've found that most bots (or their owners) notice that you're slowing their bots down significantly, and start to leave you alone after a while.

RDOS is designed to TRUST content from authenticated or logged-in users, but it shouldn't be too hard to turn that functionality off. (Let me know if you need a hand with that.)

http://www.angrypets.com/tools/rdos/

re: Spam Fighting and Lessons Learned

andyleonard — Wed, 16 Dec 2009 20:26:13 GMT

Adam: That's pretty harsh. I mean, I try to sell a few wireframe Christmas trees and you lock me out!

Aaron: I moderate comments. That caught a lot of the stuff I used to get.

:{> Andy

re: Spam Fighting and Lessons Learned

Adam Machanic — Wed, 16 Dec 2009 20:50:32 GMT

Michael: We're already running it :-) ... it's certainly saved us from a lot of stuff, but not everything.

re: Spam Fighting and Lessons Learned

AaronBertrand — Wed, 16 Dec 2009 23:51:45 GMT

Andy,

YMMV, but I don't like the idea of moderating comments. It means that I have to "okay" the conversations of others and that they can't carry on in my absence. Since there are many occasions in a given month where I don't visit the site for a week or more, I certainly would rather let a few crap comments get in than stifle discussions altogether.

re: Spam Fighting and Lessons Learned

Jimmy May, Aspiring Geek — Thu, 17 Dec 2009 04:03:49 GMT

Adam, I'm sympathetic, applaud your efforts, & am proud of your mea culpas.

re: Spam Fighting and Lessons Learned

Brian Tkatch — Thu, 17 Dec 2009 14:52:05 GMT

Adam, good story, and nice that you posted in. It's good to know you are a real person. :)

re: Spam Fighting and Lessons Learned

KKline — Wed, 23 Dec 2009 22:40:31 GMT

Great post, Adam. I especially appreciate that it probably took you as long to write the informative article as it did to write your initial scripts or run your recovery. It takes time to instruct others, so you've done the community a services.

I don't moderate comments, incidentally, but I've got alarms set whenever they come in and, when appropriate, delete them.

re: Spam Fighting and Lessons Learned

Web Design Company — Wed, 30 Dec 2009 07:32:07 GMT

Excellent post Adam.I really appreciate your effort.