New anti-spam tools coming

[Bill]

The number of spam attempts has grown enormously in the past couple months. Chris is putting a lot of time into integrating some new anti-spam tools, but they need refinement. In the last couple days, a few legitimate posts have gone into Moderator Hell - a holding pen for "questionable" posts. Most of the hell-bound posts really are spam, and by holding them, the tool gives me the opportunity to ban the spammer. At the same time, I have been releasing the legitimate posts, of course. The tools need fine-tuning, so please be patient as Chris works to get this capability up and running for us.

For a long time I have dealt with spammers manually, but recently the spam load has been so high that I can no longer deal with them effectively this way. We really have to have these automated tools. Please be patient with us as Chris works to get it all up and running for us. In the meantime, if you get a message saying "Your post was held for the moderator", send me a PM and I will get it out of there right away.

Thanks, Chris, for all you do.

Bill

[B_and_D]

Thank you Bill, and thank you Chris for all your moderating & work maintaining this board. I don't know where we'd be without you.

[ShrimpBurrito]

Thank you Bill, and thank you Chris for all your moderating & work maintaining this board. I don't know where we'd be without you.

Ditto! You guys are great!

Dave

[scrubjaysnest]

There is always a little pain with something new. Keep up the good work

[rumbleweed]

I would much rather have a post sit in "moderator hell" for a day or two than have to sort through the tons of SPAM that would otherwise clutter up this board. I have unsubscribed to several other boards I belong to simply because over 80% of the posts are spam and finding useful information is impossible.
Keep up the good work.

[Bill]

We appreciate the good words, folks. The sad fact is that this board would cease to exist without the anti-spam tools and efforts. Among the major attractions of this Board are the lack of spam and the lack of paid advertising. The content provided by all of you is amazing, but as you said, Bob, I don't think many of the Sponsors would continue to pay the fees to operate the Board if the content were to become submerged in spam. And the Board would die. Simple as that.

The spammers kill these Boards. In a very real sense, they are their own worst enemies.

Bill

[Chris]

I wanted to give you guys an update. We're working on a couple of different approaches, some working well, others causing pain. LOL. I have just changed the verification back to questions, simple questions that have lots of words and no numbers. That has had good luck with some of the other boards. Second we've joined efforts with other boards to stop this spam flood and to identify them and block them. There are some other things that we may use depending on the severity of the continued attacks.

We are not alone in this, but every year as they come at us, we all band together, figure out some solutions and put a short term end to it. I appreciate your patience as we figure this out.

I specifically want to thank Bill for all of his continued efforts. If you get a free moment, pop him an IM and thank him as well. We really do appreciate everything you do here Bill.

Thanks, and don't hesitate to contact me directly if you need to, and thanks to those who already have.

Ct..

[rumbleweed]

THANK YOU as well. I run a few smaller boards for non profits and feel your pain. We are fortunate that we get to see the results, but do not see all the effort and time to get there. Thank you and Bill for all you efforts it IS appreciated.

[TravlinOn]

Please excuse this suggestion if it is really dumb, but would it be possible, in addition to what is already being set up with code to get rid of spam, to do the following:

1. Add to the "Post Quick Reply" a "Smilies" for "S-P-A-M".
2. Add an internal procedure that counts, as such "Smilies" are responded in each unique "Quick Reply" to a suspect posting (unique meaning from different sponsors).
3. Add an internal procedure that AUTOMATICALLY blocks that poster after a count of 3 unique poster's "spam smiles", until Bill receives an eMail from the blocked poster to explain the intent of the "spammed" post and determines whether to re-establish the banned poster.
4. Add a publicly available current list of blocked posters.

The advantage of such a system is that it puts the work load out on all sponsors who routinely read the site, and that appears to be a bunch, and automates the "spam" response to where Bill only has to evaluate and act on the few "explanations" that are received.

Just trying to be helpful, and feel free to discard without response if you wish. I too have come to treasure this site and would hate to lose it. :)

[Bill]

I can't answer whether it is "possible". But you have to remember that this board, like most others, operates with a purchased software package. In our case, it is vBulletin. And no one is going to rewrite a commercial software package.

The bounds of "possible" shrink a lot when the question is rephrased as "does the vBulletin software give you the ability to ..."

When the board was brand new, almost 10 years ago, I hounded Chris to rewrite the software to make simple changes. I didn't like the login procedure, could he change it? I hate the logout procedure (still do), could he change it? Could he move this green button to the other corner of the screen, and make it blue? This version of a smiley is hard to read, could you substitute that one?

Chris was very patient, and didn't berate me, and I gradually learned the ways of the world. This may be a similar case.

Bill

[mcgyver210]

I can't answer whether it is "possible". But you have to remember that this board, like most others, operates with a purchased software package. In our case, it is vBulletin. And no one is going to rewrite a commercial software package.

The bounds of "possible" shrink a lot when the question is rephrased as "does the vBulletin software give you the ability to ..."

When the board was brand new, almost 10 years ago, I hounded Chris to make changes. I didn't like the login procedure, could he change it? I hate the logout procedure (still do), could he change it? Could he move this green button to the other corner of the screen, and make it blue? This version of a smiley is hard to read, could you substitute that one?

Chris was very patient, and didn't berate me, and I gradually learned the ways of the world. This may be a similar case.

Bill

This my main reasons why I have never liked VBulletin or any paid board format. They limit what you can do with your board as far as easy customization. I always use either PHPBB or MyBB which are free board software with lots of free & some paid support along with virtually unlimited addon scripts.

Bill one of the easiest ways to combat spam is to have more mods. This board is a target due to daily traffic & the fact it is easy to sign up for membership.

There are other things that can be done also which can include but not limited to:

1. Set up & use the built in Report Function which with most boards is farily versatile & semi-automatic.

2. You could go to all new members need to be mod approved first which isn't fool proof but cuts down on fake members & does require more mods to be available. This is a method I used on most of my board projects in the past.

3. You can limit posting permissions of all members with 0 - whatever number post count to mod approval before their post show up at first. This is a very popular well used method by many boards now days but does require more trusted mods.

4. We could also limit new posters to one forum category until they post a few post or are manually approved for their full access to their 30 days.

There are many more methods I could mention but I don't know if VBulletin would work with them due to its limitations & my inexperience with it. Most methods that I have personally see work involve more hands on mods unfortunately since most any addons are computer controlled & eventually can be beaten by Spammers.


Most people that frequent boards now days know & understand even though many of the Spam prevention methods used today are inconvenient they are necessary to control SPAM.

Good Luck with this issue it is an ongoing back & forth battle once your board is on the SPAM Radar as you know.

[Bill]

Interesting thoughts, Billy. My thoughts in response are:
This my main reasons why I have never liked VBulletin. I always use either PHPBB or MyBBNot really helpful. This board is built on vBulletin. It is unlikely that we will switch to either PHPBB or MyBB at this point, so that is off the table. Of course an Admin decision and Admin action could make it happen, so you might want to take it up with Chris.

one of the easiest ways to combat spam is to have more mods.Already under discussion. I have someone in mind. Not many members are willing to step into the snake pit, though, so we will see what he says.

1. Set up & use the built in Report FunctionAs you know, the Report Function is already set up and works. As you also know, I have asked people not to use it, since all it does is dump an email into my Inbox. Dealing with multiple email reports in my Inbox is certainly no easier than dealing with the spam reports in the thread we set up for that purpose. And it does nothing to stop the spam in the first place.

2. You could go to all new members need to be mod approved Certainly possible, and at least it is pro-active - it lets you sort out the spammer before he posts. As you note, it would require another mod, and I would be worried that a new signup might fall into the cracks between us. But that is manageable. A variant, which I would like to see, is a hold period between the moment a new member registers (this is when I can see his parameters for the first time), and the moment when the enabling email is dispatched and he can log in. Right now, the process is almost instant! Chris, it's TOO GOOD! A 48-hour hold would give me time to see the guy's information and decide what to do with him, before he is given the ability to log in. And if he can't log in, he can't post. And that's what we're looking for.

3. You can limit posting permissions of all members with 0 - whatever number post count to mod approval before their post show up at first.You're right - it certainly does increase the work load. As noted above, I would rather head them off earlier in the process, such as at Registration To me this is better than waiting for them to post, and taking them down then. And the spammers are ahead of you. We have seen several cases recently in which a new guy comes aboard and makes a very innocent first post - something like "Yeah, I agree." Then his second post is spam.

4. We could also limit new posters to one forum category until they post a few postI'm not sure how this will help. The spammers don't care what forum they post in. If you give them permission to post in one particular forum, that's where they will spam. And that's what we are trying to prevent in the first place.

In summary, most of your approaches are quite labor-intensive - hence the need for more moderators. I think that a better approach is to do a better job with the automated tools, and I think we can do it. I may be wrong.

We actually have one more option that you didn't mention. I believe you and I have talked about what I call the Nuclear Option. Chris and I have had a short discussion about it, but it needs to be discussed again.

Good Luck with this issue it is an ongoing back & forth battle once your board is on the SPAM Radar as you knowThanks, Billy. We'll keep working at it. We certainly are on somebody's radar!

Bill

[Chris]

Hi gang,
I have enabled moderation for all new members. They will not be able to post until I clear them. If this affects you as a new member, I apologize, but I won't change it until we get this resolved.

I have run a number of BB's over the past 13 years...PHPBB, YABB, etc, but the reason that we run VB is that it is very well supported. We have been and are still on occasion the target of hackers. Using VB kept us afloat, and the available mods for the forum are solid. Almost all of the top BB sites run on VB, and there is a reason for that. Enough on this subject.

At this point, I am locking down the board tighter so that we keep things out. I will personally have to verify any new user for the next couple of weeks and that will also show me if there are other things going on like data base injections. I'll spare you guys the details, but we should be able to make this stop pretty quickly if I don't let anyone but people I verify on to the board.

Thanks!
Ct..

[Chris]

Great ideas! I'm not adverse to coding changes, and we could do this, but at this point, we're going to try some more simple changes first.

This site will not go down. We're pretty scrappy when we need to be, and now we need to be. I appreciate your patience as we continue after the spammers.

Thanks
Ct..

Please excuse this suggestion if it is really dumb, but would it be possible, in addition to what is already being set up with code to get rid of spam, to do the following:

1. Add to the "Post Quick Reply" a "Smilies" for "S-P-A-M".
2. Add an internal procedure that counts, as such "Smilies" are responded in each unique "Quick Reply" to a suspect posting (unique meaning from different sponsors).
3. Add an internal procedure that AUTOMATICALLY blocks that poster after a count of 3 unique poster's "spam smiles", until Bill receives an eMail from the blocked poster to explain the intent of the "spammed" post and determines whether to re-establish the banned poster.
4. Add a publicly available current list of blocked posters.

The advantage of such a system is that it puts the work load out on all sponsors who routinely read the site, and that appears to be a bunch, and automates the "spam" response to where Bill only has to evaluate and act on the few "explanations" that are received.

Just trying to be helpful, and feel free to discard without response if you wish. I too have come to treasure this site and would hate to lose it. :)

[rumbleweed]

Chris
I ended up doing the same thing on a site I run. It was actually less work approving the few new members per week than trying to catch and remove 3 times as much SPAM. A one time short delay to sign up is a small investment for a. Spam free board.

[mcgyver210]

Hi gang,
I have enabled moderation for all new members. They will not be able to post until I clear them. If this affects you as a new member, I apologize, but I won't change it until we get this resolved.

I have run a number of BB's over the past 13 years...PHPBB, YABB, etc, but the reason that we run VB is that it is very well supported. We have been and are still on occasion the target of hackers. Using VB kept us afloat, and the available mods for the forum are solid. Almost all of the top BB sites run on VB, and there is a reason for that. Enough on this subject.

At this point, I am locking down the board tighter so that we keep things out. I will personally have to verify any new user for the next couple of weeks and that will also show me if there are other things going on like data base injections. I'll spare you guys the details, but we should be able to make this stop pretty quickly if I don't let anyone but people I verify on to the board.

Thanks!
Ct..

"WOW" it is even worse than I thought sorry you have been dealing with so much Chris. Sounds like more than standard spammers. I hope you didn't take any offense to my comments & suggestions I was just trying to pitch some ideas out there as to how we handled SPAM in the past but I am sure I am behind since I haven't modded or administered any boards in quite awhile.

I do think strict Moderation is a good way to go for now though since so far all automatic scripts always get beat at some point.


I stay pretty busy these days with my business & sites but if you need some help just ask & I will try to help out if possible.

[mcgyver210]

Chris
I ended up doing the same thing on a site I run. It was actually less work approving the few new members per week than trying to catch and remove 3 times as much SPAM. A one time short delay to sign up is a small investment for a. Spam free board.

That was the most effective thing we did also & really worked best accept for the inconvenience to new members.

[Chris]

So far only two causalties. One person was banned by IP address, and one new user that must prove himself to me before I approve him. I agree with you Bob that it's easier for Bill and I to moderate new users than spam. We'll see how it goes. If they're not using SQL injections, we should be good and can ride out the storm. If someone you know is suddenly banned, let me know.
Thanks
Ct..

[rumbleweed]

Just a quick note to the moderators to say thank you again for your efforts to keep this board clean. I know the new manual method is work on your part, but that work has not gone unnoticed. It is a pleasure to open the forum each morning to SPAM free posts.
THANK YOU

[countrygirl]

Just a quick note to the moderators to say thank you again for your efforts to keep this board clean. I know the new manual method is work on your part, but that work has not gone unnoticed. It is a pleasure to open the forum each morning to SPAM free posts.
THANK YOU

I agree!!! Thank YOU!

[Chris]

So I put up a freebie board for a site that I am working on for my wife and I's new business...trettalpc.com. It's a simple machines forum which is the follow on product from what used to be YABB which we ran prior to going to VBulletin. Now I haven't done anything to the forum other than install it. It's been up for a while now, but just an empty shell. It has the following - all from spam:
18461 Posts in 13621 Topics by 24318 Members

The funny part is that there were a maximum of 58 members online yesterday...all spammers.

Wow.
Ct..

[countrygirl]

Those people obviously have too much free time. ;)

[ShrimpBurrito]

Wow, that is just incredible. Roughly how long had the site been live?

Dave

[rumbleweed]

Most of them are probably not real people, but spam bots. I manage a local church group food bank site here and was getting 10-20 new members and posts per day (98% of which were from Russia and India). Tried lots of things then finally disabled online registration. As it is a small group of people that so post I manually add them and rest of the users use it as read only which works fine.

[Chris]

Just a couple of weeks. Pretty sad.
Ct..

Wow, that is just incredible. Roughly how long had the site been live?

Dave