analysis of a spammer

[xDamox]

Here is what I got:

 

Code:

CustomLog /var/log/apache/access.log common env=good
CustomLog /dev/null common env=!good

<IfModule mod_setenvif.c>
SetEnvIf Referer ".*charlestyrrell-ins.         !good
SetEnvIf Referer ".*networkresourceservices.     !good
SetEnvIf Referer ".*northeastmetrotec.         !good
    SetEnvIf Referer ".*reesehardin.             !good
    SetEnvIf Referer ".*vicotriajohnson.         !good
SetEnvIf Referer ".*advertisinggems.         !good
SetEnvIf Referer ".*clickobras.             !good
SetEnvIf Referer ".*nativealaaskan.         !good
SetEnvIf Referer ".*downjigger.             !good
SetEnvIf Referer ".*hedcore.                 !good
SetEnvIf Referer ".*hellwithgoogle.         !good
SetEnvIf Referer ".*isdwebstore.             !good
SetEnvIf Referer ".*redline-entertainement.    !good
SetEnvIf Referer ".*skateinstrutor.         !good
SetEnvIf Referer ".*slewfootrecrods.         !good
SetEnvIf Referer ".*syperopts.             !good
SetEnvIf Referer ".*charlestyrrell-ins.        !good
SetEnvIf Referer ".*wgostonemantel.            !good
SetEnvIfNoCase User-Agent ^(        !good
</IfModule>

 

Might need tewaking.

[hybrid]

I don't know if this is any use to you, anyweb, but I thought it would be worth asking:

 

[/url][url=http://skyzyx.com/projects/blocker/]http://skyzyx.com/projects/blocker/

[anyweb]

thanks guys

 

what I have done is to implement three things

 

1. blocked the ENTIRE netcathost C class ip range via iptables

 

Code:

DROP       all  --  195.225.177.0/255.255.255.0      anywhere

 

 

2. implemented znx's .htaccess script in the root of the webserver

 

note: in the code below znx asked me to remove the line that had HEAD in it, I still don't know why, perhaps he will explain here !

 

 

Code:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} "^(" [NC]
RewriteRule ^(.*) - [F,E=BAD:1,L]

CustomLog /var/log/apache/access.log combined env=!BAD
CustomLog /var/log/apache/access_bad.log combined env=BAD

 

3. added user/pass for the stats usage page (for now) if you want access to it contact me or znx

 

Still waiting to accomplish the following:-

 

use grep and other tools to filter out all references to the spammed URLS from my Apache access_log, referrer_log etc.

 

 

 

cheers

anyweb

[znx]

1. blocked the ENTIRE netcathost C class ip range via iptables

 

This is probably a good thing but proxies are so easy to find these days.. nevertheless could stop those that are just automating attacks

 

2. implemented znx's .htaccess script in the root of the webserver 

note: in the code below znx asked me to remove the line that had HEAD in it, I still don't know why, perhaps he will explain here !

 

Indeed the matching of the ua should be enough, therefore there is no need to remove HEAD. The removal of HEAD wouldn't restrict normal browsers of your site in anyway but certain utilities use HEAD to confirm pages exist and even some browsers to assist with preloading a site, hence unless we have to I think we should try to keep HEAD active.

 

Also maybe I should have pointed out this is the htaccess:

Code:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} "^(" [NC]
RewriteRule ^(.*) - [F,E=BAD:1,L]

 

This is the modification to logging (inside the httpd.conf you will already have a CustomLog line):

Code:

CustomLog /var/log/apache/access.log combined env=!BAD
CustomLog /var/log/apache/access_bad.log combined env=BAD

 

Unfortunately I'm not much convinced about the matching of ua, the little test we did.. it didn't seem to 403 me..as it should do :( We will need to experiment a wee bit more.

 

3. added user/pass for the stats usage page (for now) if you want access to it contact me or znx

 

This of course is the killer, no longer can your stats be googlefied and as such it is pointless for the spammers to spam referers anymore. Doesn't me we don't want to purge the spammers though.

 

Still waiting to accomplish the following:- 

use grep and other tools to filter out all references to the spammed URLS from my Apache access_log, referrer_log etc.

 

Shouldn't be too hard... taking the small snip from above we can clear the logs with relative ease, strip the HEAD entries and thats it... of course we should confirm that no others are getting purged so.

 

Test the removal with:

Code:

grep -E "^195.225.177.*HEAD" access_log > wouldberemoved

 

If all the entries in that new file are duds... then procede with:

Code:

grep -Ev "^195.225.177.*HEAD" access_log > new_access_log

 

As long as the referers are all of a similar style.. we should be fine.

 

:) we shall prevail .. no?

[znx]

DOH DOH DOH ! :)

 

i know my error now :)

 

Code:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^\( [NC]
RewriteRule ^(.*) - [F,E=BAD:1,L]

 

the ( is of course being picked up as part of a regex match.. so we needed to escape it :)

 

WOOT... so alter the htaccess.. alter the httpd.conf (CustomLog lines) and all should be well

 

PHEW :)

 

[PS: thanks to McDuck for the server space to test on]

[anyweb]

so far so good

 

those spamming basta@ds NETCATHOST (spammers)

 

are blocked by iptables ! and it's working so far

 

heres a very interesting article on the spamming subject

 

cheers

anyweb

[anyweb]

znx please check the usage stats

now i am being spammed by 888 casinos

 

in fact, ever month a new spammer seems to take up the action, my rc.firewall is getting big with all these losers ip's and having to check the stats daily to figure out whats bad from good is getting a bit annoying

 

any advice ?

 

cheers

anyweb

[YeOK]

This link is a wild card block list you could try...

 

Not mine, but someone else who is having the same problem....

 

Wild Card Block List

 

edit: Also found this if it helps...

 

CrawlerController

[znx]

any advice ?

 

I will look into.. I have a copy of your access_log and rc.firewall, will give it a look over soon .. apologies for the delay :)