What's new

Blocking Semalt Referrer

chris

Top Contributor
Hi all,

After getting a few questions from clients on what the referrer "semalt" is I thought I'd put together a short article about it and was wondering if anyone else has noticed this in their analytics, seen any impact on their stats and if or what steps you've taken to block it etc.

Personally I've been blocking it using .htaccess, but wondered what others are doing.

Thanks in advance, I'll post the article here when it's done.

Cheers,
Chris
 

helloworld

Top Contributor
Is it just this?

Code:
http://semalt.com/project_crawler.php

Seems like a good reverse marketing plot. Webmasters posting links and mentions?
 

petermeadit

Top Contributor
Is it just this?

Code:
http://semalt.com/project_crawler.php
Seems like a good reverse marketing plot. Webmasters posting links and mentions?

Yeah it is blackhat, or maybe malicious. Their full intentions are not yet clear, aside from referrer spam.

Had a lot of sites hit by them. Crawling the entire site about four times a day. Chewing up bandwidth and jacking the bounce rate through the roof.

Because they don't identify as a crawler, they come in as visits which bounce 100%. If you have hundreds or thousands of pages on your site, then that is a lot of pages with a 100% bounce rate.

Will be interesting to see what their full intentions are.

Anyone else had these guys hit their site?
 

nina

Top Contributor
Actually I signed up for it - the free one and every day I get an email that states where I am placed in google with up to 50 names, for free - I have since then deleted my account as I was unable to see the value of it to be honest.

What shits me though is that I can't hide my own site - eg.. they don't give you a javascript file to put into your site so you can open an account and put any url in there and track them - that is the part I don't like about it.
 

chris

Top Contributor
Because they don't identify as a crawler, they come in as visits which bounce 100%. If you have hundreds or thousands of pages on your site, then that is a lot of pages with a 100% bounce rate.

I ended up adding this to .htaccess on a few sites and the bounce rate and time-on-site is back to normal.

Code:
# block semalt.com
RewriteEngine on
RewriteCond %{HTTP_REFERER} semalt\.com [NC]
RewriteRule .* - [F]
 

petermeadit

Top Contributor
Update on the semalt.com referrer issue.

Of course the script crawler.php runs not only from the main domain but also from a whole bunch of hosts, and sends referral traffic that bounces like 33 .semalt .com and 42 .semalt .com I even found traffic coming from hosts on unrelated domains like semalt .somedomain .com etc. not sure if the owners are aware they are running the semalt crawler or not.

Also the code in the .htaccess file which blocked the referrer, still actually resolved to a 403 error page and still I wanted to keep it off our sites completely.

So we decided to just redirect the traffic back.

Here is the code we ended up using:

Code:
# Block visits from semalt.com
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*semalt\.com [NC]
RewriteRule (.*) http://www.semalt.com [R=301,L]

Just in case you might like to know.


Thanks to Oz for thinking of sending the traffic back to them... :)


Cheers All
 
Last edited:

nt81

Top Contributor
We've also blocked them for clients at work.

Quite a few hosting accounts were getting smashed. Nobody gives any fark for search engine traffic other than Google / Bing / Yahoo anyway.

Seems to have fixed quite a few bandwidth issues lately. Bots be loco!
 

Ash

Top Contributor
So we decided to just redirect the traffic back.

Here is the code we ended up using:

Code:
# Block visits from semalt.com
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*semalt\.com [NC]
RewriteRule (.*) http://www.semalt.com [R=301,L]

Thanks Peter - just used this on 4 of my sites. I wonder though, any risk of them biting back?
 

petermeadit

Top Contributor
Thanks Peter - just used this on 4 of my sites. I wonder though, any risk of them biting back?

Biting back? Not sure, as James says we have seen them become quite defensive on twitter, and other platforms. They claim what they are doing is legit.

Of course the main issue we have with them is if they are going to crawl the site that is fine, but at least send traffic as a crawler, instead of a referrer that bounces 100%. Of course if this or any other referrer bounces 100% for all of the pages on my site, over and over again, then we will block it. This goes for any unwanted traffic, it gets blocked in the .htaccess file.
 

petermeadit

Top Contributor
Thanks Peter - just used this on 4 of my sites. I wonder though, any risk of them biting back?

Finding that this code is not blocking referrer any more. Not sure how they are doing it, but the traffic still hits the site and bounces 100%.
This really is referrer spam, and if you go to the website and follow the instructions and enter your website to get it removed, be prepared for further problems.

Of course the major impact of this is the User Signals, which is a major ranking factor. So if you are getting lots of referral traffic that bounces then that may affect your rankings.

Also other sites have started using referrer spam in the same way.
 

petermeadit

Top Contributor
Just another update to this one. The code I was using in my .htaccess stopped working and I was still getting hits from semalt that were bouncing 100% for sites with less traffic, this can have a big impact.
Here is a bit a bit of code to use. I have not seen any traffic in about a week now, so it really looks to be working.

BTW I found this on a WordPress thread somewhere, so I did not come up with it, but thanks to whoever did.

Code:
# Block semalt
SetEnvIfNoCase Via evil-spam-proxy spammer=yes
SetEnvIfNoCase Referer evil-spam-domain.com spammer=yes
SetEnvIfNoCase Referer evil-spam-keyword spammer=yes
SetEnvIfNoCase Via pinappleproxy spammer=yes
SetEnvIfNoCase Referer semalt.com spammer=yes
SetEnvIfNoCase Referer poker spammer=yes

Order allow,deny
Allow from all
Deny from env=spammer

# End Block semalt
 

whiterabit

Regular Member
I've been hit by the referrer spam and I have found a possible solution but it means that you have to update your .htaccess file. can anyone confirm if either of the following code snips would work:

Version 1
#block semalt
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*semalt\.com [NC]
RewriteRule (.*) http://www.semalt.com [R=301,L]

Version 2
#block referring domains
RewriteEngine on
RewriteCond %{HTTP_REFERER} semalt\.com [NC]
RewriteRule .* – [F]

Thanks
MIKE
 

petermeadit

Top Contributor
Hi whiterabit
I tried the kind of blocking you are trying with no luck. There is a plugin called Analytics Spam Blocker. It is by Luke Williamson http://lukewilliamson.com.au/ It creates the following output which you could just paste into your .htaccess file if you like. But might be better to install the plugin.

# Analytics Spam Blocker - Start
SetEnvIfNoCase Referer semalt.com spambot=yes
SetEnvIfNoCase Referer darodar.com spambot=yes
SetEnvIfNoCase Referer buttons-for-website.com spambot=yes
SetEnvIfNoCase Referer fbdownloader.com spambot=yes
SetEnvIfNoCase Referer descargar-musicas-gratis.com spambot=yes
SetEnvIfNoCase Referer baixar-musicas-gratis.comsavetubevideo.com spambot=yes
SetEnvIfNoCase Referer srecorder.com spambot=yes
SetEnvIfNoCase Referer kambasoft.com spambot=yes
SetEnvIfNoCase Referer ilovevitaly.com spambot=yes
SetEnvIfNoCase Referer ilovevitaly.co spambot=yes
SetEnvIfNoCase Referer ilovevitaly.ru spambot=yes
SetEnvIfNoCase Referer blackhatworth.com spambot=yes
SetEnvIfNoCase Referer priceg.com spambot=yes
SetEnvIfNoCase Referer backgroundpictures.net spambot=yes
SetEnvIfNoCase Referer embedle.com spambot=yes
SetEnvIfNoCase Referer extener.com spambot=yes
SetEnvIfNoCase Referer extener.org spambot=yes
SetEnvIfNoCase Referer fbfreegifts.com spambot=yes
SetEnvIfNoCase Referer feedouble.com spambot=yes
SetEnvIfNoCase Referer feedouble.net spambot=yes
SetEnvIfNoCase Referer japfm.com spambot=yes
SetEnvIfNoCase Referer joinandplay.me spambot=yes
SetEnvIfNoCase Referer joingames.org spambot=yes
SetEnvIfNoCase Referer iskalko.ru spambot=yes
SetEnvIfNoCase Referer musicprojectfoundation.com spambot=yes
SetEnvIfNoCase Referer myprintscreen.com spambot=yes
SetEnvIfNoCase Referer slftsdybbg.ru spambot=yes
SetEnvIfNoCase Referer edakgfvwql.ru spambot=yes
SetEnvIfNoCase Referer openfrost.com spambot=yes
SetEnvIfNoCase Referer openfrost.net spambot=yes
SetEnvIfNoCase Referer openmediasoft.com spambot=yes
SetEnvIfNoCase Referer serw.clicksor.com spambot=yes
SetEnvIfNoCase Referer socialseet.ru spambot=yes
SetEnvIfNoCase Referer sharebutton.net spambot=yes
SetEnvIfNoCase Referer cityadspix.com spambot=yes
SetEnvIfNoCase Referer screentoolkit.com spambot=yes
SetEnvIfNoCase Referer softomix.com spambot=yes
SetEnvIfNoCase Referer softomix.net spambot=yes
SetEnvIfNoCase Referer softomix.ru spambot=yes
SetEnvIfNoCase Referer gobongo.info spambot=yes
SetEnvIfNoCase Referer myftpupload.com spambot=yes
SetEnvIfNoCase Referer websocial.me spambot=yes
SetEnvIfNoCase Referer luxup.ru spambot=yes
SetEnvIfNoCase Referer ykecwqlixx.ru spambot=yes
SetEnvIfNoCase Referer soundfrost.org spambot=yes
SetEnvIfNoCase Referer seoexperimenty.ru spambot=yes
SetEnvIfNoCase Referer cenokos.ru spambot=yes
SetEnvIfNoCase Referer star61.de spambot=yes
SetEnvIfNoCase Referer superiends.org spambot=yes
SetEnvIfNoCase Referer vapmedia.org spambot=yes
SetEnvIfNoCase Referer econom.co spambot=yes
SetEnvIfNoCase Referer vodkoved.ru spambot=yes
SetEnvIfNoCase Referer adcash.com spambot=yes
SetEnvIfNoCase Referer videofrost.com spambot=yes
SetEnvIfNoCase Referer youtubedownload.org spambot=yes
SetEnvIfNoCase Referer zazagames.org spambot=yes
SetEnvIfNoCase Referer 7makemoneyonline.com spambot=yes
SetEnvIfNoCase Referer ranksonic.info spambot=yes
SetEnvIfNoCase Referer hulfingtonpost.com spambot=yes
SetEnvIfNoCase Referer viandpet.com spambot=yes
SetEnvIfNoCase Referer a-hau.mk spambot=yes
SetEnvIfNoCase Referer cfsrating.sonicwall.com:8080 spambot=yes
SetEnvIfNoCase Referer yougetsignal.com spambot=yes
SetEnvIfNoCase Referer cenoval.ru spambot=yes
SetEnvIfNoCase Referer bestwebsiteawards.com spambot=yes
SetEnvIfNoCase Referer bestwebsitesawards.com spambot=yes
SetEnvIfNoCase Referer simple-share-buttons.com spambot=yes
SetEnvIfNoCase Referer adviceforum.info spambot=yes
SetEnvIfNoCase Referer o-o-6-o-o.com spambot=yes
SetEnvIfNoCase Referer o-o-8-o-o.com spambot=yes
SetEnvIfNoCase Referer humanorightswatch.org spambot=yes
SetEnvIfNoCase Referer smailik.org spambot=yes
SetEnvIfNoCase Referer s.click.aliexpress.com spambot=yes
SetEnvIfNoCase Referer social-buttons.com spambot=yes
SetEnvIfNoCase Referer 4webmasters.org spambot=yes
SetEnvIfNoCase Referer best-seo-offer.com spambot=yes
SetEnvIfNoCase Referer best-seo-solution.com spambot=yes
SetEnvIfNoCase Referer buttons-for-your-website.com spambot=yes
SetEnvIfNoCase Referer www.Get-Free-Traffic-Now.com spambot=yes
SetEnvIfNoCase Referer free-share-buttons.co spambot=yes
SetEnvIfNoCase Referer theguardlan.com spambot=yes
SetEnvIfNoCase Referer googlsucks.com spambot=yes
SetEnvIfNoCase Referer buy-cheap-online.info spambot=yes
SetEnvIfNoCase Referer forum69.info spambot=yes
SetEnvIfNoCase Referer meendo-free-traffic.ga spambot=yes
SetEnvIfNoCase Referer www.kabbalah-red-bracelets.com spambot=yes
SetEnvIfNoCase Referer pornhub-forum.ga spambot=yes
SetEnvIfNoCase Referer pornhubforum.tk spambot=yes
SetEnvIfNoCase Referer anal-acrobats.hol.es spambot=yes
SetEnvIfNoCase Referer youporn-forum.ga spambot=yes
SetEnvIfNoCase Referer sexyteens.hol.es spambot=yes
SetEnvIfNoCase Referer amanda-porn.ga spambot=yes
SetEnvIfNoCase Referer generalporn.org spambot=yes
SetEnvIfNoCase Referer depositfiles-porn.ga spambot=yes
SetEnvIfNoCase Referer rapidgator-porn.ga spambot=yes
SetEnvIfNoCase Referer torture.ml spambot=yes
SetEnvIfNoCase Referer domination.ml spambot=yes
SetEnvIfNoCase Referer webmaster-traffic.com spambot=yes
SetEnvIfNoCase Referer youporn-forum.uni.me spambot=yes
SetEnvIfNoCase Referer www.event-tracking.com spambot=yes
SetEnvIfNoCase Referer free-share-buttons.com spambot=yes
SetEnvIfNoCase Referer free-social-buttons.com spambot=yes
SetEnvIfNoCase Referer guardlink.org spambot=yes
SetEnvIfNoCase Referer redtube-talk.ga spambot=yes
SetEnvIfNoCase Referer sanjosestartups.com spambot=yes
SetEnvIfNoCase Referer trafficmonetize.org spambot=yes
SetEnvIfNoCase Referer sitevaluation.org spambot=yes
SetEnvIfNoCase Referer 100dollars-seo.com spambot=yes
SetEnvIfNoCase Referer dailyrank.net spambot=yes
SetEnvIfNoCase Referer howtostopreferralspam.eu spambot=yes
Order allow,deny
Allow from all
Deny from env=spambot
# Analytics Spam Blocker - End

Here is the link to the plugin https://wordpress.org/plugins/analytics-spam-blocker/

HTH
 

Community sponsors

Domain Parking Manager

AddMe Reputation Management

Digital Marketing Experts

Catch Expired Domains

Web Hosting

Members online

No members online now.

Forum statistics

Threads
11,098
Messages
92,044
Members
2,394
Latest member
Spacemo
Top