I’ve posted about this subject before in “How to Deter Scrapers and Hotlinkers“, which discusses a bit more of the hands-on and some web based tools you can use to help protect your content.

Many of us have may have read all sorts of articles and other blog posts that delve into the legalities, copyright laws, rights and so forth. That’s not something I’m going to discuss as I think there’s too much discussion (and not enough action). If not, just take a look on Google and you’ll find a plethora of posts on the subject. The fact of the matter remains that we can file as many DMCA notices, cease and desist letters, etc. as we want. Often the content thief (called a “scraper” or “splogger”) does not care. After all they already have your content and you can cry and stamp your feet as much as you want – Many of them will simply not budge, especially if they are out of your geographical area or jurisdiction.

A case in point, on one of my other (self hosted) wordpress blogs, several pages were regularly being lifted by a web site in China. The only recourse was that Google removed the stolen content from search results (the hosting provider, etc. did nothing) – Even though Google was removing their search results, that didn’t stop them from adding more (from my stolen content)! Locally (North/South America and Europe) however, that blog has had a 100% success record in take downs of stolen content found on blog sties such as wordpress.com (Automattic), blogspot.com (Google), etc. Providing I followed their DMCA procedures, all issues were resolved.

In my opinion, this approach is ineffective, costly (with long distance faxing) and quite frankly a waste of time (at one point I was filing about 15 DMCA’s a day for that blog). Why? We need to be proactive not reactive! A client of mine uses the tagline “Predictable is preventable” for his blog and security business. And he’s right! We need to deter, curtail and control such theft attacks against our blog content.

It’s difficult to stop such activity, until you can see it occurring, as such, I hope the following tools will help you:

Grab yourself a copy of Antileech. It’s a plugin that does not stop sploggers, rather (in the developers words) “…produces a fake set of content especially for them that includes links back to your site and sends it only to them. When they steal this content, it appears online just like normal, except now you’ve turned the tables on them and have provided them with useless content…” The benefits here are that sploggers seldom read all the content. They have an automatied system grabbing thousands of pages – And now they will have backlinks to your original content, inviting the reader of the splog (containing fake content) to visit your blog instead. That’s link love!

Another effective tool in your arsenal is ©Feed, which allows you to place a digital fingerprint and copyright notice in your content feed (RSS). For those not familiar, in most cases, your RSS is used to facilitate content theft. What’s RSS? Common Craft’s video below easily explains it:

In the words of the ©feed developers “…You can use html. You can add the IP of a feed reader and digital fingerprint for an explicit key. There can also be a domain name for a whitelist and this domains became not the message [sic]. The plugin search for this key at content theft [sic]. It is furthermore possible to add comments and related posts to the feed. For the related post feature it uses a database-search for the content. You can use the plugin “Simple Tagging” for related posts in a feed. The copyright notice can be added even when using entry excerpts…”

Needless to say, the above two tools are very robust and take a proactive approach.  There are some other resources I’ve bumped into over time, that provide good reading and further insight:

• Jamming the scraper signals
• How You Can Stop Dirty Feed Scrapers In 3 Easy Steps
• Benefiting from Blog Scrapers
• Google – Duplicate content due to scrapers

Hopefully the above helps you (especially those who emailed me asking)!

Do you have any good solutions? What tools do you use? Feel free to comment your thoughts below.

• Top 100 of the Best (Useful) OpenSource Applications.
• Why I Quit Windows and Switched to Linux.
• Why is Linux Faster than Windows?

I enjoyed the traffic those posts provided (and still provide to this day) as they helped my blog grow. I think they were popular because they provided a valuable resource as well as elicited fair bit of discussion, as some of the posts and resulting comments were very outspoken and opinionated. One of the above posts (Why I Quit Windows and Switched to Linux) was a very personal story describing some of my career experiences and how they effected my professional life and thoughts.

Imagine my surprise when I found another blog with the identical content on it. So much so, that the author who scraped my content claimed it as his own! At that time I had more than enough technical knowledge to initiate actions that would catch-out scrapers, but I’d not yet fully experienced some of the nitty-gritty administrative aspects such as contacting the hosting providers, filing DMCA notices et al.

For that matter, why would this be an important issue? Why should bloggers (or any web site owner for that matter) take actions to mitigate such issues? Well a few good reasons immediately come to mind:

1. It’s annoying. A blogger puts a fair bit of work into his or her post, only to find it copied elsewhere and used to earn revenue (usually via advertising) for the content thief.



2. It can impact your web hosting cost if you have a busy site. Remember, most hosting accounts have a monthly bandwidth allowance. Exceed the monthly bandwidth and the blog owner incurs extra cost. But wait!… How did scrapers cause my cost to increase? Simply put, they copied the text content onto their blogs and linked the images (in that content) from my blog. This resulted in the text being duplicated on their site and the images being stored on my hosting account. When a web browser viewed the content on their page, it was pulling the images (for that content) from my hosting account, for which I had to pay the bandwidth.



3. It can reduce your Google (SEO) ranking. How does that happen? When Google finds content, it tries to determine if the content is original (not copied from another site) and proceeds to provide it with page ranking data. It is conceivable that content can be copied and receive a page rank (and inclusion in Google search results), without the original blog article being yet found by Google. If it get’s found by Google later, how can you ensure that your original becomes noted as the original post? Don’t misunderstand me, search engines like Google do try to remove duplicate content, but it becomes difficult when your original content becomes listed as the duplicate.

One important issue I realized is that you cannot be overly emotional about such actions. When trying to fix such issues, you need to work with various parties such as Google DMCA, Hosting Providers, etc. Sending them flippant or angry letters is not going to get you the help you need. Remember, large organizations and businesses deal with such issues daily and they are not impressed with theatrics. You need to have these people on your side. Remaining calm, assertive and professional goes a long way to getting support (as does courtesy).

Let’s define a couple things before we move on…

• A scraper is someone who copies your content and places it on another site, without your permission. In essence this is theft of your intellectual property.
• A hotlinker is someone who displays your images on another site, and uses the coding on their page is such a way so as to pull the image that is stored on your server (hosting account), for display on another page. As I mentioned earlier, this is tantamount to bandwidth theft.

I’m often asked how I discover that my content has been scraped or hotlinked. There are several tools that when used on a regular basis, can help you reduce the amount of content thieves.

Review your web statistics.

All good hosting accounts have built in web statistics. In my opinion AWStats ranks highly. AWStats has a feature that displays “Links from an external page (other web sites except search engines)”

The display will show you the URL of the page linking to you as well as the number of hits to your page. It will also tell you how many page loads (of your pages) that URL initiates. For example if another site URL causes 10 hits on my site, then there should also be 10 page loads. If not something’s up. Take a look at the image below (clicking on the image will show you the large version).

AWStats hotlinker example

What you’re seeing in the edited image above, is that another site has displayed something (from my site) 1645 times but never actually sent a visitor to my site. In other words my content (an image in this case) was shown on another site, but no page visits (referals) were ever recorded coming from that site. I visited the URL in question, and sure enough, the site was hotlinking to one of my images.

Use online services.

One service I’ve experimented with is Copyscape. They are a site that provides a service which scans other web sites, providing you with the URL of copied content. In my case I found scraped content (stolen from my other blog) during the writing of this very post. That is… one of my original blog posts, was found via Copyscape to be duplicated verbatum on another web site in another country. They were using it to sell their advertising space and also had hot linked to all my images. While they did include a link to my original post and did list my URL as the “Original link”, they did so without permission and were using my complete work for their own profit.

A Google search will provide you with a plethora of other sites that provide such services, I’m only mentioning Copyscape as one good example. To further take advantage of such tools, its most advantageous to include a very unique sentence in any given post and search for it in Google (that’s often a very quick method to catch sites that copy your valuable content).

Once you’ve been able to isolate a specific URL that has hotlinked and/or scraped content, what can you do about it?

Here’s how the process should work:

1. Find and isolate a specific URL that is using your content.
2. Evaluate the need to take action. Are they still sending you traffic or referrals in some fashion? The bottom line question you could ask yourself is “Does their copied content really do my blog enough harm that I have no option but to follow through?” If you’ve answered yes, action is the next step.
3. Take action.

But wait! What actions are there? Can I really protect myself from a scraper or hotlinker in another country?

Before moving into they types of actions (tools) you can use to protect your content, it’s important to keep one salient point in mind:

If you have some content that is so important, private or valuable, etc. and you do not want other people to copy it, then DO NOT POST IT ON THE INTERNET. If you post something of excessive value on the Internet, no measure of copyright protection is going to prevent an individual from accessing it and copying it. But (there’s always a “but”), there are ways you can impede the profitability and earning power of copied content and in some cases injecting your own revenue generating systems into content copied from your site.

So what kinds of actions can we take?

Make sure you provide appropriate copyright notifications within every page of your blog. If you intend to permit your content to be shared, Creative Commons provides a great copyright tool wherein you can specify how your content is shared. You’ll find that tool at Creative Commons. You can select your jurisdiction in a drop down menu on the upper right side of the home page.

Is the copied content being served by Google Adsense? If so, you can issue an infringement notice at Digital Millennium Copyright Act – Google AdSense. I’ve found the best method is to fax the notice to the number they provide. In practice, I’ve found that it takes a few days for this to work through their system. However, they have always acted professionally, responsible and have indeed taken action.

As a side note: If you are looking for notification templates you can use, take a look at Copyright Law and SEO Part 3 (Sample DMCA Notifications, in HTML and MS Word Format), found on the McAnerin International web site.

Personally, I’ve experienced an excessive number of blogs copying content to Blogger.com based blogs. If you review their Blogger Content Policy, you’ll also find that notifications are to be sent to Google at Digital Millennium Copyright Act – Blogger. Again, I’ve found the response to be incredibly fast and Blogger.com is very quick (in my experience) to remove the violating content.

Many scrapers capture and repost your content by tapping into the RSS feed of your blog.

Side note: If you’re not very familiar with RSS feeds, Commoncraft has a great video, RSS in Plain English.

The fact that they are simply capturing and re-posting your RSS feeds may be indicative of an automated system (with little human intervention), where the scraper may not necessarily read your copied content. If that’s the case, you could try embedding a link back to your blog. There’s a great Wordpress plugin that will automatically do just that: RSS Footer. (One helpful thing is the RSS Link Tagger for Google Analytics, which helps with the tracking of non-adwords advertising campaigns. If you log into Google Analytics and go to your created campaign and view the traffic sources information you’ll garner more information).

Try earning a bit of revenue from your stolen feed content. If you’re a Google Adsense publisher, try the “AdSense for Feeds” option within your “Adsense Setup”. Additionally, the RSS Footer plugin above, I think can also be used to embed advertising content from any other affiliates or advertisers you subscribe to.

For those who are a little stronger at coding issues, you may want to try out a great little Wordpress plugin called “From RSS?“, where, as their site says: “…do something extra for your RSS subscribers, you might want to give them a little bit of extra content, or simply leave out some annoying footer about subscribing to the RSS feed. This plugin facilitates that…”

You can of course Report a Spam Result to Google which they use to help “…maintain the quality of Google search results.”

Finally another tactic that’s available is to report the site to the hosting company. There must be a ton of resources to help isolate who is the ultimate network provider (for the server that hosts the offending web site), of which I primarily use two:

• Whois lookup and Domain name search and
• Netcraft’s Search Web by Domain.

Using the domaintools.com site, I can see which DNS servers are managing the domain (ostensibly indicating who’s hosting it). And with Netcraft’s tools, I can determine who owns the IP address block that the website and server is one, thereby opening another venue of recourse.

In conclusion, I’d like to remind everyone of on salient issue. There may always be hotlinkers and scrapers of our content. I don’t think anyone can stop all of them, as doing so would in all likelihood prevent legitimate visitors from viewing your content, instead the primary objective of this post was simply to impart the concept of DETERENCE, so as to reduce such activities.