I think those are excellent questions.

What I was trying to point out with my post is that maybe we give too much credit to the search engines when pages are near duplicates as opposed to exact duplicates, in thinking that they might penalize them for duplicate content. The paper does explore different ways to identify duplicate content, but notes how difficult detection of near duplicates can be.

You probably know the difference between a penalty and a filter. A penalty would be when one site wouldn’t rank as high as it should because there was another site that duplicated its content. A filter is when a duplicate doesn’t appear in search results because a duplicate is showing instead.

It’s possible that if there was two sites, one with a .com, and another with a .co.uk, that if I searched in the US I might see the .com version, and if I searched in the UK, I might see the .co.uk version. But if their content was the same, I wouldn’t see both appearing in the search results. Neither has been penalized in that instance, but both have been filtered.

With booking.com and bookings.net, when I search for “booking” or “bookings” I’m seeing the .com version, and the other is being filtered.

As for the same content in different languages, I believe that both Vanessa Fox and Adam Lasnik of Google have both said that if the content is in different languages, then Google doesn’t perceive the sites as duplicates.

On my search for “bookings” I get the .com version first, and then I see the .nl version right after it. It isn’t being filtered or penalized, just as Vanessa and Adam suggested.

I’d be happy to discuss duplicate content in a new thread at the forums if you would like. I have more new material to discuss on the topic that I think is pretty interesting, including how search engines might treat sites that use templates.

Second, those sites you mentioned are different in far more ways than just having different top level domains. They are in different languages: although they essential substance may be different, they are significantly different content because of the language issue.

I’d suggest going into the forums to search for our past discussions on duplicate content or start a new one!

All there sites are indexed and are ranking high while having 100% exactly the same content.

So if this filter/penalty would be used than, this is one of the sites that for sure would be penalized.

Please give your comment