Duplicate content is a typical occurrence on the internet and in numerous cases can hurt search engine positions. While the search engines could not always technically penalise webmasters for reused content, there remain a lot of ways it can hurt.

Duplicate content in Google
The way Google handles reused content has been debated a lot in latest memory. This is essentially because of a video Google’s Greg Grothaus uploaded, in which he discusses at length, the way Google handles a selection of different components of the Duplicate Content conversation.
Joachim Kupke, Sr. Software Engineer of Google’s Indexing Team repeated much of what Grothaus declared. He said that Google has plenty of infrastructure for content duplication elimination :
- redirects
- detection of recurring URL patterns ( the facility to ‘learn’ recurring url patterns to find copied content )
- precise contents – most lately crawled version
- earlier content
- contents minus things that don’t change on a site Kupke recounted to avoid dynamic URLs when possible ( though Google is “rather good” at dumping fools ). If all else fails, use the canonical link part.

Kupke calls this a “Swiss armed forces Knife” for copied content issues.

Google asserts the canonical link component has been enormously successful. It did not even exist last year, and is has grown incredibly. It has had a big impact on Google’s canonicalization choices, and two out of three times, the canonical tag really changes the organic call in Google. Google claims a standard mistake is designating a 404 as canonical, and this is generally due to nonessential relative links. So, avoid changing rel=”canonical” designations, and avoid designating permanent redirects as canonical. Also, don’t refuse directives in robots.txt to annotate copy content. It makes it harder to see fools, and disallowed 404s are a bother.

There’s an exception and that’s that interstitial login pages may be a good applicant to “robot out,” according to Kupke. Kupke claims that canonical works, but indexing needs time. “Be patient and we’re going to use your appointed canonicals.” Cleaning up an existing part of the index takes even longer, and this may leave fools serving for some time notwithstanding rel=canonical, Kupke adds. At SMX, Google announced cross domain rel=canonical is coming inside this year. So as an example, if the Chicago Tribune has an article on the Times, and the rel=canonical points to the Chicago Tribune then Google will only credit the Chicago Tribune with the content.