Duplicate content is a typical occurrence on the internet and in numerous cases can hurt search engine positions. While the search engines could not always technically penalise webmasters for reused content, there remain a lot of ways it can hurt.

Duplicate content in Google
The way Google handles reused content has been debated a lot in latest memory. This is essentially because of a video Google’s Greg Grothaus uploaded, in which he discusses at length, the way Google handles a selection of different components of the Duplicate Content conversation.
Joachim Kupke, Sr. Software Engineer of Google’s Indexing Team repeated much of what Grothaus declared. He said that Google has plenty of infrastructure for content duplication elimination :
- redirects
- detection of recurring URL patterns ( the facility to ‘learn’ recurring url patterns to find copied content )
- precise contents – most lately crawled version
- earlier content
- contents minus things that don’t change on a site Kupke recounted to avoid dynamic URLs when possible ( though Google is “rather good” at dumping fools ). If all else fails, use the canonical link part.

Kupke calls this a “Swiss armed forces Knife” for copied content issues.

Google asserts the canonical link component has been enormously successful. It did not even exist last year, and is has grown incredibly. It has had a big impact on Google’s canonicalization choices, and two out of three times, the canonical tag really changes the organic call in Google. Google claims a standard mistake is designating a 404 as canonical, and this is generally due to nonessential relative links. So, avoid changing rel=”canonical” designations, and avoid designating permanent redirects as canonical. Also, don’t refuse directives in robots.txt to annotate copy content. It makes it harder to see fools, and disallowed 404s are a bother.

There’s an exception and that’s that interstitial login pages may be a good applicant to “robot out,” according to Kupke. Kupke claims that canonical works, but indexing needs time. “Be patient and we’re going to use your appointed canonicals.” Cleaning up an existing part of the index takes even longer, and this may leave fools serving for some time notwithstanding rel=canonical, Kupke adds. At SMX, Google announced cross domain rel=canonical is coming inside this year. So as an example, if the Chicago Tribune has an article on the Times, and the rel=canonical points to the Chicago Tribune then Google will only credit the Chicago Tribune with the content.

Google and Microsoft have both agreed deals with Twitter and Microsoft has also penned one of its own with Facebook to integrate Twitter and Facebook updates into Bing search engine results. Google will also be adding tweets to search results.

Google’s Marissa Mayer said, ” We believe that our search results and user experience will greatly benefit from the inclusion of this up-to-the-minute data, and we look forward to having a product that showcases how tweets can make search better in the coming months. That way, the next time you search for something that can be aided by a real-time observation, say, snow conditions at your favorite ski resort, you’ll find tweets from other users who are there and sharing the latest and greatest information.”

There is every possibility that Google will be making a deal like this with Facebook, but even if that doesn’t come off, their deal with Twitter and Bing’s two deals for sure make it all the more important for marketers to be found in real-time searches and Facebook/Twitter in general.

That said, we don’t know all the details about how Google and Bing will be integrating its Twitter and Facebook results into the rest of their results yet. Bing has made available a beta tool for people to mess around with for searching tweets with the search engine. “You can now search for what people are saying all over the web about breaking news topics, your favorite celebrity, hometown sports team, and anything else you use Twitter to stay on top of today,” says Paul Yiu of Bing’s Social Search team.