Since Google AdSense launched, there has been rumors and speculation about the possibility of the AdSense bot (officially known as the “Mediapartners-Google/2.1″ and unofficially as the “mediabot”) including some of its information into the regular Google search index. But no one has ever seemed to have concrete evidence of this happening. Some webmasters have maintained some discussions with Matt Cutts over the past few years about this issue, and they have always been assured that they are completely separate and they are always careful the two never cross contaminate each other.

But on SEO Rockstars this week, Greg Boser (aka WebGuerrilla) mentioned that he had seen mediabot information showing up in the natural search index, and my ears perked up. The end result was a whole bunch of duplicate content due to the fact that we were serving the AdSense bot the old url, and Googlebot the new one. The first one is taken from this site (JenSense.com). The following two are from a site URL I cannot reveal, but I included them to illustrate the problem is across multiple sites and covering multiple date ranges (JenSense is indexed regularly and there were no cache dates back that far).

First off, the AdSense support site clearly states that the two bots serve complete different purposes and should not affect the other.

“While our bot (starting with ‘Mediapartners-Google’) does crawl content pages for the purpose of targeting ads, this crawl is not associated with our main index crawl.”

There is the possibility that there was an accidental cross over taking place if the AdSense team was keeping cached copies of the pages serving AdSense for quality checking purposes, such as checking to see if a publisher is serving the mediabot something different than what Joe Surfer sees when visiting the page.

I could not find any evidence of multiple sites I checked that were not already indexed getting any sort of indexing boost via the mediabot. Webmasters usually wouldn’t think to include the mediabot in any special headers or robots.txt instructions they have for the regular googlebot.

You can read the whole article at Jense.com