Duplicate URLs are being generated

My google webmaster tools is showing many node URLs on the site being gotten duplicate. Actually I used Quick Tabs module almost a month ago and it created duplicate URLs such as

http://example.com/content/abc-def-ghi?quicktabs_1=0
http://example.com/content/abc-def-ghi?quicktabs_2=0
http://example.com/content/abc-def-ghi?quicktabs_8=2, etc

for the actual URL

http://example.com/content/abc-def-ghi.

Actually I wasn't aware of this nature of Quick Tabs module and started using it thus creating lots of tabbed blocks. When I saw it generating duplicate URLs in Google, I immediately stopped using it and uninstalled it as well. It has been more than 10 days now I have uninstalled it.

But the probem is Drupal is still generating duplicate URLs in the similar fashion as mentioned above for the new nodes being created. This has affected my sites traffic very adversely. I have already added a nofollow in the robots.txt file but it seems to be of no use.

Please help.

Login to post comments

Up to 4 weeks before it's in google

FlemmingLeer - Mon, 2009-09-21 15:17

Hi Nik,

What an unsuspected behavior. :/

Google will still have the links 3-4 weeks in their database from the date of your change to robots.txt

Did you test the new disallow rule inside google webmaster tools in:
google.com/webmasters/tools/crawl-access ?

Even a turtle reaches it´s goal...


What does your robots.txt

J. Cohen's picture
J. Cohen - Mon, 2009-10-12 18:30

What does your robots.txt file look like?

This should work:

User-agent: *
Disallow: /*?quicktabs

See this for general Drupal/robots.txt tips.

--
» Twitter » Blog » Website


Canonical URL

sk33lz - Tue, 2009-11-10 20:22

For all who don't know, there is a new tag out there on the SEO world, and has been adopted by all 3 major search engines similar to that of XML sitemaps. It maybe a better solution in this case, as I have actually been using it in practice already for this very reason.

The tag is called a Canonical Link and it specifies what URL you want the search engines to recognize as the content of a given page. This content must be similar, and while using Quicktabs, this might just be a better solution than disallowing the content all together through robots.txt. In some cases, you may want Google to index the content and canonically display what they feel is best using the new tag.

There was a specific module I was using to handle this tag in drupal, Canonical URL, found here:
http://drupal.org/project/canonical_url

The project has recently been deprecated as Nodewords 6.x-1.2+ handles Canonical URLs. You can get Nodewords here:
http://drupal.org/project/nodewords