sitemap and clean urls

Posted by osviweb on October 28, 2007 at 10:14pm

Hallo
this is my first post, I hope I do it properly.

I'm tring to optimize drupal for seo. I'm using clean urls with pathauto and semplified urls. To avoid duplicated content between clean urls and nodes I've disabled the spiders to visit the nodes pages by editing the robots.txt file
with
Disallow: /agenzietui/node$

Now the problem is that the sitemap module is generating a sitemap which includes all the nodes pages and not the semplified urls pages.

Submitting this sitemap to a search engine but telling to it not to visit the nodes$ pages I think I'm going in a controversy and doing wrong.

I've tried to allow taxonomy in the sitemap module but I still see the nodes links in it.

What is the best solution to avoid the duplicated content and have the sitemap with clean urls list and not nodes pages?

thanks

Comments

This Sitemap Module appears

Posted by Z2222 on October 28, 2007 at 11:17pm

This Sitemap Module appears broken.

You don't need to submit XML sitemaps to search engines. It doesn't help with your rankings and as long as your internal linking is good, search engines will index your pages.

Try the Global Redirect Module

Posted by Ben Finklea on October 29, 2007 at 2:41am

I think the Global Redirect Module will solve your problems.

I did a Podcast on it a little while ago: SpryDev SEO Podcast Episode 3 - The Global Redirect Module

Cheers!

Ben Finklea, CEO
SpryDev Search Engine Marketing home of the Drupal SEO Podcast
We guarantee web profits.
512-989-2945 x204
mobile: 512-632-4222
f: 512-857-0212
ben@sprydev.com

Ben Finklea
Chair, Drupal Branding and Marketing* Committee

*BAM!

Redirect warnings

Posted by Michelle on October 29, 2007 at 11:55am

Just don't use global redirect with xmlsitemap. I just got on webaster tools and had a big red warning about all the redirects in my sitemap because of the xmlsitemap module putting node/XX paths in. There's an outstanding issue on the module but the author doesn't seem very interested so far. I removed the module from my site. Google claims the redirects in there won't actually hurt you but I'm not taking any chances.

Michelle

See my Drupal articles and tutorials or come check out life in the Coulee Region.

XML Sitemap would be making sense

Posted by s.Daniel on October 29, 2007 at 12:36pm

But as you said it issn't function propperly. The global redirect module won't help much as when using it you'd still point SE to wrong/outdated adresses in the XML file. G**gle for example will give you a warning in Webmaster-Tools telling that there is an error with your sitemap. Something like "too many redirects".

Anyhow we can't say if that would delay the crawling process or hurt your rankings. Probably not. On the other hand - people still discuss weather xml sitemaps help at all to increase crawls speed / rankings.

I personally won't use the module for new sites untill this is fixed.

Sebastian

Drupal XML Sitemaps

Posted by Z2222 on October 29, 2007 at 6:43pm

On the other hand - people still discuss weather xml sitemaps help at all to increase crawls speed / rankings.

I don't think that XML sitemaps provide significant benefit.

My thought is that Google created them to tune/debug their own system. They can gather and analyze a lot of data and then say, "webmaster says they have x pages; googlebot says y". Then the engineers can go in and look at specific examples and make adjustments to the crawlers/indexers.

It won't hurt your site to remove the XML sitemap. It doesn't appear to hurt a Web site's rankings to put in a totally incorrect sitemap -- though I wouldn't recommend it.

The point of sitemaps was never to increase rankings; only to make sure that the entire site is indexed. If your internal linking structure is good, and you have enough inbound links, search engines are going to index your site anyway. The <priority> element just shows your desired relative priority between your pages, but I think that internal/external linking is still a bigger factor.

It's best to just avoid the XML Sitemap module because it's broken...

Thanks

Posted by Michelle on October 29, 2007 at 7:14pm

I'm just learning all this SEO stuff and thought submitting a sitemap to Google would help. I never even looked at the site map it was sending until I happened across an issue about image nodes not using the alias. It was then that I looked at mine and realized 90% of it wasn't using the alias. No clue why the other 10% was working. So I posted on that issue that I was having the same problem with all my node types. The maintainer then closed it with no answer because it was filed for the dev release. I pointed out that I wasn't using the dev release and having the same problem. At that point, it got marked duplicate of another issue which, when I went and looked, was marked fixed! It's obviously not fixed but I give up. Good to know I don't actually need this module. I'll just delete it and move on. :)

Michelle

See my Drupal articles and tutorials or come check out life in the Coulee Region.

xml sitemap is important for ranking and usability

Posted by osviweb on October 29, 2007 at 9:58pm

in my opinion the sitemap is important for internal link matrix and pr distribution. Is not the must for optimization but is a green flag.
A global redirect would be another confusion and too many redirects...
I think there is a patch for the xml sitemap here http://drupal.org/node/143994 on post 54 but I'm not sure how to install it.
I thank you for confirming this problem and giving ideas and turnarounds.

New Version!?

Posted by s.Daniel on November 11, 2007 at 4:27pm

A new version cam out yesterday http://drupal.org/project/xmlsitemap
Did anyone try weather it fixes this issue?