adding nofollow on "unimportant pages"

greggles's picture
public
greggles - Fri, 2007-03-30 18:11

At drupalcon I had some discussions with chad jones about adding a rel=nofollow attribute to certain links in Drupal. Specifically we discussed pages like the user login page, the feed links, and other similar links which are on most every page of every drupal site but which are not "important" pages where you want to send your "link juice".

We talked about several possible solutions to this. I brought up that it could be done at the theme level or in a module on a per site basis. Chad seemed to prefer the idea of having it in core.

I wanted to bring that up as a point of discussion back here in this group and also point to this article from search engine journal (and the webmasterworld article it links to) that discusses whether or not this strategy is a good thing in general.

So - thoughts?


-1

Christoph C. Cemper@drupal.org's picture
Christoph C. Ce... - Fri, 2007-03-30 20:24

-1

I fully agree with the author of the cited article

Here is my opinion on using the NoFollow for linking to such internal pages; doing so is idiotic.

100% ACK

NoFollow was invented to neutralize trackback spam and the sites linked in it

if you link to your own site and say "hey- this contact page is not important, it maybe spam"
then you really got something wrong

Chad's opinion comes from 2002-SEO thinking that you can "horde" the linkjuice of pages... AKA PAGERANK

Forget it folks...

Christoph C. Cemper
- http://www.cemper.com
- http://www.marketingfan.com


good to hear your opinion

greggles's picture
greggles - Fri, 2007-03-30 23:28

Seems like we are likely to have some good debate on this topic, then ;)

--
Knaddisons Denver Life | mmm Free Range Burritos


It's all about about clarity...

chadj@drupal.org's picture
chadj@drupal.org - Sat, 2007-03-31 16:39

The idea is not about hording pagerank, It's about clarity -- providing a clean presentation of only content to the search engines. Although this is not as much a problem with Drupal, other content systems (vBulletin is a good example) present hundreds of administrative links on every page which end up sending the search engine spiders into endless loops of cruft.

I've hacked vBulletin to add rel=nofollow to all non-content links and it really works great. The result is that my topics show up in the search engines perfectly without duplication or confusion. This avoids the awkward approach taken by so many forum administrators of creating a duplicate "archived" set of content more easily readable by the spiders.

With rel=nofollow, a site can present to the spider a very clean set of content. The webmaster decides what is and is not content. This is similar to using robots.txt but with some significant differences in effect. Rel=nofollow simply means "don't follow" which is exactly the intent here. The idea that it means "don't trust" is pure speculation without any evidence I've ever seen to back it up.

Rel=nofollow abuse is always in relation to external links. I would venture to assert that rel=nofollow links INTERNALLY are highly reliable because a webmaster will not intentionally block internal pages unless they really don't contain useful content. Whether or not a wembaster should nofollow links to a "contact us" page is another issue. And I agree that this is not a good idea.

Besides, if it were truly possible to harm a page by linking to it with rel=nofollow, this would constitute a powerful attack vector against your competition. Google would never allow such an obvious security hole in their algorithm.

ChadJ

Ecommerce SEO Checklist
Free Website Monitor


robots.txt

JohnForsythe's picture
JohnForsythe - Mon, 2007-04-02 04:37

This is what robots.txt is for. Nofollow won't do much if the URLs are already indexed, and it won't stop Googlebot from visiting them even if they're not indexed. It's a misnomer, really. It should be called nopagerank.

Aside from that, nofollowing your feeds is generally a waste of time. They will end up linked from aggregators eventually, and Google will assign PR accordingly. In fact, I'd go so far as to say it could hurt you to nofollow them.

--
John Forsythe
Need reliable Drupal hosting?


"Nopagerank" please... ;)

chadj@drupal.org's picture
chadj@drupal.org - Mon, 2007-04-02 06:39

Hi John,

"Nopagerank" -- that's a good one. And I agree about not "nofollowing" feeds.

BTW, I did not mean to suggest that using nofollow should be forced on people. I've personally experimented with it and had great success in certain areas. There's just a lot of passionate conjecture out there in the rumor mill.

Many people mistakenly think that robots.txt and nofollow have the same effect -- they do not. Nobody really knows how Google calculates nofollow links (ie. does nofollowing 99 out of 100 links on a page result in the one remaining link having more PR vote behind it?) My one big nofollow experiment seems to indicate that it does have roughly this effect.

We do know that robots.txt blocks pages from the index but does not stop pagerank from being passed on -- to now unreachable pages. If link authority can be intentionally focused on useful content, robots.txt is not the way.

SEO folk understand the importance of quality inbound links -- but it seems that the art of internal linking structure gets neglected. If the search game were baseball, inbound links would be the pitcher and the site's link structure would be the batter. After all, internal linking is essentially the game of controlling and directing incoming link authority. I think a big part of SEO is figuring out how to organize linking to please both human and search engine visitors.

It seems to me that one of Drupal's great strengths is a sparse default linking structure with very little waste of link authority. This is reflected in effective search results even without fancy "noindex" / "nopagerank" trickery.

ChadJ

Keyword Marketing Ladders
Slam Spam


why is it idiotic?

rlnorthcutt@drupal.org's picture
rlnorthcutt@dru... - Fri, 2007-05-04 07:52

See http://www.searchenginejournal.com/how-google-yahoo-askcom-treat-the-no-...

Google is leading the search engines (near 50% of all traffic), so optimizing your site to them is a good idea. According to Google (see the article), they not only like the use of "nofollow", they encourage it!

Irregardless of the original intent of nofollow (fighting comment spam), the fact remains that this is simply another mechanism that dictates the bot behavior. With proper (white hat) useage, the structure of the site can be optimized for indexing by spiders while still being easily navigable by humans.

Of course, I can see how improper useage of this tool could actually hurt your rankings and indexing, but any tool can cause collateral damage if its not properly used or understood.

Ultimately - whether or not you care to use this technique on your site (or your clients :) is a personal choice. Every site is different and there is no "one way." That being said, there is obviously a call from the community for this ability, and the community is what its all about.

Please note that I don't necessarily think its the best solution to spamming or even to optimization - but if its out there and being used, then those who are interested should be able to take advantage of it.


threading...

greggles's picture
greggles - Fri, 2007-05-04 12:24

So, you replied to me rather than Cristophe. Cristophe said it was idiotic - not me. Just for the record. I think it's a good idea but...

..as, you said: there is no "one way." which I agree with completely. That's why I think it has to be done per site which basically means at the theme layer. Unless there are some links that we can say "this should be nofollowed 100% of the time" then the rest...don't bother.

You also said That being said, there is obviously a call from the community for this ability, and the community is what its all about. can you expand on what you mean? I'm not clear, so I don't want to assume anything.

Thanks for your thoughts.

--
Knaddisons Denver Life | mmm Free Range Burritos


Nofollow is useless

Funana's picture
Funana - Fri, 2007-05-04 15:16

Hello everybody,

my 2 cents: Nofollow is useless. It once was invented to stop commentspam and it didnt work to do this. Mainly because spammers dont care if the links will not get "linkjuice" because they want humans to click on their links. Even in wikipedia it's a big joke. There are still dozends of wikipedia clones who dont contain nofollow and if you have a link in wp it will be cloned too. Plus, if you have a link in wp it drives real human traffic to your page no matter how google handles these kind of links.

I still prefer a robots.txt solution for excluding pages on my sites from being indexed. And I don't think that anyone should use nofollow.

Have a nice day!

__

SEO Tips For Successful Drupal Sites
__
Cape Verde News & Community
My Info Collection


Nofollow

jmiami's picture
jmiami - Sun, 2007-06-10 00:27

Nofollow does not mean "do not follow" — it means "do not vouch". It should not be used on internal links. Robots.txt is for blocking robots, not rel=nofollow.

More about how to use rel=nofollow here:
http://microformats.org/wiki/rel-nofollow
http://googleblog.blogspot.com/2005/01/preventing-comment-spam.html

More about how robots do follow nofollow:
http://en.wikipedia.org/wiki/Nofollow

Another reason that you should use robots.txt vs. rel=nofollow is that someone could just deep-link to your nofollowed page and it will still get indexed.

The standard way to restrict search engine crawlers is with robots.txt. There are very few situations (if any) where rel=nofollow should be applied to internal links.


see chad's comments

greggles's picture
greggles - Sun, 2007-06-10 14:19

We do know that robots.txt blocks pages from the index but does not stop pagerank from being passed on -- to now unreachable pages. If link authority can be intentionally focused on useful content, robots.txt is not the way.

Did you see those prior to commenting?

--
Knaddisons Denver Life | mmm Free Range Burritos


robots vs rel=nofollow

J. Cohen's picture
J. Cohen - Fri, 2007-08-03 15:16

Many people mistakenly think that robots.txt and nofollow have the same effect -- they do not. Nobody really knows how Google calculates nofollow links (ie. does nofollowing 99 out of 100 links on a page result in the one remaining link having more PR vote behind it?)

I agree with this:
* robots.txt and rel=nofollow do not have the same effect
* no one knows how Google calculates nofollow links

My take on it:
There is more than one search engine -- we just have the specifications to go on: Robots.txt was created to block robots from sections of your site; Rel=nofollow was created to say "I don't vouch for that link".

Why not take the next step and just add rel=nofollow everywhere? For example, to all URLs that point to:
example.com/forward/*
example.com/comment/reply*
example.com/image-name?size=thumbnail
example.com/forum/8?sort=asc&order=Last+reply
etc.

I don't think that search engines work that way -- and even if some of them do, they will eventually get fixed. It's easy to identify an over-SEO'd site by the massive numbers of internal nofollowed links.

I would avoid internal nofollow links. I don't know the answer to the issue about PR being passed to pages blocked with robots.txt, but I wouldn't worry about it. We don't have complete information on the algorithm. In practice you can rank large, complex sites for competitive keywords without using any internal nofollow.


now somewhat possible

greggles's picture
greggles - Wed, 2007-07-11 14:37

This is now somewhat possible for links that are sent through the filter mechanism. You can filter to either add nofollow or remove it based upon the patch in http://drupal.org/node/113851 which builds on the existing nofollowlist module http://drupal.org/project/nofollowlist

--
Knaddisons Denver Life | mmm Chipotle Log | The Big Spanish Tour


How To Get More Pages Indexed With Nofollow

LeonidShamis - Wed, 2007-09-05 05:13

Hi,

I've just come across this article about the subject of "nofollow" - How To Get More Pages Indexed With Nofollow.

I was happy to find out that many Drupal's contributed modules support "rel=nofollow". However, some clarity and guidelines on building the right links architecture would really be great.

Remember: there is a

Alexander Pas - Sun, 2007-11-18 17:42

Remember: there is a difference between nofollow and noindex!

So how does one add nofollow at the theme level?

wickwood - Tue, 2008-02-19 05:26

Like everyone else here, I have an opinion, and I want to use the nofollow to give link structure to push the Page Rank through the site. This link structure for Search Engines is, by necessity, different than the link structure useful for human navigation. Humans need lots of links to get around the site easily, the Search Engines actually like to be led through your site.

For those of you who don't want to use this strategy, then don't use it. But I'm a believer that it is a smart thing to do, and it's not something you can do my robots.txt noindex, because I do want the pages indexed, I just want to control how the Search Engine perceives the pages being linked together.

But I'm fairly new to Drupal, and I have yet to find any description that actually tells me how I can add nofollow to certain links in my site. I would like to create tiers within my site to push the page rank and optimize for certain keywords.

For example, the link structure for the Search Engines would be:

  • Home Page (Tier 1) links to Category Pages Tier 2, but all other links are nofollowed, and links between category pages are nofollowed.
  • Tier 2 Category Pages links to Tier 3 Content Pages. Again all other links are nofollowed, and links between content pages are nofollowed.
  • Tier 3 Content Pages link sback to Tier 1 Home Page (with keywords I'm optimizing for), and again all other links are nofollowed.

To see how different link structures controlled by nofollow, can push PR around the sites pages checkout this calculator:
http://www.webworkshop.net/pagerank_calculator.php

Another way of accomplishing this, and what was done before nofollow was invented, is with javascipt links that SE-bots can't follow.

So like I said, this is something I want to do, there is no question in mind about that. I just want to find a resource that will tell me how to do it in Drupal.

Thanks in advance!
Steve

I'm still trying to figure this out...

wickwood - Tue, 2008-09-23 03:49

If anyone would give me some direction on how exactly this can be done at the theme level, I would GREATLY appreciate it!!

Thanks in advance!

Steve

Pr sculpting is a very

litelus's picture
litelus - Thu, 2008-11-13 16:06

Pr sculpting is a very usefull tool, as long as it doesn't take you too much to implement it. That is not the case if it came in the form of a drupal module. A module that ads nofollow to pages that have no need for pr (eg: login) would be of much use (either as a core module or as a regular module).