Robots META tag

The Robots META tag can be used to control how robots function when they crawl specific pages. The most common reason for adding this tag would be to prevent robots indexing and/or following any links on the page.

Example:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

The following values can be added the CONTENT attribute:

• INDEX
• NOINDEX
• FOLLOW
• NOFOLLOW

Multiple comma-separated values are allowed and the default is "INDEX, FOLLOW" (if no Robots META tag is supplied).

Remaining combinations:

<META NAME="ROBOTS" CONTENT="INDEX, FOLLOW">
 
<META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW">
 
<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">