Missing Robot Directives

Technical

Notice

This error appears when a page has no meta robots directives defined. Here's what it means and how to specify them correctly.

What this error means

Meta robots directives are instructions that tell Google and other search engines how to treat a page. They can be defined through a meta tag in the HTML or via an HTTP header called X-Robots-Tag. When neither is present, the page has no explicit instructions. This doesn't mean the page will disappear from Google. Search engines have a default behavior when they don't find directives, which is normally to index the page and follow its links. But relying on default behavior isn't the best approach. Having meta robots directives explicitly defined gives you control over which pages get indexed, which don't, and how their links are crawled. Without them, you're leaving those decisions in Google's hands. This error is especially relevant on pages that shouldn't be indexed, such as admin pages, internal search results, or duplicate content. If nothing is specified, Google may decide to index them anyway.

Why defining meta robots directives matters

Not having meta robots directives isn't a disaster, but it is a control opportunity you're letting slip by. The more clearly you communicate with Google, the better it understands your site. Defining them is quick, and it's reassuring to know that each page behaves exactly as you want.

SEO ranking impact

Without explicit directives, Google decides on its own what to index and what not to. This can cause pages that shouldn't appear in results to end up indexed, competing with the ones that actually matter.

User experience consequences

If Google indexes pages that aren't meant for end users, such as admin pages or duplicate content, they can show up in search results and create confusion or a poor first impression.

Crawling and indexing obstacles

Without clear instructions, Google may spend crawl time on pages that don't add value. That reduces the efficiency with which your site's truly important content gets crawled and indexed.

How to fix it step by step

If this error showed up in your audit, here are the steps to leave it behind.

Step 1

Decide how you want Google to treat each page

Before adding anything, be clear about what Google should do with that page. Should it index it and follow its links? Should it not index it? Should it not follow its links? The directive depends on the answer.

Step 2

Add the meta robots tag in the HTML

For most public pages, add <meta name="robots" content="index, follow"> inside the <head>. If the page shouldn't be indexed, use noindex. If its links shouldn't be followed, use nofollow. You can do this through the SEO plugin you use (Yoast, Rank Math, etc.) or directly in the code.

Step 3

Verify the directive has been added correctly

Once the change is published, open the page source and look for <meta name="robots"> inside the <head>. Check that it appears and contains the right values for that page.

Does your site have this problem?

Audit your website for free and discover if this and other SEO errors are affecting your ranking.

See all the errors we detect