All articles

Published: Last updated:

TL;DR

Check the public URL, incoming links, crawl access and indexing instructions separately. Keep sitemap and canonical signals consistent; neither guarantees indexing.

Website pages organised in a search-index drawer with a validation check

A published page is not automatically a page Google will show in search. Search engines need to discover its address, access its content and decide whether to include it in their index. Even after indexing, the page may not appear for the searches you expect.

When a useful service page seems missing, inspect that specific page before changing the whole website. The following checks help distinguish a discovery problem from an access problem or a decision about the content itself. If the missing result is a Maps listing, start with the business visibility diagnosis instead.

Begin with the address a customer would use

Open the page without being signed into your website. Does it load successfully at the intended public address? Does it unexpectedly redirect, show a login screen or return an error? A page visible in your editor may still be unavailable to visitors.

For example, a plumber may create a new boiler-servicing page but leave it in draft mode. Adding its address to a spreadsheet or telling customers about it will not make the unpublished content accessible. First check the public page and its publication settings.

Follow links to the page

Find a sensible route from the homepage or a related service page. Important pages should not depend on someone guessing the URL. Descriptive links help visitors understand where they are going and help crawlers discover content. The internal linking guide shows how to connect useful pages without crowding navigation.

Count the links needed only as a practical navigation check. There is no universal maximum click depth that guarantees indexing or rankings. A clear service hierarchy can be more useful than putting every page in a crowded menu. Ask whether the route makes sense to a customer looking for that particular service.

Check crawling and indexing controls separately

The robots.txt file gives supported crawlers rules about which URLs they may access. It is not a security barrier. Google’s robots guidance explains that a blocked page’s address can still appear in search if discovered elsewhere.

An indexing exclusion, such as noindex, has a different purpose: keeping a page out of search results. Google needs to be able to crawl the page to read that instruction. Blocking the same page in robots.txt can prevent it from seeing the exclusion. Ask your developer to check the combination rather than adding more rules at random.

Private customer information needs proper access controls. Neither a robots rule nor noindex prevents someone with the address from opening an otherwise public page.

Compare the sitemap with the preferred page address

A sitemap lists URLs you want search engines to discover. Check that an important public page is included where your website’s sitemap is intended to cover it. Look for old domains, staging URLs and retired pages that no longer belong there.

Google’s sitemap documentation makes clear that submission does not guarantee crawling or indexing. The sitemap also does not replace useful navigation for customers.

Some content is available at several addresses. A canonical annotation indicates which equivalent or very similar version you prefer. Google treats this as a signal rather than a binding instruction. Its canonical guidance recommends consistent signals, so internal links, sitemap entries and canonical annotations should not point to conflicting versions.

Use Search Console to investigate the actual page

Inspect the intended URL and read the reported reason if it is excluded. Check whether Google selected a different canonical version, has not discovered it or could not retrieve it. Record the findings before requesting changes. Reports can reflect an earlier crawl, so compare them with the current page and the timing of any recent edits.

If access and discovery look sound, review whether the page provides useful, distinct information. Do not duplicate it at a new URL simply to get another submission attempt.

If you need technical SEO support, get in touch with the affected URLs and the Search Console messages. Specific evidence makes it easier to identify the underlying rule or template causing the problem.

Need help applying this to your site?

If something in this article rings true for your business, let's have a quick conversation about it.

Start a conversation