Google not indexing your new pages: find the cause and fix it
From the work/ 15 min read
Why Google is not indexing a new page: sitemap, URL Inspection, robots.txt versus noindex, canonical, internal links, and what Google says about timing.
- SEO
In this article (11)
When Google is not indexing a new page, check in this order: whether it found the page through the sitemap or links, whether robots.txt lets it crawl, whether there is a noindex, whether the canonical points to another URL and whether it returns a 200. The URL Inspection tool in Search Console tells you which step it stopped at.
Below is that diagnosis step by step with what Google Search Central says, what each Page indexing status means, and a Node script that checks a URL's robots.txt, noindex, canonical and sitemap. We ran it against a local test site and against public pages on ecomlabs.dev.
The sitemap and Search Console
Google Search works in three stages: crawling, indexing and serving results. Per its guide to how Search works, not every page makes it through each stage, and Google does not guarantee that it will crawl, index or serve your page. Before it crawls a page, it has to know the page exists. It discovers URLs in two ways: it extracts links from pages it already knows, or it reads a list you send it, the sitemap.
A minimal sitemap with one new page looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://example.com/blog/new</loc>
<lastmod>2026-11-18</lastmod>
</url>
</urlset>
- The first line declares the XML and its encoding.
urlsetopens the list with the sitemap protocol namespace.- Each
urlis one page. locholds the full URL, with protocol and host: Google asks for absolute URLs, not relative paths.lastmodis the date of the last change. Google uses it only if it is consistently and verifiably accurate. It ignorespriorityandchangefreq, so you can leave them out.
Per the sitemap documentation, one file holds up to 50MB uncompressed or 50,000 URLs; beyond that, split it and list the parts in a sitemap index. You submit it in the Sitemaps report in Search Console, through the Search Console API, or with a Sitemap: line in robots.txt. Google calls it a hint: submitting it does not guarantee that Google downloads it or crawls its URLs. It is still the recommended route when you have many new pages, and the report shows when Google read it and whether it hit errors.
One useful detail from the Page indexing report help: Google says that if your site has fewer than 500 pages, you probably don't need that report and can just check your key pages. On a small site, start with the specific URL that is missing.
Scroll the figure sideways to see all of it
Inspect the URL
Paste the full URL into the search bar at the top of Search Console; it has to belong to the property you have open. The URL Inspection tool shows Google's indexed version, not the live page. The verdict at the top reads "URL is on Google" or "URL is not on Google". Careful: "on Google" does not guarantee that the page shows up in results.
Below it, in the page indexing section, read the fields in the order the page travels:
- Sitemaps and Referring page: how Google found it. If both are empty, Google has no clear path to it. The help notes that the tool sometimes shows no referring page even when one exists.
- Last crawl: if it is empty, Google has not visited yet.
- Crawl allowed?: if it says no, a robots.txt rule blocks it.
- Page fetch: whether Google could get the page from your server. It can succeed while the page stays out of the index for another reason.
- Indexing allowed?: if the page disallows indexing, the reason shows here, for example a noindex.
- User-declared canonical and Google-selected canonical: if they differ, Google indexes the other URL.
Scroll the figure sideways to see all of it
The Test live URL button runs the check in real time. Use it to confirm a fix, but know its limits: it only confirms that Google-InspectionTool can access the page, it does not test duplicates or canonicals, and it does not tell you the page will be indexed. There is also a daily quota of live inspections per property.
What each Page indexing status means
If the page shows as not indexed in the Page indexing report, the reason has a name. These are the ones new pages hit most, summarized from the Search Console help:
| Status | What Google says | What you do |
|---|---|---|
| Discovered - currently not indexed | Found but not crawled yet: Google rescheduled the crawl because it expected to overload the site. That is why the last crawl date is empty. | Make sure it is in the sitemap and linked from indexed pages, then wait. |
| Crawled - currently not indexed | Crawled but not indexed. Google does not index every page; it is common and does not necessarily mean a problem. | If the page matters, inspect it and rule out blocks and duplicates. |
| URL blocked by robots.txt | A robots.txt rule stops the crawl. | Remove or narrow the rule. |
| URL marked 'noindex' | Google found a noindex directive when it tried to index the page. | Remove the tag or the header. |
| Duplicate, Google chose different canonical than user | You marked the page as canonical, but Google thinks another URL makes a better canonical. | Compare both pages and review your canonical signals. |
| Page with redirect | A non-canonical URL that redirects; it is not indexed, the target may be. | Link to and list the final URL in the sitemap. |
| Not found (404), Soft 404, Server error (5xx) | The response was not a valid page. | Make the URL return a 200 with content. |
"Crawled - currently not indexed" causes the most confusion: Google came, found nothing blocking it, and still chose not to index the page for now. Google asks you to use URL Inspection to rule out problems on your site, and keeps a form to report an indexing issue only for a complete, site-wide problem.
Scroll the figure sideways to see all of it
robots.txt, noindex and canonical
These three signals get mixed up and do different jobs. Per the introduction to robots.txt, the file is mainly there to avoid overloading your site with requests, and it is not a mechanism for keeping a page out of Google: a blocked URL can still appear in results, without a description. To keep a page out, use noindex or password protection.
The classic launch problem is a robots.txt carried over from staging. Here are two versions of the same file, before and after:
# Before: copied from staging, blocks the whole site
User-agent: *
Disallow: /
# After: what production should serve
User-agent: *
Disallow: /admin
Sitemap: https://example.com/sitemap.xml
User-agent: *opens a group of rules for any crawler that has no group of its own.Disallow: /matches every path: Google cannot crawl any page, new or old.- In the fixed version,
Disallow: /adminblocks only the admin area. Paths are case-sensitive, and when two rules match, the longer one wins. Sitemap:takes the full URL of the sitemap and can go anywhere in the file.
The costliest mistake is combining robots.txt with noindex. The noindex documentation is blunt: for noindex to work, the page must not be blocked by robots.txt, because if it is, the crawler never sees the noindex rule. A page blocked by robots.txt cannot show its noindex to Google. These are the two forms of noindex and the canonical a new page should carry:
<meta name="robots" content="noindex">
X-Robots-Tag: noindex
<link rel="canonical" href="https://example.com/blog/new">
- The
metatag withname="robots"goes in the<head>and applies to every search engine that supports it; withname="googlebot", only to Google. X-Robots-Tagis the equivalent HTTP header. It also works for non-HTML resources such as PDFs, video and image files.link rel="canonical"names the preferred URL. On a new page it should point to itself, as an absolute URL.
On canonicals, the guide to consolidating duplicate URLs ranks the signals: redirects and rel="canonical" are strong signals, sitemap inclusion is a weak one. Google may pick another URL if it decides the page is a duplicate, and it asks you not to use robots.txt or noindex to choose a canonical. On a two-language site, each version is canonical for itself and the pair is linked with hreflang; we cover that in hreflang in Next.js. The report help also notes that Search Console does not detect alternate language pages as alternates.
Check the signals with a script
To avoid checking every URL by hand, we wrote check-indexable.mjs: a dependency-free script for Node 18 or later that reads what Google reads for one URL. It downloads robots.txt and applies the longest matching rule (on a tie, the least restrictive one, as Google documents), fetches the page without following redirects, looks for noindex in the header and the meta tag, compares the canonical with the URL, and looks for the URL in the sitemaps robots.txt declares. The full script is in the guide. This is what it printed against a local test site and one public URL:
$ node check-indexable.mjs http://127.0.0.1:62204/blog/new-post
OK http://127.0.0.1:62204/blog/new-post
$ node check-indexable.mjs http://127.0.0.1:62204/drafts/launch
FAIL http://127.0.0.1:62204/drafts/launch
- robots.txt blocks it: Google cannot crawl it, so it never sees a noindex
- meta robots: noindex
$ node check-indexable.mjs http://127.0.0.1:62204/blog/new-post-copy
FAIL http://127.0.0.1:62204/blog/new-post-copy
- canonical points to http://127.0.0.1:62204/blog/new-post
! not in the sitemap
$ node check-indexable.mjs https://ecomlabs.dev/blog/hreflang-nextjs-bilingual-site/
FAIL https://ecomlabs.dev/blog/hreflang-nextjs-bilingual-site/
- redirects (308) to /blog/hreflang-nextjs-bilingual-site: only the target is indexed
! not in the sitemap
- The test site's new page passes: it returns 200, has no noindex, its canonical is itself and it is in the sitemap, which the script found through a sitemap index.
/drafts/launchhas both problems at once: robots.txt blocks it and it also carries a noindex. Google will never see that noindex.- The copy points its canonical at another URL and is not in the sitemap: if Google indexes anything, it will be the other URL.
- The last one is a real public page typed with a trailing slash: it returns a 308 to the version without the slash, so only the target gets indexed. Without the slash, the same URL passes.
Lines starting with - are failures and make the script exit with code 1; lines with ! are warnings. Missing from the sitemap does not block indexing, but it removes one way to discover the page. The script does not replace URL Inspection: it cannot know which canonical Google picked or whether Google saw the page as a duplicate, and it does not run JavaScript, so it misses a tag injected in the browser.
Internal links
The report help puts it plainly: Google needs a way to find a page in order to crawl it, which means the page must be linked from a known page or from a sitemap. A page that exists only in the sitemap, with no links from the rest of the site, is an orphan: Google can discover it, but it never reaches it from the pages it already visits.
Per the guide to crawlable links, Google uses links as a relevance signal and to find new pages to crawl, and it can generally only follow a link if it is an <a> element with an href attribute:
<!-- Google can follow this -->
<a href="https://example.com/blog/new">New article</a>
<!-- Not recommended: Google may try to parse them, no guarantee -->
<a routerLink="blog/new">New article</a>
<span href="https://example.com/blog/new">New article</span>
<a onclick="goto('/blog/new')">New article</a>
- The first is an
<a>with anhref, the format Google documents as crawlable. routerLinkwithouthrefrelies on JavaScript to navigate.- A
spanis not a link, even with anhref. onclicknavigates in code: there is no URL in the markup to follow.
Link the new page from where Google already goes: the home page, the category or the blog index, and two or three related articles. In a store, the collection and related products do that job. If you also want Google to understand what you sell on that page, add product structured data. For a brand-new site, Google says the best first step is to request indexing of your home page.
Request indexing after you fix it
Once the cause is fixed, do not request indexing blind. This is the order:
- Inspect the URL and click Test live URL. If the test still sees the noindex or the block, the fix did not reach production: check the deploy and the cache.
- If the test passes, click Request indexing. If the page passes a quick check, it goes into the indexing queue; if the live test says it is not indexable, you cannot request it.
- With many pages, do not go one by one: there is a daily limit on requests, and Google recommends submitting a sitemap with the updated pages marked by
lastmod. - If you fixed an issue that affects many URLs in the report, use Validate fix on that status. Google checks a few URLs first and then the rest; validation typically takes up to about two weeks, and sometimes much longer.
- If you changed robots.txt, the robots.txt report in Search Console lets you request a recrawl of the file.
Asking more than once does not speed anything up. The page on how to ask Google to recrawl your URLs says there is a quota for submitting individual URLs and that requesting a recrawl multiple times for the same URL or sitemap won't get it crawled any faster.
How long Google indexing takes for a new page
Google gives no single number, and its own pages give different ranges. This is what they say, word for word:
- URL Inspection help: "Indexing typically takes only a day or so, but can take much longer in some cases." In the same section: "Indexing can take up to a week or two."
- Page indexing report help: "It can take a week or so for Google to start crawling and indexing a new page or site", and "When you add new content, it can take a few days for Google to index it."
- Ask Google to recrawl: "Crawling can take anywhere from a few days to a few weeks."
None of these is a promise: a request does not guarantee the page will appear in the index. The report help adds that Google recrawls your pages on its own and that you don't need to ask unless there is an important change Google hasn't noticed for a while (a week or more). The working rule that follows: if a week has passed, the page is in the sitemap, it has links and URL Inspection shows nothing blocking it, inspect it again and read the status. Do not request indexing again just because the page has not appeared yet.
Common mistakes
- Blocking a noindex page in robots.txt. Google cannot crawl it, so it never sees the noindex, and the URL can still appear without a description.
- Launching with the staging robots.txt or noindex. Check both in production after every launch.
- A canonical inherited from a template. If every page points to the home page, or the new page points to the old one, Google indexes the other URL.
- Relative URLs in the sitemap. They must be complete, with protocol and host.
- Linking to or listing a URL that redirects, for example with a trailing slash when the canonical has none. Only the target is indexed.
- An orphan page. It is in the sitemap, but no page on the site links to it with an
<a href>. - Requesting indexing every day. There is a quota and it does not speed up crawling.
- Treating "Crawled - currently not indexed" as a technical fault. Google says it is common; rule out blocks with URL Inspection first.
Frequently asked questions
How long does Google take to index a new page?
Google gives several ranges: "a day or so" after a request, "up to a week or two" in the same help article, and "a week or so" for a new page or site. None of them is a guarantee.
Does Request indexing make Google index the page?
No. If the page passes a quick check, it goes into the queue, but the help says submitting a request does not guarantee the page will appear in the index. Requesting it several times does not speed up the crawl either.
Is blocking a page in robots.txt enough to keep it out of Google?
No. robots.txt stops crawling, not indexing: if other pages link to the URL, it can appear without a description. To keep it out, use noindex and do not block it in robots.txt, or put it behind a password.
What do I do about "Discovered - currently not indexed"?
Google knows the page but postponed the crawl. Check that it is in the sitemap, that pages Google already crawls link to it, and that your server responds well. If the page matters, request indexing once and wait.
Checklist
- The URL returns 200, with no redirect.
- No robots.txt rule matches the path, and the page is never both blocked and noindexed.
- No noindex in the meta tag or in the
X-Robots-Tagheader. - The canonical points to the page itself, as an absolute URL.
- The full URL is in the sitemap, and the sitemap is submitted in Search Console or declared in robots.txt.
- At least one indexed page links to it with an
<a href>. - The live URL test passes before you request indexing.
- You requested indexing once and checked again after a week, not before.
The guide has the full script, the diagnosis order on one page and the checklist to print.
Saw this on Instagram? Comment INDEXAR on the Reel and I'll send you the guide.
Written by

Published November 19, 2026