First: Check if there is a problem
Before looking for the cause, make sure the problem actually exists. Type into Google:
If there are no results or significantly fewer than the number of pages on the site — read on.
Also open Google Search Console → Indexing → Pages. There Google itself explains why each page is not indexed.
12 Reasons and Fixes
noindex tag on the page
The most common reason for "invisibility" in Google. The page code contains a tag that directly prohibits indexing:
Such a tag may have appeared by mistake during development, or if the CMS automatically sets noindex for certain types of pages (tags, categories, search pages).
Check the code of each page (Ctrl+U in the browser, search for "noindex"). In WordPress — check Yoast SEO or RankMath, visibility settings for each post type.
Blocked in robots.txt
The robots.txt file can block Googlebot from crawling entire sections of the site. Open yoursite.com/robots.txt and look for lines like:
If the line Disallow: / — this blocks the entire site. Delete or replace with Disallow: (empty value — allows everything). Check via the robots.txt testing tool in Search Console.
Site is password-protected or under maintenance
If the site requires authorization or returns a 503 code — Googlebot cannot access the content. This is typical for new sites launched "quietly".
Make sure the site is accessible without authorization. Check via Google Search Console → URL Inspection → "Test Live URL".
Low crawl budget — Googlebot cannot keep up
Google allocates a certain "crawl budget" to each site — the number of pages it is ready to crawl in a given time. For new and small sites, this budget is minimal.
If the site has thousands of pages or changes frequently — Googlebot simply cannot crawl all URLs.
Use the Google Indexing API for priority pages. Remove "junk" URLs from the sitemap (tags with 1 article, empty categories, search queries). Check site speed — a slow site gets a smaller crawl budget.
Duplicate content and canonical
If multiple URLs have the same or very similar content, Google chooses one "canonical" URL and may ignore the others. Common issues:
- HTTP and HTTPS versions of the site without a redirect
- www and non-www versions
- Pages with and without trailing slash: /page and /page/
- URL parameters: /page?utm_source=... instead of /page
Set the canonical tag on all pages pointing to the "primary" version. Configure 301 redirects. In Search Console check the "Discovered — currently not indexed" report — it often contains duplicate information.
Thin or low-quality content
Google may refuse to index pages with little unique text. At risk:
- Pages with fewer than 300–500 words
- Automatically generated content
- Template pages with minimal unique content (tags, months)
- Content copied from other sites
Either significantly improve the content or close such pages with the noindex tag. It is better to have 50 quality indexed pages than 500 "invisible" ones.
Speed up indexing with IndexFast
Once you eliminate technical problems — send all URLs directly via the Google Indexing API. You will see the result in a few hours, not weeks.
Try for free →Server errors — 4xx and 5xx codes
If Googlebot receives errors when requesting pages — it stops visiting them and removes them from the index. Codes 404 (not found), 500 (server error), 503 (service unavailable) — all of this reduces crawling activity.
Search Console → Indexing → Pages → "Not found (404)" and "Server error (5xx)". Fix or configure 301 redirects for deleted pages. Check server logs for mass errors.
Slow page load speed
Google враховує Core Web Vitals при ранжируванні і при визначенні crawl budget. Дуже повільний сайт (LCP > 4 секунди) отримує менший бюджет сканування.
Check via PageSpeed Insights (pagespeed.web.dev). Key metrics: LCP (Largest Contentful Paint) under 2.5s, FID under 100ms, CLS under 0.1. Optimize images, enable caching, connect a CDN.
No links to the page
Googlebot finds new pages mainly through links. If there are no internal or external links to a page — it may remain "invisible" even if it is in the sitemap.
Add internal links from other popular pages. Make sure new articles appear in "Read Also", "New Publications" sections, etc. Or use the Indexing API to directly notify Google.
JavaScript rendering blocks content
If your site is built on React, Vue or Angular and content is rendered only on the client side — Googlebot may not see the text of the pages. It "sees" only an empty HTML shell.
Configure Server-Side Rendering (SSR) or Static Site Generation (SSG). Check via Search Console → URL Inspection → "View Crawled Page" — it shows exactly what Googlebot sees.
Young domain — just needs time
For new domains (less than 6 months old), Google usually adds pages to the index more slowly. This is normal — algorithms verify the reliability of new resources.
Get a few quality external links (from authoritative sites in your niche). Regularly publish quality content. Use the Indexing API to speed things up. Register the site in Search Console and add a sitemap.
Google filter or penalty
In rare cases, a site may receive a manual penalty from the Google team (for link manipulation, spammy content, cloaking, etc.) or an automatic algorithmic penalty.
Check Search Console → Security & Manual Actions → Manual Actions. If there is a penalty — fix the violations and submit a review request. For algorithmic filters — improve content quality and link profile.
Step-by-Step Action Plan
If your site is not indexed — act in this order:
- Check robots.txt — make sure there are no blocks
- Check the noindex tag on problematic pages
- Open Search Console → Pages → read the exclusion reasons
- Check site accessibility and server response codes
- Check speed via PageSpeed Insights
- Eliminate the found problems
- Send URLs via Google Indexing API to speed up indexing
IndexFast shows which URLs from the sitemap were successfully sent via the API, and which returned an error — this helps quickly identify technical problems with specific pages.
Automatic Indexing — Simple and Fast
After eliminating the causes — launch re-indexing of the entire site via IndexFast. 500 URLs in one click, result within 24 hours.
Try for free →