noindex and meta robots explained: when to use them and how to remove them
What noindex does, the difference between meta robots and X-Robots-Tag, why staging leaks kill rankings, and how to check a live page.
Short answer
A noindex directive tells search engines not to show the page in results. It can come from a meta robots tag in HTML or an X-Robots-Tag HTTP header. Use noindex for thank-you pages, private tools, and thin duplicates—not for pages you want to rank. If a public page is stuck out of Google, check for leftover noindex from staging, CMS defaults, or headers first.
If Google will not index a page, check for noindex first with the website checker or meta tag checker, then follow the indexing troubleshooting guide.
What noindex means
noindex is a robots directive: “you may crawl this URL, but do not show it in search results.” It is appropriate for pages that should stay out of Google—not for your homepage, pricing, or blog posts you want to rank.
Related directives you will see beside it: nofollow (do not treat links as endorsement), noarchive, nosnippet. For most marketing sites, the dangerous one is leftover noindex.
Meta robots vs X-Robots-Tag
Two common places to set it:
<!-- In HTML <head> --> <meta name="robots" content="noindex, nofollow" /> <!-- Or HTTP response header --> X-Robots-Tag: noindex
- Meta robots lives in the HTML. Easy to leave on from a staging theme or SEO plugin “discourage search engines” toggle.
- X-Robots-Tag lives on the HTTP response. Useful for PDFs and non-HTML files; also easy to miss if you only view-source the body.
Google also understands googlebot-specific meta tags. If any applicable rule says noindex, treat the page as noindexed until proven otherwise.
When to use noindex
- Thank-you / confirmation pages after forms or checkout.
- Internal search result URLs and thin filter combinations.
- Staging or preview deployments that are somehow publicly reachable.
- Duplicate print views or parameter pages you cannot canonicalize cleanly.
Do not noindex a page just because it ranks poorly—fix the content and links instead.
How to find and remove it
- Fetch the live URL (not localhost). Confirm
meta name="robots"and response headers. - In WordPress, check Settings → Reading (“Discourage search engines”) and SEO plugins (Yoast / Rank Math page-level robots).
- In Next.js and similar frameworks, search for
robots: { index: false }inmetadataor layout defaults. - Remove noindex, deploy, re-check with Shipcheck, then use Search Console → URL Inspection → Request indexing.
noindex vs canonical vs robots.txt
| Tool | Use when |
|---|---|
| noindex | Page must stay out of results but can still be linked or visited. |
| Canonical | Several URLs show similar content; you want one preferred URL to rank. |
| robots.txt Disallow | You want to reduce crawling of whole sections (admin, APIs)—not a substitute for noindex. |
| 301 redirect | The URL permanently moved; send users and signals to the new address. |
Questions
What does noindex do?
It asks search engines not to include the page in search results. Crawlers may still fetch the URL; they just should not show it in Google or Bing results.
Is noindex the same as blocking in robots.txt?
No. robots.txt can stop crawling of a path. noindex is usually discovered after the page is fetched. If robots.txt blocks the page, engines may never see a noindex tag on it.
How do I remove noindex?
Delete meta name="robots" content that includes noindex, and check response headers for X-Robots-Tag: noindex. Publish, then use Search Console URL Inspection and request indexing if needed.
Should I noindex my whole site while building?
Prefer a password, staging host, or auth gate. Sitewide noindex on production is a common launch disaster when someone forgets to turn it off.
Tools for this guide
Free website checker
Paste a public URL for SEO, GEO, and AEO scores from server HTML, each with a fix prompt. The free Chrome extension still helps on localhost and JS-heavy apps.
Open tool →Canonical tag checker
Check rel=canonical for missing tags, www/https mismatches, and paths that do not match the live URL.
Open tool →Meta tag & social preview checker
See your Google, X, LinkedIn, and Slack previews and fix titles, descriptions, and OG images.
Open tool →More guides
H1 and heading structure for SEO: one clear headline, then a logical outline
How many H1s a page should have, how H2/H3 help SEO and accessibility, and the mistakes AI builders and page builders make with headings.
Read · 8 min →Favicon SEO and sizes: what to ship so tabs, SERPs, and bookmarks look right
Which favicon sizes to include, how Google may show icons in search, SVG vs PNG vs ICO, and how to check a live site’s icons.
Read · 7 min →Best free website SEO checker (2026): what to look for and which tools fit
How to choose a free website SEO checker with no signup: server-HTML vs live-tab tools, must-have checks, and when Shipcheck, Lighthouse, or specialist extensions win.
Read · 10 min →