Indexing for AI search:
Sitemaps, rendering and canonicals
An engine can only cite a page it has crawled and indexed. For Google AI Overviews and AI Mode the page must be indexed and eligible for a snippet. Submit a sitemap with accurate lastmod dates, use IndexNow, render content on the server, and keep canonicals and noindex tags clean.
pages indexed on Google after ten fixes, up from 42
ThriveStack citedby researchpages ever cited by AI over the same eight weeks
ThriveStack citedby researchmajor AI crawlers rendered JavaScript in Vercel’s study (vendor data)
Vercel, Dec 17, 2024extra requirements for AI Overviews beyond being indexed, per Google
Google Search CentralWhy does indexing come before citations?
Citation is the last step in a chain. A page is crawled, then indexed, then retrieved for a search, then cited. A break early in the chain stops everything after it. In our own case, fixing indexing first took pages cited by AI from 2 to more than 100.
Can AI crawlers read your pages?
Vercel found none of the major AI crawlers run JavaScript. If your text only appears after scripts run, AI crawlers see an empty page. Read the table by rendering method, then check your top pages with JavaScript turned off.
Rendering and AI crawlers:
Which methods put text where bots can read it
| Method | What the crawler gets | Readable by AI crawlers? |
|---|---|---|
| Server-side rendering | Full text in the HTML | Yes |
| Static pre-rendering | Full text in the HTML | Yes |
| Client-side rendering | Text built in the browser by JavaScript | Often not |
| Text in images or PDFs only | No page text to quote | Rarely |
Which indexing checks matter most for AI?
- XML sitemap with every page you want cited and an accurate lastmod
- IndexNow for Bing, which Copilot draws on
- One canonical URL per page, used in sitemap and links
- No noindex or nosnippet on pages you want in AI Overviews
- Key text in the server HTML, not built by JavaScript
- Index status checked in Google Search Console
Bing says it treats accurate lastmod as a key freshness signal and ignores changefreq and priority. Google says no special AI files or markup are needed.
How do you see indexing and AI crawls per page?
Content Audit joins Google index status, AI bot hits and citations for each sitemap page. A good sign is a funnel where most sitemap pages are crawled and indexed. A warning sign is a big drop between sitemap and indexed.
123- 1Sitemap to Google crawled and indexed, plus AI bot hits on the same URLs.
- 2Connect Google Search Console to overlay real index status.
- 3How many tracked prompts cite each page.
Research for this lesson.
Our own studies behind this lesson. Each one has the key charts and numbers, and a link to the full article.
ThriveStack citedby joins Search Console index status, AI crawler hits and citations for every sitemap page, so you see where the chain breaks.
All 10 questions this page answers.
These questions come from a query fan-out of "how to win AI search visibility". Each one has a short answer here, and the lesson above goes deeper.
Does AI search require Google indexing?Informational
For Google AI Overviews and AI Mode, yes. Google says a page must be indexed and eligible to show with a snippet. Other engines use their own crawlers, such as OAI-SearchBot and PerplexityBot, and Copilot draws on Bing, so check those too.
How to make pages eligible for AI OverviewsImplementation
Google says there are no extra requirements. Let Googlebot crawl the page, get it indexed, and do not block snippets with nosnippet. Then answer the subtopics the fan-out will search, in clear passages near the top of each section.
AI search canonical tag best practicesCommercial
Point each page’s canonical at the one URL you want cited, use the same URL in the sitemap and internal links, and never point a canonical at a redirect. Mixed signals can lead an engine to cite a duplicate or skip the page.
XML sitemap for AI discoveryInformational
List every page you want found, with an accurate lastmod date. Bing says it treats accurate lastmod as a key freshness signal for AI-powered search and ignores changefreq and priority. Pair the sitemap with IndexNow so changes are picked up fast.
JavaScript rendering for AI searchInformational
Vercel found none of the major AI crawlers render JavaScript (Dec 2024). ChatGPT’s crawler fetches JavaScript files but does not run them. Anything that appears only after JavaScript runs may be invisible to AI crawlers, even if Google can render it.
SSR vs CSR for AI visibilityCommercial
Server-side rendering sends the full text in the HTML, so AI crawlers can read it. Client-side rendering builds the page in the browser with JavaScript, which the major AI crawlers do not run. Use server rendering or static pre-rendering for every page you want cited.
Noindex and AI search visibilityInformational
A noindex page drops out of Google’s index, so it cannot appear in AI Overviews or AI Mode. Other crawlers may still read it, but treat noindex as removal from Google’s AI features. Check that no page you want cited carries it by mistake.
Internal linking for AI searchInformational
Internal links help crawlers find pages and show how topics connect. Link each pillar page to its cluster pages and back, use anchor text that names the topic, and make sure no page you want cited is an orphan with no links in.
Duplicate content and AI retrievalInformational
Duplicates split signals and can lead an engine to cite the wrong URL or an old version. Pick one canonical page, redirect or canonicalize the copies, and keep facts the same anywhere text is repeated.
AI search technical SEO auditImplementation
Check robots.txt for AI bots, CDN bot settings, server-rendered HTML, status codes in logs, a sitemap with lastmod, canonicals, noindex and snippet tags, speed and Search Console index status. In our own study, ten fixes raised indexing from 42 to 192 of 199 pages in eight weeks.
Sources for this lesson
- Google Search Central: AI features and your website Checked Nov 17, 2026
- Bing Webmaster Blog: Sitemaps in AI-powered search (Jul 31, 2025)
- Vercel: The rise of the AI crawler (Dec 17, 2024) Vendor claim, Vercel network data
- Bing: IndexNow Checked Nov 17, 2026
- ThriveStack: How we ranked on Google AI Overviews for Gemini Our own research
Figures were checked against each page on Nov 17, 2026. Vendor studies are labelled as vendor claims.
