Rendering: what AI fetchers see when your page needs JavaScript
Advice about JavaScript and search is written for Googlebot, whose behavior is documented in detail. Google executes JavaScript in a deferred second pass. The clients that fetch pages for AI answers are mostly not documented to do anything of the kind. On a site whose main content appears only after client-side scripts run, that difference decides whether an engine sees your content or an empty page. Crawl access, sitemaps, canonicals and speed belong to technical SEO. This page is the deep version of one paragraph from it.
Two models of fetching a page
Google's model has three phases: crawling, rendering and indexing. Its JavaScript SEO basics documentation states that Googlebot queues all pages with a 200 HTTP status code for rendering, that a headless Chromium then renders the page and executes the JavaScript, and that a page "may stay on this queue for a few seconds, but it can take longer than that". Rendering is real, deferred and conditional. Microsoft documents the same capability, announcing in 2019 that bingbot runs JavaScript and renders pages with the Microsoft Edge engine.
The other model is simpler: request the URL, receive the response body, parse the HTML, extract text and links, stop. No browser, no script execution. Anything that exists only in the DOM after scripts have run is not there to extract. Such a client reads a different document than your browser does.
What each AI company documents about JavaScript
OpenAI. The crawler documentation describes four user agents, OAI-SearchBot, OAI-AdsBot, GPTBot and ChatGPT-User, along with robots.txt handling, full user-agent strings and published IP ranges. It contains no statement about JavaScript execution or rendering. The example string for OAI-SearchBot carries a Chrome version number, which describes the string the fetcher sends, not the engine behind it.
Anthropic. The crawler support article describes ClaudeBot, Claude-User and Claude-SearchBot, the robots.txt directives that control them, support for the non-standard Crawl-delay extension, and a published IP list. It contains no statement about JavaScript execution or rendering.
Perplexity. The bots documentation describes PerplexityBot and Perplexity-User, separating automatic crawling from user-initiated fetches. It contains no statement about JavaScript execution or rendering.
The accurate summary is that none of the three documents it. That is a weaker claim than saying they do not execute JavaScript, and the difference matters. An undocumented behavior may already be in place, may change without announcement, and cannot be planned against with confidence. Which token controls which behavior is covered in AI crawlers.
Measurement fills part of the gap. A network-level study published in December 2024, covering 569 million GPTBot fetches, 370 million Claude fetches and 24.4 million PerplexityBot fetches over one month, reported that none of the major AI crawlers rendered JavaScript, and that the ChatGPT and Claude crawlers requested JavaScript files (11.50 and 23.84 percent of their requests) without executing them. The same study found Gemini using Googlebot's infrastructure and rendering fully. That is one network, one month, and more than a year ago, and it describes behavior rather than promising it. Treat it as evidence, not as documentation.
The consequence on a client-rendered site
If your primary content is written into the page by client-side JavaScript, a fetcher that does not execute scripts receives the shell: a title, some meta tags, an empty mount point and script references. There is no article in it. The page looks complete in every browser you test with, because your browser runs the scripts. The failure is invisible from the surface a human uses to check.
Rendering strategies, judged on one question
These strategies differ in many ways. Only one is relevant here: is the main content present in the initial HTML response, before any script runs.
| Strategy | Main content in the initial HTML response |
|---|---|
| Client-side rendering | No. The response is a shell; scripts fetch and insert the content. |
| Server-side rendering | Yes. The HTML is generated per request and contains the text. |
| Static generation | Yes. The HTML is generated at build time and served as a file. |
| Incremental or on-demand regeneration | Yes. Pages are served as pre-generated HTML; regeneration affects how fresh it is, not whether the content is there. |
| Hydration on top of server-rendered HTML | Yes for whatever the server rendered. Anything that appears only after hydration is not. |
The axis says nothing about which approach is better for users. A client-rendered application simply needs a path that serves content without scripts.
How to check your own pages
- Fetch one of your URLs with a plain HTTP client that does not execute scripts, for example
curl, and save the response body. - Read the saved HTML for your actual sentences, headings and internal links. Not the title tag, not a JSON blob you assume something parses.
- Open the same URL in a browser and compare against the rendered DOM. Whatever appears only there is what a non-executing fetcher never sees.
- Repeat per template: home, article, category, product, search results. Sites frequently server-render one template and client-render another, so one spot check per site proves nothing about the rest.
Related traps
Content behind interaction. Tabs and accordions may carry their text in the HTML and hide it with CSS, or they may fetch it on click. Both behave identically for a reader; only the source tells you which one you have.
Pagination that depends on events. Infinite scroll and click-to-load hide everything past the first batch from any client that does not scroll or click. Where the remaining items matter, give them crawlable links to numbered pages.
Lazy loading. Lazily loading images is normal and low risk. Lazily loading text removes content from the initial response, which is exactly what decides this question.
This is a different question from ranking in Google
Google renders, so a client-rendered site can be indexed and can rank. That tells you nothing about whether a fetcher that does not render can read the same page. A site can hold strong positions in Google Search and be effectively absent from answer engines that never see its text. How the engines differ in retrieval and citation is compared in engines compared.
Verifying after you change something
Verify a rendering change where the fetchers actually arrive. Your server logs show which agents requested which URLs and what status they received, covered in server logs. Edge rules can block or challenge those same agents before a request reaches your application at all, covered in CDN and WAF rules. Improving the HTML only helps if the request gets through.