How to check if ChatGPT can read your website

1. Check robots.txt for OpenAI’s crawlers

OpenAI uses three user agents, each with its own job:

• OAI-SearchBot builds the index behind ChatGPT search. Block it and your pages cannot show up as sources there.
• ChatGPT-User fetches a page when a user asks ChatGPT about it or shares a link.
• GPTBot collects data for training future models. Blocking it does not remove you from ChatGPT search.

A rule like User-agent: * with Disallow: / blocks all of them, sometimes by mistake on a site that was never meant to be private. Test your live file with the robots.txt tester.

2. Make sure the content is in the HTML

Crawlers read the HTML your server sends. Text that only appears after JavaScript runs in the browser may not be seen by every crawler. Scan your page on OnPage.dev: the word count and the “How an AI reads this page” view show what is in the HTML.

3. Give answers that can be quoted

AI assistants quote passages that answer a question directly. Pages that open with a clear answer, use question-style headings and keep paragraphs short are easier to quote. The AI readiness score in every OnPage.dev scan checks these points.

Optional: add an llms.txt

An llms.txt file gives language models a short summary of your site. It is a proposal rather than a standard, but it is quick to add with the llms.txt generator.

Questions