What is llms.txt and how to create one
llms.txt is a small Markdown file at the root of a website that tells language models what the site is about and which pages matter most. It was proposed in September 2024 as a simple companion to robots.txt and sitemaps.
What goes in it
- A heading with the name of the site.
- A short summary in a quote block.
- Sections with links to the most useful pages, each with a short description where it helps.
# Acme Running
> Running shoes and advice for flat feet, trail and road running.
## Shop
- [Trail shoes](https://acme.example/trail): grip and support for rough terrain
- [Road shoes](https://acme.example/road): light shoes for daily runs
## Advice
- [Running with flat feet](https://acme.example/advice/flat-feet)Does it help?
Honestly: it is a proposal, not an official standard, and AI companies have not all said whether they read it. It takes a minute to create, does no harm and gives a clean summary of your site to any tool that does read it. That is why many sites add it alongside robots.txt and a sitemap.
Create one in a minute
The llms.txt generator builds a first version from your sitemap and home page. Rename the link titles, keep the pages that matter and upload it as /llms.txt.
Questions
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may fetch. llms.txt describes what your site is and links to the best pages. It does not allow or block anything.
Should I list every page?
No. List the pages that help someone understand your site and its main topics. A short, curated list is more useful than a copy of the sitemap.