- Home
- Artificial intelligence
- llms.txt Generator
llms.txt Generator
Introduce your site to AI systems in a file they can read in one pass.
Local generation in Markdown, following the llms.txt proposal published in 2024. Nothing is stored, and there is no usage limit.
What this tool is for
When a language model has to understand your site, it runs into an obstacle: your pages are built for humans. Menus, banners, boxes, formatting — most of the HTML is noise to a machine looking for facts.
The llms.txt file offers a simple answer: a Markdown document, at the root of the site, setting out plainly what the site does and which pages matter. A model can read it in one go instead of crawling a hundred pages.
Let us be precise about its status: it is not an official standard. It is an open proposal from 2024, formally adopted by no major player. Adoption is progressing mainly among publishers, starting with technical documentation.
Why do it anyway: an hour’s work, no risk, and a head start if adoption takes off. The cost-benefit ratio is favourable — a reasonable bet, not an obligation.
How to read the results
The title and summary open the file. They are the first thing read: one sentence saying what the site does, with no marketing. “Free SEO tools” is more useful than “the innovative solution for your visibility”.
The thematic sections group your pages by subject. That organisation helps as much as it informs: it shows the structure of your site.
The annotated links are the heart of the file. Each important page with one sentence of description. Do not list everything: twenty well-chosen pages beat two hundred.
The optional section gathers what is secondary. Models with little context can ignore it without losing the substance.
Publishing happens at the root of the domain, like robots.txt. Then check with the AI crawler access checker that the agents you care about can reach it.
Common mistakes
Presenting it as a standard. It is not one. Selling it as compulsory is dishonest, and would set you up for disappointment if adoption does not follow.
Listing every page in it. The principle is selection. A file with three hundred entries loses its point: you may as well let the sitemap be crawled.
Writing marketing copy in it. Empty phrases carry no facts. A model cites what it can check and attribute, not adjectives.
Thinking it replaces robots.txt. They do the opposite of each other: one restricts access, the other directs reading. The robots.txt stays indispensable.
Writing it then forgetting it. A file describing a two-year-old version of the site does more harm than good. Keep it up to date with your important pages.
Going further
- AI Crawler Access CheckerCheck whether GPTBot, ClaudeBot and PerplexityBot can read your pages
- Googlebot SimulatorSee a page exactly as a search engine crawler sees it
Frequently asked questions
Is llms.txt an official standard?
No, and that has to be said plainly. It is an open proposal, born in 2024, that no major player has officially adopted. OpenAI, Anthropic and Google have not committed to reading it. Adoption is progressing among publishers, not yet among engines.
Why create one then?
Because the ratio of cost to potential benefit is favourable. The file takes an hour to write, carries no risk, and puts you ahead of those who will discover the subject in two years if adoption takes off. It is a reasonable bet, not an obligation.
How does it differ from robots.txt?
They do the opposite of each other. Robots.txt says what should not be crawled; llms.txt says what is worth reading and how to understand it. One restricts, the other directs. They are complementary and do not replace each other.
What should go in it?
A short account of what the site does, then an organised list of your important pages with one descriptive sentence each. The format is simple Markdown. The aim is for a model to understand your site from one file rather than by crawling a hundred pages.
Do I also need an llms-full.txt?
Only if your documentation lends itself to it. That second file holds the full text of your reference pages, to be read in one go. It makes sense for technical documentation, much less for a commercial site.
Does it improve my Google rankings?
No, no effect on ranking at all. The intended benefit is different: being better understood, and so better cited, by conversational assistants. It is a channel distinct from classic search, growing in importance without replacing it.