Is llms.txt an official standard?

No. It is a community proposal that gained traction through adoption rather than through any standards body. There is no specification you can be compliant with and no conformance test.

Do the major AI engines actually read it?

Support is partial and poorly documented. Several AI-native developer tools and documentation platforms clearly consume it. Whether the large consumer assistants use it as a retrieval input is not publicly confirmed by the providers, so any vendor stating otherwise is going beyond the evidence.

Is llms.txt the same as robots.txt?

No, and conflating them is the most common mistake. robots.txt withholds permission to crawl and is broadly respected. llms.txt offers a curated guide to content and carries no access-control meaning whatsoever. Blocking a crawler in robots.txt while listing the same pages in llms.txt achieves nothing.

Will llms.txt improve my citation rate?

There is no reliable public evidence that it does on its own. It plausibly helps a system that already fetches it find your best pages faster. Retrievability, structure and authority are where measurable gains come from.

llms.txt Explained: What It Does and What It Doesn't

What it is

llms.txt is a Markdown file served at the root of your domain — https://example.com/llms.txt — containing a short description of what your site is and a curated list of links to your most useful pages, each with a one-line summary.

The reasoning behind it is sound. An AI system arriving at a large site faces navigation designed for humans: menus, promotional modules, cookie banners, related-content rails. A single file that says "here is what this site is, and here are the twenty pages that actually matter" is a genuine convenience.

What a real one looks like

The format is deliberately plain. An h1 with the site name, an optional blockquote summary, then h2 sections containing link lists.

# AEO Landscape

> Independent guides, research and tool comparisons for
> Answer Engine Optimization.

## Guides

- [What Is Answer Engine Optimization?](https://aeolandscape.com/guides/what-is-answer-engine-optimization): Definition, scope and limits of AEO.
- [The AEO Metrics That Matter](https://aeolandscape.com/guides/the-aeo-metrics-that-matter): The six metrics worth reporting, and the sampling discipline behind them.

## Tool comparisons

- [CiteCue vs Profound](https://aeolandscape.com/compare/citecue-vs-profound): Mid-market automation against enterprise coverage.

## Reference

- [AEO Glossary](https://aeolandscape.com/glossary): Definitions of common terms.

That is the entire specification in practice. Some sites also publish llms-full.txt containing the complete text of their key pages in one file, which suits documentation sites where a model may want everything at once.

What it is not

It is not an access-control mechanism. robots.txt says what a crawler may fetch and is widely respected. llms.txt says "these pages are useful" and grants nothing. If you block GPTBot in robots.txt and then list your pricing page in llms.txt, the page stays unfetchable. This confusion is common enough to be worth stating twice.

It is not a ranking signal. Nothing in the format assigns importance that an engine is obliged to honour. Listing a page does not privilege it.

It is not a substitute for retrievable pages. The links point at your real URLs. If those URLs need JavaScript to render their content, llms.txt has helped a crawler find a page it still cannot read.

It is not universally consumed. This is the honest part that vendor blog posts tend to skip. Adoption among AI-native developer tools and documentation platforms is clear. Whether the large consumer assistants use it as a retrieval input is not confirmed by their providers. Anyone telling you ChatGPT reads your llms.txt is asserting something not publicly established.

So why bother?

Because the cost is an afternoon and the downside is zero.

The realistic case for it: the file is trivial to produce, it cannot hurt, adoption is plausibly increasing, and the exercise of writing it is independently useful. Deciding which twenty pages actually matter, and describing each in one line, is a content audit in disguise. Teams routinely discover during this exercise that their most valuable page has no clear description, or that four pages cover the same ground.

The unrealistic case, which you will encounter: that llms.txt is the key to AI visibility. It is not. It is a nice-to-have, and treating it as a strategy is a way of avoiding the harder work of retrievability, structure and authority.

How to write one worth having

Curate, do not dump. Twenty well-chosen pages beat two hundred. A sitemap already exists for exhaustiveness; this file is for editorial judgement.
Write real descriptions. The one-line summary after each link is the most valuable part. Say what the page answers, not what section it lives in.
Group by user intent, not by your site's information architecture. "Getting started", "Reference", "Comparisons" beats "Blog", "Resources", "Content hub".
Serve it as `text/plain` or `text/markdown` at the domain root, and make sure it returns 200 to non-browser user agents. A file behind bot mitigation is not a file.
Keep it current. A stale llms.txt pointing at retired URLs is worse than none, because it advertises neglect to exactly the systems you are courting.
Do not contradict `robots.txt`. Cross-check the two. Every URL you list should be one you permit.

Automating it

Maintaining the file by hand is where this falls down. It gets written once during a burst of enthusiasm and is stale within two quarters.

Generating it from your CMS at build time is the durable answer, and it is what this site does: the file is produced from published content on every request, so it cannot drift out of date. Some AEO platforms will also generate and serve it for you — CiteCue's AI Auto-Fix does this as part of serving optimised page variants, which removes the maintenance problem entirely for teams without engineering capacity to spare.

If you are choosing between hand-maintaining a file that will rot and having a tool keep it fresh, the tool wins on the only axis that matters, which is whether it is still accurate in six months.

The honest summary

Publish one. Spend an afternoon, not a sprint. Then go and fix your crawler access, because that is where the measurable gains are.