llms.txt: what it is, and whether it actually does anything

llms.txt is a proposed convention: a markdown file at your site root that tells an AI assistant what your site is and which pages are worth reading.

Most writing about it falls into one of two camps. Either it's presented as a new standard you're already behind on, or it's dismissed as pointless. Both are overstatements, and the honest position is more useful than either.

What it actually is

A markdown file at https://yoursite.com/llms.txt. The convention is:

# Acme

> Acme builds inventory software for independent hardware stores.
> Free for a single location, from $40/month for chains.

## Main pages

- [Pricing](https://acme.com/pricing)
- [How it works](https://acme.com/how-it-works)

## Guides

- [Stock counts without closing the store](https://acme.com/guides/stock-counts)

An H1 with the site name, a blockquote summary, then H2 sections of markdown links. Some sites also publish llms-full.txt containing the full text of key pages, so an assistant can read the content without following links.

How it differs from things you already have

This trips people up, because there are now three files at your root that all sound like they do the same job.

robots.txt grants or denies access. It's about permission: which crawlers may fetch what.

sitemap.xml is an exhaustive machine-readable list of every URL you want indexed. It's about completeness. A large site's sitemap has thousands of entries and no editorial judgement in it at all.

llms.txt is neither. It's short, curated and written for a reader. The whole point is the editorial judgement. You're saying "of the four thousand URLs in my sitemap, here are the twelve that matter, and here's one sentence on what we do."

That's why generating one purely from a sitemap gets you most of the way but not all of it. The list of pages can be derived. The summary line can't.

Does anything read it?

Here's the part usually left out.

No major engine has publicly committed to reading llms.txt. Not OpenAI, not Anthropic, not Google, not Perplexity. There is no announced support, no documentation in any engine's publisher guidance, and no independent study showing that publishing one changes how often a site is cited.

What exists is a proposal, some early adoption by developer-tooling companies, and a lot of blog posts describing it as though the first two facts were the third.

So the case for publishing one is not "it works." It's:

• It costs one file and about ten minutes.
• It cannot hurt. It's a static text file at a URL nobody has to visit.
• If adoption does happen, you're already done.
• Writing it forces you to state, in one sentence, what your company does and which pages matter. That's a useful exercise whether or not a machine ever reads the result.

That's a real but modest case. If a tool or agency tells you llms.txt is required, or quotes you a figure for how much it improves your citation rate, ask where the number came from.

Where it sits in the order of work

Below crawler access, and below structured data.

If GPTBot gets a 403 from your CDN, your llms.txt is a file nobody can fetch. If your pages carry no Organization markup, an engine doesn't reliably know what entity your site belongs to, and a curated link list won't fix that. Both of those are things engines demonstrably use. llms.txt is a thing they might.

The sensible order:

1. Check crawler access. Engines can't cite what they can't fetch.
2. Check your structured data: Organization, sameAs, and a product entity.
3. Publish llms.txt. Cheap, plausible, unproven.

Writing one

Generate the page list from your sitemap, then edit. The parts to do by hand:

The summary line. One or two sentences: what you do, who for, and roughly what it costs. Write it as though explaining to someone who has never heard of you, because that's exactly the situation. Don't write marketing copy. An assistant summarising you will reuse the phrasing, and "the leading platform for modern teams" tells it nothing it can repeat.

The selection. Cut aggressively. A file listing four hundred pages has the same problem as a sitemap: no judgement in it. Twenty is plenty for most sites.

The grouping. Give the sections names a stranger would understand. "Guides" and "Pricing" beat "Resources" and "Plans."

Our llms.txt generator reads your sitemap and drafts the file with the structure in place and the page list filled in, so the work left is the summary and the cut. It doesn't publish anything for you, and it leaves the summary line blank on purpose, because that line is the only part that carries information nobody else can supply, and generating it would defeat the point of the file.