Home › Tools › llms.txt Generator

llms.txt Generator

Build a valid llms.txt. Measured against 22 live files: every link in them is absolute, the median file has 74 entries and is 15 KB. Nothing is uploaded.

By Hongtao Ren · Updated 2026-09-22

Paste your site name, a one-line summary, and your pages or tools. Get a properly formatted llms.txt you can drop at your domain root. Every default in this page comes from parsing the 22 llms.txt files that large sites actually publish, not from reading the spec. Nothing is uploaded.

How to use it

1. Fill in your site name and a one-line summary that says what the site does, not what it wants to rank for.
2. Enter your site origin. Every one of the 7,268 links in the 22 files we measured is an absolute URL, so this is the default.
3. Add one entry per page or tool, in the format Title | /path | description. Paths may be relative — they get joined to your origin.
4. Copy the result and save it as llms.txt at your domain root, next to robots.txt.

What llms.txt is actually for

llms.txt is a plain-text file at your domain root that describes what your site offers, written to be read by language models rather than by people. It is a proposal rather than a standard — no major crawler has committed to obeying it — but it costs one small file and it is the only place where you get to describe your own site in your own words.

The practical value is not ranking. It is correction. A model that has only seen scraped fragments of your site will guess what it does. A well-written llms.txt replaces that guess with your own description.

Why absolute URLs are the default now

We parsed every markdown link in the 22 files we collected on 2026-09-27: 7,268 links, and 7,268 of them are absolute. Not one relative path. That is not a narrow majority — it is unanimous, and it is the strongest convention in the whole format.

The reason is practical rather than stylistic. An llms.txt is read out of context: pasted into a chat, pulled into an index, or fetched by a crawler that never requested the host. In all three cases a relative path has nothing to resolve against, and the reader is left with a string it cannot turn into a URL.

This page used to default to relative paths, on the argument that they survive a domain move. That argument is real but it is the rarer failure. We changed the default when the count came back unanimous; the relative option is still there, and it is the right choice when you serve identical content from several hostnames or mirror the site on a staging domain.

Our own llms.txt adoption survey covers the other half of this — who publishes the file at all, and the redirects and soft 404s that hide half of them.

What the 22 real files look like

The median file is 15,010 bytes, 74 links and 115 lines. The smallest is 2,818 bytes (Netlify, 27 links); the largest is 1,724,189 bytes — 1.7 MB — from Search Engine Land, which lists 5,349 blog posts with Published, Last Modified and Excerpt sub-bullets under each. That file is a sitemap wearing a different name.

Skeleton: 21 of 22 start with an H1 on the very first line, and 20 of 22 follow it with a blockquote one-liner describing the site. That # Title then > one-line summary shape is the closest thing the format has to a house style, and this generator produces it.

Below that there is no shared vocabulary. The 22 files use 203 different H2 headings between them, and only one word appears more than three times: Optional, in 10 of the 22. 'Documentation' and 'Solutions' appear three times each; everything else once or twice. Pick section names that describe your content — nobody is matching on them.

How big is too big

Across the sample, the median file spends 199 bytes per link. A 74-entry file at 15 KB is roughly 4,000 tokens, which fits comfortably anywhere. The question is not whether your file fits in a context window; it is whether a reader that has to choose will keep the whole thing.

The 1.7 MB outlier is instructive because nothing about it is broken. Every link is valid, every post is real. But a reader with a budget will truncate, and truncation keeps the head of the file — so a file that large spends its most valuable position on whatever happened to be published most recently.

A working rule: keep the index under about 100 entries. If your content genuinely needs thousands, put a curated index in llms.txt and the full text in llms-full.txt, which is what the two-file split exists for. We also found 111 duplicate URL occurrences across the 22 files — the same link listed under two sections — which is pure waste in a file that is read for signal.

Writing entries that are actually useful

The description field is where most of these files fail. 'Home', 'About us', and 'Our services' tell a reader nothing it did not already know. Write what the page does: 'Convert SubRip subtitles to WebVTT, preserving timing and cue numbering.'

Be specific about constraints, because they are the thing a reader cannot infer. If a tool only handles certain input formats, or if output is approximate, say so in the description rather than leaving it to be discovered.

Order matters less than completeness, but put the pages you would want someone to see first at the top. A reader that truncates will keep the head of the file.

The description line: what the data says

Measuring per site rather than pooling, the median site attaches a description to 100% of its list entries. Descriptions run a median of 12 words, and the shortest and longest per-site medians are 4 and 22 words. Short is normal; a paragraph is not.

Four sites are the exception, and they are the ones worth learning from: Search Engine Land omits the description on 100% of its 5,349 entries, Etsy on 94%, Heroku on 76% and Yoast on 53%. Heroku's entries look like - [About Heroku](https://www.heroku.com/about/) — no description at all. When the title is already descriptive that is defensible; when the title is a navigation label like 'About', the description was the only content in the line.

Two numbers get misread here. Pooling all 7,268 links gives a 23% description rate, which looks like a widespread failure — but that single 5,349-entry file is 74% of the sample. Per-site medians are the honest statistic. The same trap applies to .md links: pooled, 77% of links point at a Markdown or text version, but the median site points 0% of its links there. Only 9 of the 22 files link to .md at all, and they are documentation sites — Node.js, Cloudflare, Netlify, Vercel. Pointing at Markdown is a docs convention, not a general one.

llms-full.txt: five sites do it, one pretends to

Five of the 22 adopters also serve llms-full.txt: Shopify, GitHub, Atlassian, Coinbase and Cloudflare. Cloudflare's is 166 KB against a 17 KB index — about ten times the size, which is exactly what the split is for: a small index that fits anywhere, plus a full-text version for readers that want all of it.

A sixth, Vercel, returns 200 for llms-full.txt with 2.6 MB of HTML. That is a soft 404 — the status code says success and the content says otherwise. If you check your own file with a status-code-only check you will make the same mistake, and so will anyone else's.

One caution from the same data: GitHub's two files are 28,650 and 28,658 bytes, eight bytes apart. We only kept the body of llms.txt, so we cannot confirm they are identical, but two files that close in size are not an index and a full version. If your llms.txt already contains everything, publishing a copy under a second name adds nothing.

Checking the file after you publish it

Publishing is where most of these go wrong, and the failure modes are all invisible from inside a browser. Four checks, in order:

Status and type. curl -sI https://yoursite.com/llms.txt should return 200 with a text content type. A 200 that returns HTML means a catch-all route is swallowing the path — this happened to six of the 78 domains we surveyed, including Reddit and Pinterest.

Bare host vs www. curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://yoursite.com/llms.txt. Eleven of the 22 adopters only become visible after a www redirect, so a file that answers on one host and 404s on the other is half-published.

BOM. curl -s https://yoursite.com/llms.txt | head -c 3 | xxd should not print efbbbf. Four of the 22 files start with a UTF-8 BOM, which turns the first H1 into a plain paragraph for strict Markdown parsers. If you generated the file on Windows, check this one.

Entry count. grep -c '^- \[' llms.txt. Compare it with what you meant to publish — a truncated paste is the most common way entries go missing. For redirect behaviour specifically, our redirect chain survey shows how often the first hop is the one that breaks.

When you should not publish one

If your site has no stable set of pages worth handing over, an llms.txt is a snapshot that is stale the day you write it. News homepages, social feeds and stores whose inventory turns over hourly are in this group — and they are well represented among the 39 of 78 domains that return a clean 404. For most of them that is the correct answer, not an oversight.

If you cannot keep it updated, don't start. A stale list does not merely fail to help; it asserts things that are no longer true, and a reader has no way to tell which lines have rotted.

And if your actual goal is to keep crawlers out, none of this applies. That is robots.txt, and for AI crawlers specifically it is the per-bot rules covered in our AI crawler survey. llms.txt describes; robots.txt permits.

What this file will not do

It will not improve your rankings, and no search engine currently reads it as a ranking signal. Treat it as documentation for machines, not as an SEO tactic.

It also does not replace robots.txt. If you want a crawler to stay out, that is still robots.txt — the robots.txt tester is the tool for checking that file.

Related tools