How this was measured
On 2026-09-27 we requested https://<domain>/llms.txt for the same 78 hand-picked domains used by our other surveys โ news, commerce, SaaS, developer tooling, social, streaming, reference, SEO, education, finance, cloud, local and government. Every request was attempted twice: once over a direct connection and once over a local proxy, because many of these domains are unreachable from our network without one.
A file counted as adopted only if all three things were true: the final status was 200, the response body was not HTML, and the URL we landed on still had /llms.txt as its path. That third condition is the one that matters most and the one most casual checks get wrong. 51 of the 78 domains answered the bare-host URL with a redirect, and 50 of those were ordinary www normalisation โ https://bbc.com/llms.txt → https://www.bbc.com/llms.txt. A checker that does not follow redirects reports 51 sites as "no llms.txt" when 11 of them have one sitting right there.
A checker that follows redirects blindly is wrong the other way. Two domains redirect /llms.txt somewhere else entirely and serve a 200 when they get there, and both would be counted as adopted by anything that only looks at the status code. We excluded them and list them separately below.
Finding 1: 28% adoption, and half the sample says 404
22 of 78 domains (28.2%) serve a real llms.txt. The remaining 56 break down like this:
39 (50.0%) return a clean 404. Not a soft 404, not a redirect โ an honest 404. This is the majority answer, and it is worth saying plainly because most writing about llms.txt implies adoption is further along than it is. Half of well-known sites have not engaged with the file at all.
6 return 200 with an HTML page. trello.com, reddit.com, pinterest.com, twitch.tv, khanacademy.org and linkedin.com. Five of them are single-page-app or CDN fallback behaviour: the route does not exist, so the framework renders its shell and returns 200. khanacademy.org answered with a 3 KB bot-challenge page, the same response this domain gave in our earlier surveys. linkedin.com is the reverse case: the bare host answered 200 with HTML, but following the redirect produced a 404.
2 redirect to somewhere that is not llms.txt. spotify.com lands on /int/why-not-available/, which is a geo block โ we fetched from a Chinese IP address, and that page is what Spotify serves this region. cdc.gov lands on /index.html. Both return 200, so both look like success to a status-code check, and neither has a file.
9 could not be determined. reuters.com answered 401, bloomberg.com, npmjs.com, netflix.com, yelp.com, nih.gov and gitlab.com answered 403, uber.com answered 406, and washingtonpost.com timed out on both paths. These are excluded from the adoption rate rather than counted as non-adopters, because we genuinely do not know.
Finding 2: the split is about audience, not about size
Aggregate adoption hides the actual signal. Broken out by category, the 78 domains divide almost perfectly along one line โ whether the site's content is documentation-shaped:
cloud 4/5 (Cloudflare, Vercel, Netlify, Heroku) · finance 4/5 (Stripe, PayPal, Coinbase, Square) · SEO 4/7 (Semrush, Yoast, Search Engine Journal, Search Engine Land) · SaaS 5/9 (GitHub, Atlassian, Slack, Notion, Asana) · commerce 3/6 (Etsy, Shopify, Target).
And on the other side: news 0/8 · social 0/6 · reference 0/5 (Wikipedia, Mozilla, W3C, Archive.org, Wikimedia) · streaming 0/5 · local 0/5 · government 0/3. Developer tooling is the surprising one at 1/9 โ only nodejs.org of Stack Overflow, npm, Docker, Kubernetes, Python, Rust, Go and MDN.
The pattern is not company size and it is not technical sophistication. W3C publishes the specs the web runs on and does not serve an llms.txt; Etsy does. What separates the two groups is whether the site has a body of reference material it wants a machine to read. Stripe, Cloudflare, Vercel, Netlify and Heroku all sell to developers and all maintain large documentation sets โ a curated index of that documentation is a product asset. A news homepage has no stable canonical set of pages to hand over, and a social feed has even less.
If you are deciding whether this file is worth publishing, that is the question to ask: do you have a set of pages a machine should read instead of your marketing copy? If yes, you are in the group that adopted. If no, you are in the group that did not, and the group that did not is larger.
Finding 3: the files are much bigger than the proposal suggests
The median file among the 22 adopters is 15,010 bytes with 74 links. The smallest is Netlify at 2,818 bytes and 27 links. Both are far larger than the short, hand-curated index the original proposal describes.
The largest is searchengineland.com at 1,724,189 bytes across 21,440 lines with 5,351 links โ 115 times the median. That is not a curated index; it is a generated dump of the site's content. Whether an automated export serves the purpose better than a hand-written one is not something this survey can settle, but it is worth naming, because the median is being pulled by files like this and "typical" is doing a lot of work in the number above.
Finding 4: four files break their own heading with a byte-order mark
21 of 22 files open with an H1. 22 of 22 have at least one ## section. 10 of 22 include the optional ## Optional section the proposal defines. Structurally, adoption means following the format.
But four files โ searchenginejournal.com, searchengineland.com, paypal.com and heroku.com โ begin with a UTF-8 byte-order mark (U+FEFF) sitting immediately in front of the #. The character is invisible in an editor, which is exactly why it survives review.
We tested what it does rather than assuming. Rendered through Python-Markdown, "\ufeff# Hello" produces <p># Hello</p> โ a paragraph โ while "# Hello" produces <h1>Hello</h1>. The heading is lost. CommonMark, which most markdown parsers implement, does not classify U+FEFF as whitespace, so this is not specific to one library.
One file has no H1 at all: notion.so opens with a blockquote description and no title heading. That is a deliberate-looking deviation rather than an error, and it is the only one of the 22.
The fix for the BOM is one setting. Save the file without a BOM โ in most editors it is a toggle in the save dialogue, and if the file is generated by a build step, check whether your templating layer is prepending one. Then verify by rendering the first line rather than by looking at it.
Finding 5: the companion file is rare
The proposal describes a second file, llms-full.txt, holding the complete content rather than an index. 5 of the 22 adopters publish it: cloudflare.com (166,160 bytes), atlassian.com (35,333), github.com (28,658), shopify.com (18,863) and coinbase.com (8,926). The other 17 publish the index alone.
Cloudflare's is eleven times the size of its own llms.txt, which is what the two-file split is supposed to look like. That it is the exception rather than the rule says the pair is being treated as one deliverable, not two.
What to do with this
Check your own site by following redirects and checking the final path. Half of the adopters here are behind a www redirect, and two non-adopters return 200 on a path that is not /llms.txt. Status code alone will give you the wrong answer in both directions. curl -sSL -o /dev/null -w "%{http_code} %{url_effective}" https://example.com/llms.txt gives you both halves.
Make sure it is served as text, not rendered by your app. Six of the sites here serve an HTML shell at a route that does not exist. If your framework catch-all route answers unknown paths with a 200, your llms.txt route is already broken in the same way โ and so is every future route you add.
Save it without a BOM. Four of 22 failed this, and the failure is invisible in every editor that does not show invisible characters.
Keep it an index, not a dump. The median file here is 15 KB and 74 links. The one file that ran to 1.7 MB and 5,351 links is an outlier for a reason: the value of the format is curation, and a dump removes the reason to read it.
Decide using your content shape, not the trend. Adoption clusters almost entirely in companies with documentation to hand over. Half of well-known sites have not published one and are not obviously worse off. If you have a stable set of pages a machine should prefer over your marketing copy, publish it โ the llms.txt generator on this site writes the structure and lets you paste in your own links. If you do not, the 39 sites returning a clean 404 are the more honest precedent.
Limitations
78 hand-picked, well-known domains, not a random sample of the web. Categories are our own manual grouping, several contain five domains, and a single adoption moves a rate by 20 points โ read the per-category counts as descriptions of these specific sites, not as estimates of a category.
All requests came from one IP address in China on one day. That is directly visible in the data: spotify.com redirected to a regional unavailability page, and nine domains answered 401, 403, 406 or nothing. Those nine are excluded rather than counted as non-adopters. A different vantage point would almost certainly find a few more adopters among them.
We read the initial HTTP response only. A file served by a worker, an edge function or client-side routing would not appear here, and none of the exclusions were tested a second time by a different method.
The BOM finding was verified with one parser, Python-Markdown. We did not test every parser, so treat "the heading is lost" as demonstrated for that one and likely for other CommonMark implementations, not as measured across all of them.
Nothing here says llms.txt affects search ranking. It is a proposed convention, not a standard, and no major search engine has said it uses the file. This survey measures what sites do, not what doing it earns you.
Reproduce it
The script and the shared domain list ship with this site. node scripts/survey-llmstxt.mjs fetches every domain and stores the raw responses; --report re-derives every number from that stored data without touching the network; --evidence prints the first three lines of every file judged to be a real llms.txt, which is how we checked the 22 verdicts above; --refollow and --refull re-run individual passes. Our robots.txt, canonical, hreflang and AI crawler surveys use the same domain list from different angles.