Home โ€บ Guides โ€บ How AI Search Engines Choose What to Cite

How AI Search Engines Choose What to Cite

AI search engines cite pages they can find, understand, and trust. Here is what makes a page citable, and how this site is built to be one.

The short answer

An AI search engine cites a page when three things are true at once: the page can be found and read by the system doing the answering, the page contains a fact stated plainly enough to be quoted, and something independent suggests the page is worth trusting. Miss any one of the three and the page is invisible to the answer, however good its content is.

That is the whole mechanism in one sentence, and it is worth holding on to, because most advice about "optimising for AI search" is really advice about one of these three conditions dressed up as something new. There is no separate secret ranking for AI answers. There is a retrieval step, a reading step, and a trust step, and each of them is mundane.

One more thing to say up front: no major AI company has published the exact formula its system uses to choose citations. Anyone who quotes precise weights, or a list of confirmed ranking factors for AI answers, is guessing. What follows is the behaviour that is observable from the outside โ€” what the systems demonstrably do, and what site owners can verify for themselves.

First, the page has to be found and read

An AI answer is built from pages the system retrieved. Retrieval needs the page to be crawlable: a live URL, a 200 status, no robots rule blocking the AI crawler, and content present in the served HTML rather than injected only by JavaScript after load. A page that renders beautifully in a browser but ships an empty shell to a crawler has given the retrieval step nothing to work with.

This is the same crawlability that traditional search needs, and the checks are the same. Fetch the URL with curl and read what comes back. If the body is an empty container and a bundle of script tags, a retrieval system may never see the words you wrote. If your robots.txt blocks the AI crawlers by name, you have told them not to read the page at all โ€” and several sites do this deliberately, because being cited is a choice, not an obligation.

The llms.txt file sits in this same step. It is a plain-text index, published at the root of a site, that lists the pages the site considers important, each with a one-line description. It is a proposed convention, not a standard, and no major AI company has formally committed to reading it. Treat it as cheap insurance: it costs minutes to publish, and any system that does look gets a curated map instead of having to guess which pages matter. Our own survey of 78 well-known sites found 22 serving one, which tells you adoption is real but far from universal โ€” see the llms.txt survey for the numbers.

Second, the answer has to be quotable

Retrieval finds candidate pages; the system then has to extract an answer from them. The pages that get cited are usually the ones where the fact is stated in a form that can be lifted out intact: a short paragraph that answers one question, a definition in a single sentence, a table with labelled rows, a list of steps in order.

This is why the structure of a page matters more for AI citation than its length. A 3,000-word essay that never states its conclusion plainly is harder to quote than a 600-word page that opens with the answer and then explains it. The first paragraph of this guide is written that way on purpose: it is the answer, stated before the explanation, in language that survives being lifted out of context.

Practical consequences follow. Put the conclusion first, before the background. State numbers and dates as plain text, not only inside images or charts. Use headings that describe what the section answers, so a system scanning the page can tell which part is relevant. And avoid burying the key fact in the middle of a long paragraph where it cannot be extracted without the sentences around it. None of this is new writing advice โ€” it is the same clarity that helps human readers โ€” but AI retrieval rewards it more directly, because a machine cannot skim the way a person can.

Third, someone else has to vouch for the page

The hardest condition to fake, and the one most often skipped, is independent corroboration. A claim that appears on one site and nowhere else is a claim, not a fact. Systems that answer questions lean on pages that are linked to, listed in directories, mentioned in roundups, or cited by other pages on the same topic.

That is why the familiar signals still matter: links from real pages, listings in relevant directories, mentions in articles that are themselves trusted. They are not a separate "AI SEO" tactic. They are the same authority signals search engines have used for decades, doing the same job: telling the system that other people found this page worth pointing at.

It also explains why a brand-new page from an unknown site is rarely cited on its first day, even when its content is correct. The page has no history of being pointed at, so the system has no independent reason to prefer it over ten other pages saying the same thing. Time, links, and mentions are the slow part of this, and there is no shortcut that survives contact with a system designed to distrust shortcuts.

What this looks like on SerpPrism

This site is built around the three conditions above, and it is a useful worked example because every part of it is verifiable. Each tool page states plainly, in its first paragraph, that the tool is free, needs no sign-up, and runs entirely in the browser โ€” three facts a system answering "is there a free tool that does X" needs in quotable form. The site publishes an llms.txt file at its root listing the tools and guides with one-line descriptions, so any crawler that reads the convention finds a curated map. And the guides library, including our AI crawler survey, publishes dated measurements with the method and the raw counts, so a claim on this site can be checked rather than taken on trust.

Note what this example does not claim. We do not claim any AI system has cited this site, because we have not verified that it has. We do not quote a citation rate, a share of AI answers, or a survey of what AI systems prefer, because we have not measured any of those. The example is about the structure being in place, not about a result we cannot evidence.

That distinction is the point of this guide. The work is to make a page findable, quotable, and corroborated. Whether a particular system cites it on a particular day is its decision, and any report that tells you otherwise is selling certainty it does not have.

What does not help

Stuffing pages with the exact phrasing of popular questions does not help, because retrieval is not keyword matching and the system reads the page, not the query log. Adding a block of hidden text, white-on-white keywords, or an invisible FAQ does not help either, for the same reason it never helped in traditional search: the content is judged on what is visible and coherent, and hidden text is a spam signal wherever it is found.

Publishing an llms.txt file and changing nothing else does not help much either. The file is a map, not a destination. If the pages it points at are thin, unquotable, or uncorroborated, the map only helps a system find them faster and dismiss them faster.

Finally, beware any tool or service that promises placement in AI answers. Citation is not an inventory you can buy, and the systems that answer questions have no advertising slot for it. A service that charges for "AI visibility" is charging for work you could do yourself โ€” writing clearly, structuring pages so answers can be extracted, and earning mentions โ€” with a guarantee attached that nobody can honour.

What to do on your own site

Check crawlability first. Fetch your key pages and confirm they return 200 with the content in the HTML. Check whether your robots.txt blocks AI crawlers, and decide deliberately: blocking is a valid choice, but it should be a choice, not an accident.

Rewrite the opening of each important page so it answers its own question. One paragraph, plainly stated, before the background. If you cannot write that paragraph, the page does not yet know what it is about.

Publish an llms.txt file. List the pages that represent what the site does, each with one line saying what it is for. Fifteen useful entries beat two hundred exhaustive ones. The llms.txt generator on this site writes the format for you.

Earn the corroboration. Get the page listed where people in your field actually look, and linked from pages that already get read. There is no faster honest version of this step.

Re-check after every redesign. A new template that moves content behind JavaScript, or a robots.txt edit that blocks a crawler group, can silently undo all four steps above. The failure is invisible in a browser, which is why it has to be checked rather than assumed.

Limitations

Everything here is inferred from observable behaviour, not from a published specification, because no such specification exists. AI systems change how they retrieve and cite without announcement, so a tactic that works this year may not work next year โ€” the three conditions are structural, which is why they are the safest thing to build on, but they are not a promise.

There is also a measurement gap worth naming. We can verify that our own pages are crawlable, quotable, and listed, and we can verify that other sites publish llms.txt files. We cannot verify, from the outside, which pages a given AI system retrieved for a given question, because that retrieval is not observable. Any guide that reports such numbers without explaining how they were measured is reporting something it cannot know.

About the author

Hongtao Ren (ไปปๅฎๆถ›) โ€” Developer based in Xi'an, China. Builds browser-based tools and JetBrains IDE plugins. He built and maintains SerpPrism.

Corrections are the most useful thing you can send. If a tool or guide here gives you a wrong answer, that is a bug, not a judgement call โ€” use the contact page.

Questions

How do AI search engines decide which websites to cite?

They cite pages they can find and read, that state the fact plainly enough to quote, and that other sources corroborate. All three conditions matter, and no major AI company has published the exact formula.

Does publishing an llms.txt file get my site cited by ChatGPT?

Not by itself. It is a proposed convention no major AI company has formally committed to, so treat it as cheap insurance: a curated map for any system that does look. The pages it points at still have to be quotable and corroborated.

Should I block AI crawlers in robots.txt?

Only if you have decided you do not want AI systems reading your content. Blocking is a valid choice, but check that it is deliberate: a block on an AI crawler group also removes any chance of being cited.

Does writing longer content help AI citation?

Length by itself does not. A short page that states its answer plainly in the first paragraph is easier to quote than a long page that never states its conclusion. Structure beats length.

Can I pay to appear in AI answers?

No. There is no advertising slot for citations, and no service can guarantee placement. The work is writing clearly, structuring pages so answers can be extracted, and earning mentions from other pages.