Does llms.txt actually move AI citations, and how would you tell

Search for llms.txt and you get ten pages that tell you what it is. The specification, a Reddit thread, four tool vendors, a docs platform, a trade-press write-up and a video. Two of those pages put the real question in their own titles, some version of "and should you use it", and then answer a different one underneath. Nobody is being dishonest. They answer the definitional question because it is the one that can be answered.

The question a person actually types into a search box before a planning meeting is whether putting this file on their site will change what an AI assistant says about them. That question has no published answer. Not a contested answer, not a weak answer, no answer, and that includes from us.

This post is about why, what is actually deployed right now, and what a test that could settle it would have to look like. We did the measurable part ourselves. On 2026-09-02 we fetched forty named developer tool and SaaS documentation sites and recorded what each one serves to a machine: whether an llms.txt exists and returns real text, whether an llms-full.txt exists, whether a per page markdown mirror is reachable, whether an OpenAPI spec sits at the conventional path, what robots.txt says to four named AI user agents, and whether the documentation renders without JavaScript. Thirty four of the forty serve an llms.txt. That is the deployment picture and it is a fact. It is not, and this is the whole point, an answer to the question in the title.

Before any of the detail, one framing that makes the rest of the argument tractable.

"Does llms.txt work" is four different claims stacked on top of each other, and they need four different instruments. Almost every argument about this file is two people defending different rungs of the same ladder.

The file exists and is fetchable. One HTTP request per site settles it. This is the rung we measured, and it is the easiest to get wrong in a boring way, because a host that answers a missing path with its HTML error page at status 200 will pass any check that reads the status code alone.

A crawler requests it. Server logs filtered by user agent settle it, over a window long enough to mean something. Two people have published readings of this and they disagree by a wide margin, which turns out to be the most interesting fact on the whole term.

A model uses it to decide what to read. Nothing you can observe from outside settles this. It needs either a disclosure from the provider or a controlled probe that varies the file and holds everything else still. Google has published that it does not treat the file specially, which is a partial answer for one surface.

Adding it changed a citation outcome. This needs a frozen query set, a held out arm, a fixed window and a null distribution built from your own data. Nobody has published one. Every confident sentence you have read about llms.txt is either about one of the first two rungs or is a claim about the fourth with the evidence of the first.

The reason this matters operationally, rather than as a taxonomy exercise, is that the four rungs move independently. A file can be fetched thousands of times and change nothing. A site can have no file at all and be cited constantly, because it is cited through a forum thread and a comparison page it does not control. Collapsing four observations into one yes or no is how a team spends a quarter on the wrong rung.

We wanted the deployment picture before touching the argument, because most of the writing on this term reasons about a hypothetical site rather than about the ones that actually exist. So we went and looked at forty of them on one day, and recorded six things about each.

The method, in full, so you can repeat it or break it. Forty named developer tool and SaaS documentation properties, chosen because they are the sites a developer or an assistant is most likely to land on: infrastructure and platform vendors, database and observability companies, model providers, and the documentation-heavy end of the API market. For each one, six live HTTP requests with curl on 2026-09-02, between 18

and 18 UTC, five concurrent workers, three retries per request, gzip decoded, sending an ordinary desktop browser user agent. We recorded the status code, the content type, the byte size and the body, and classified each result rather than trusting the status alone.

Four controls were declared before the run, each naming the result it had to return, because a sweep with no control that has to fire is not a sweep. A synthetic path on a real host had to come back absent, and it did, at 404. A file we knew existed had to come back present, and it did, at 8,839 bytes. A client rendered page had to come back as JavaScript-only, and it did, returning 10 characters of visible text with no script executed. A server rendered page had to come back server rendered, and it did, at 8,644 characters. All four fired, which is the only reason any count below is worth reading.

Here is what came back.

Thirty four of the forty serve an llms.txt that returns real plain text, which is 85.0 percent. Twenty five serve a per page markdown mirror reachable by appending .md to the documentation URL, 62.5 percent. Seventeen serve an llms-full.txt, 42.5 percent. Eight expose an openapi.json at the conventional path off the documentation root, 20.0 percent. Thirty seven serve a readable robots.txt.

The shape of that result is the finding, more than any individual number. The index file is now the norm at this end of the market rather than an experiment somebody is trying. If your reason for hesitating is that shipping one would look eccentric, that reason expired. Four in five of the documentation sites a developer reads every day already have one.

Two of the forty answered a machine path with HTML at status 200. That is the soft 404 shape and it is worse than a plain 404, because a fetcher that checks only the status code records it as a hit, stores an HTML error page as if it were content, and reports a healthy deployment forever. We caught it by recording the content type and by testing whether the body started with an angle bracket. A monitoring setup that pings the URL and alerts on non-200 would have been green on both of those sites indefinitely.

Our own error, since it is the more useful half

The first run of this probe was wrong, and the way it was wrong is the reason the second run has retries in it.

Four sites returned a connection failure on the llms.txt request. Curl could not complete the request, reported a status of zero, and the classifier turned that into "absent". The headline came out at 31 of 40 rather than 34. Three of those four, checked by hand afterwards, serve a perfectly real llms.txt: one at 578 bytes, one at 12,552, one at 17,674. The fourth genuinely returns 404.

A failed request and a missing file produce identical output unless you separate them deliberately, and the error ran in the direction that made our number more interesting. A lower deployment figure is a better story. That is exactly the direction a measurement error is hardest to notice, and it is why the probe now retries three times with backoff and records a transport failure as unknown rather than as absence.

What this sample cannot tell you

It is a convenience sample and not a random one, and every limit that follows from that is real.

It cannot tell you what share of the web carries an llms.txt. Forty sites chosen because they are prominent documentation properties will over-report any practice that early adopters adopt first, and this is exactly such a practice. The 85.0 percent describes the leading edge of documentation engineering, not the web, and the gap between those two is enormous.

It cannot tell you whether any model requested any of these files. The probe reads configuration over HTTP. It has no access to anyone's server logs and it cannot see a retrieval.

It cannot tell you whether the mirror carries the same content as the page it mirrors. We checked that a markdown response came back. We did not diff it against the HTML, and a practitioner who scanned 300 sites reports that three of the seven markdown variants he found dropped body content the HTML carried, all three on the same platform.

It cannot tell you anything about the two sites we could not fetch. One documentation entry point returned 404, which probably means the URL moved rather than that the site is broken. One returned 403 to our user agent, which is a block and is recorded as unknown rather than folded into either column.

What would falsify the headline figure is straightforward: re-run the same list and get a materially different count. The site list, the script and the raw results are published with this post's research dossier, so that is a check anyone can run rather than a claim you have to take.

The thirty four files we found are not the same kind of thing, and treating them as a single category is a large part of why the evidence about them looks so contradictory. The size spread alone should stop anyone reasoning about llms.txt as though it were one artefact.

The smallest was 342 bytes. The largest was 2,329,071 bytes, which is over two megabytes. Thirteen of the thirty four came in under 5 KB, eighteen sat between 5 KB and 50 KB, and three were over 50 KB.

A 342 byte file is a signpost. It names a handful of entry points and expects the reader to follow links. A two megabyte file is a corpus dump, and whatever a model does with it, it is not the same operation. If a study reports that llms.txt files are rarely read, and a second study reports that they are used heavily, the possibility that they were looking at different artefacts wearing the same filename deserves a hearing before either is called wrong.

There is a second version of the same split. Seventeen sites also serve an llms-full.txt, which inlines the content rather than linking to it. So on those sites there are two machine surfaces with overlapping jobs, and two files means two chances to drift out of sync with the pages they describe. Nothing in the specification says which one a reader should prefer, and nothing on the sites we probed indicated a preference either.

One number from the probe surprised us enough that we went back and re-checked the parsing before believing it, and it survived that re-check. Twenty five of the forty serve both files, and the pattern underneath that count is stricter than we expected.

Twenty five sites serve both an llms.txt and a per page markdown mirror. Nine serve the index alone. Six serve neither. And zero, out of forty, serve a markdown mirror without also serving an index.

That is a strict subset, and on a sample of forty it is unlikely to be coincidence. Whatever teams believe about these two surfaces, they adopt them in one order, and the index comes first. Two readings are available and we cannot separate them with this data. Either teams treat the index as the entry point and the mirror as an enhancement on top of it, or, more prosaically, the documentation platforms that generate one generate both and the ordering is a product decision made by four or five vendors rather than by forty teams.

The second reading is worth taking seriously, because it would mean a large part of what looks like industry consensus about machine readable documentation is a default in somebody's build pipeline. If that is what is happening, then counting deployments measures platform market share more than it measures belief, which is a caution that applies to our own 85.0 percent as much as to anyone else's number.

The specification leaves the contents largely to you, and the size spread we measured shows teams have taken very different views of what that means. Since the evidence does not settle the question, here is the reasoning we would use, presented as reasoning rather than as a finding.

Write it for the reader that has a budget. Whatever fetches this file is going to keep a slice of what it gets and discard the rest, so an index that lists two hundred pages in no particular order spends its whole allowance on an inventory. The thirteen files under 5 KB in our sample are doing something defensible: naming a small number of entry points and letting links do the rest.

Order it by what a stranger needs first. Quickstart, authentication, the two or three endpoints that carry most of the usage, then errors. Not your navigation order, which is organised by product area because that is how your team thinks, and not alphabetically, which encodes nothing.

Annotate each link with one line about what is on the page. This is the part that gets skipped and it is the part that does the work, because it is what lets a reader decide whether to spend a fetch. A bare list of forty URLs and titles is a sitemap wearing a different filename, and you already have a sitemap.

Include the error strings. This is the least common suggestion here and we think it is the most valuable, because a developer who hits an error pastes it into an assistant before opening any documentation. A page that names the literal error text, and an index entry that names it too, is reachable by the query people actually run. Nothing in the specification asks for this and nothing prevents it.

Leave out the marketing pages. A pricing page in an index intended for a machine reading your documentation is a fetch spent on something that does not answer a technical question, and if the reader is a coding agent it is worse than neutral.

Then date it and put a refresh on a schedule, because the stale index is the failure mode that grows on its own while the file keeps returning 200 perfectly.

None of that is validated. It is derived from what the file has to survive rather than from any measurement of what a model does, and we are labelling it that way rather than presenting a set of preferences as a result. If somebody publishes a controlled comparison of index structures, we will take their answer over this reasoning without hesitation.

Sometimes, and the two best published readings disagree with each other by a wide margin. Here is the most useful pair of facts on this entire term, and no page currently ranking for it puts the two of them side by side, which is why the question feels unsettled.

In June, an analysis of the Ahrefs llms.txt study across 137,000 sites reported that 97 percent of llms.txt files are never read. It added two details that matter more than the headline. Zero AI bots go looking for llms.txt files that do not exist, meaning nothing is probing for the file speculatively. And 12 percent of the requests that do arrive come from tools studying the format rather than from anything consuming it, which means a naive log count over-reports real usage by roughly an eighth before you start.

In August, an operator published the opposite reading from his own logs. More than a million AI bot hits across thirty days on a property that shows up in AI answers, and his summary was that OpenAI and Anthropic used llms.txt heavily to answer users, index pages and train models. He added that markdown pages were crawled about half the time and that the bots still used HTML, which is a useful corrective in its own right.

Two server-log readings. Both first-hand. Both from people with no obvious reason to misreport. Opposite conclusions.

The reconciliation is not that one of them is wrong, and this is the paragraph we think is missing from the public conversation. A 137,000-site population assembled from a crawl is dominated by its middle: sites that publish an llms.txt because a plugin generated one, that no assistant was ever going to fetch, that rank for nothing an assistant answers. Of course 97 percent of those are never read. A single property that already surfaces in AI answers and publishes machine readable content is the far tail of that same distribution. It is not a counterexample to the study. It is the two percent, describing itself.

Which means the honest answer to "do AI crawlers read llms.txt" is conditional rather than universal, and the condition is about you rather than about the file. If assistants already reach your site, they may well request the file. If they do not reach your site, the file changes nothing, because nothing goes looking for it. The Ahrefs finding that zero bots probe for files that do not exist is the load-bearing sentence there, and it is the one that gets quoted least.

That has an uncomfortable implication for the way the file is usually sold. It is offered as a way to get discovered. The evidence, such as it is, points the other way: it is a way for something that already finds you to find you more efficiently. Those are different products and they belong to different stages of a programme.

There is exactly one genuine experiment in public on this question, and it deserves better treatment than either the sceptics or the advocates currently give it. It is a real four-arm comparison with a stated outcome measure, and the outcome it measured is not the one people quote it for.

A documentation platform served the same content four ways: raw HTML, plain markdown, markdown with an llms.txt index, and markdown with the llms.txt inlined on every page. Then they had Claude Code and Codex navigate that documentation and counted every 404 each agent hit before it found the answer. HTML alone finished last by a wide margin, with 15 to 30 times more 404s than markdown plus an llms.txt index.

Take that seriously. It is a four-arm comparison with a stated outcome measure and a directional result, which puts it ahead of every opinion published on this term including, until we ran our probe, ours. Two boundaries have to travel with it, and neither is a reason to dismiss it.

The party who ran it sells the product the result favours. That is disclosed rather than disqualifying, and the correct response is to weight it as an interested party's result and to want a replication, not to throw it away.

The second boundary is the one that gets lost, and it is the more important. The outcome measured is 404s encountered by a coding agent navigating documentation. That is a NAVIGATION result. It says an agent that is already on your documentation site, already trying to find a specific page, wastes fewer requests when a machine readable index exists. It says nothing whatsoever about whether an answer engine cites you when a buyer asks a question somewhere else entirely. Reading "15 to 30 times fewer 404s" as evidence that llms.txt raises citations substitutes one question for another, and the substitution is invisible because both questions sound like "does it help".

The distinction between what ranks and what a model actually pulls is not hypothetical. Our own reading of citation sources found the overlap far smaller than teams assume.

Google has published that you do not need it, and that is the single most load-bearing counter-argument on this term. It appears in Google's own documentation, under a heading that says what you do not need to do, and the boundaries around that sentence matter as much as the sentence.

Google's own AI optimization guide puts it plainly: you do not need to create new machine readable files, AI text files, markup, or Markdown to appear in generative AI search. It adds that Google may discover, crawl and index many kinds of files in addition to HTML, and that this does not mean the file is treated in a special way.

That guidance surfaced in a technical SEO thread that ran to 211 upvotes and 106 comments, which is a fair indication of how much of the practitioner conversation it settled and how much it did not.

Now read the edges, because they are as informative as the claim.

It is a statement about Google Search and its generative features. It is not a statement about ChatGPT, about Claude, about Perplexity, or about any assistant that is not Google. Four surfaces, one of which has published a position. Treating that one position as settling all four is the mirror image of the vendor overclaim, and it is a mistake we see made by people who consider themselves sceptics.

It is also not a statement about coding agents. An agent working inside a repository, resolving a package name and going looking for a reference page, is a different reader running a different retrieval, and Google's guidance about its own search index has no bearing on it. That is precisely the reader the four-arm navigation experiment above was testing.

And a smaller point that is easy to miss: the sceptical position was itself tested rather than assumed. A practitioner probed whether Google's "ignores" was a hard rule and found that Google knows about these files, reads and displays information from them, and can cite them in responses, while concluding that none of this looked useful for ranking. That is the correct shape of a rebuttal. It separates "the file is fetched and can surface" from "the file helps you rank", concedes the second and establishes the first, and it does not turn a fetch into a lift.

While everyone argues about the suggestion file, the permission file sits quietly deciding whether any of it is reachable at all, and on most of the sites we probed nobody has touched that file with these crawlers in mind. Thirty two of thirty seven are running an inherited answer.

Of the thirty seven sites in our sample with a readable robots.txt, five name GPTBot in a group of its own. Four name ClaudeBot. Four name PerplexityBot. Three name Google-Extended. Everybody else inherits whatever the star group says, and the star group on a documentation site was written years ago for search crawlers.

Thirty two of thirty seven let a rule written before these agents existed decide what those agents may fetch. That is not a criticism, and in most cases the inherited answer happens to be permissive, so nothing breaks. It is a description of where the decision actually got made, which is not where anyone thinks it did.

One site in the sample makes the asymmetry visible. It names GPTBot and ClaudeBot and blocks both, while leaving PerplexityBot and Google-Extended to inherit an allow from the star group. Read that as a policy and it is incoherent: two of the four AI crawlers are refused and two are welcomed, with no principle separating them. Read it as history and it makes sense, because two bots were named at some point and the other two arrived later. Either way, that site is running two policies and one of them was never chosen.

The practical version of this is short. Robots.txt is four decisions, not one, and until you write four lines you have made none of them. Decide separately for GPTBot, ClaudeBot, PerplexityBot and Google-Extended, and know that Google-Extended is the Gemini opt out and does not affect Google Search, which is the single most common misunderstanding in this area.

There is one more asymmetry worth stating plainly. A robots block beats an llms.txt listing every time. If you list a page in your index and a rule elsewhere refuses the crawler, the page is not fetched and nothing about the listing changes that. Permission is upstream of preference, and a team that spends a week curating an index while a WAF rule quietly returns 403 to two of the four agents has optimised the wrong layer.

It fails in four ways that a status code cannot see, which matters because a status code is the check almost everyone runs. Each of these returns a perfectly healthy 200 while delivering nothing useful, and two of the four showed up in our own sample.

The soft 404. The path does not exist and the host answers with its own HTML error page at status 200. We found this on two of our forty. A fetcher reads a 200, stores the error page, and reports a healthy deployment. The fix costs nothing: check the content type, or check whether the body starts with an angle bracket, rather than checking the status alone.

The stale index. The file lists pages that have since moved. Every entry now costs a redirect or a miss, which is the opposite of the efficiency the file exists to provide. This is the failure that grows with time and it is invisible from the outside, because the index itself keeps returning 200 perfectly.

The partial mirror. The markdown version drops content the HTML version carries. A practitioner who scanned 300 sites found three of the seven markdown variants he identified dropped body content, all three on one platform, and nothing in either representation surfaces the difference. A reader taking the mirror gets a quietly reduced page and has no way to know.

The unvaried cache. Two representations served at one URL with no Vary header, so a cache in front of the site can hand the wrong version to the wrong visitor. The same scan found six of seven markdown variants missing that header. This one is a correctness bug rather than a documentation problem, and it is the sort of thing that produces a support ticket nobody can reproduce.

None of these is exotic and none of them is caught by a monitor that pings a URL and alerts on non-200. If you ship an index and a mirror, the check worth building is a weekly pass that resolves every link in the index, compares each markdown response against its HTML counterpart by word count, and asserts a text content type on both.

The reason this argument never resolves is that the two sides are standing at different points along a chain of five steps, and almost nobody names the chain out loud. A claim about the first link and a claim about the last link sound identical and are not.

Fetched. Something requests the file. Observable in your access logs and observable nowhere else. This is the rung the two log studies argue about.

Parsed. The links inside resolve to real pages. A stale entry breaks here, silently, and the file keeps returning 200 while doing so.

Retrieved. One of those pages is actually pulled for the specific question a buyer asked. Nothing about listing a page compels a retrieval on any particular query.

Used. The answer leans on that page rather than listing it beside five others. The gap between a decorative citation and a load-bearing one is real and almost nothing measures it.

Attributed. Your domain shows up in a source list a reader can see and click.

Published work on llms.txt stops at the first link, or occasionally at the third. The claim people want to make is about the fifth. That distance, four links wide, is the entire disagreement, and it is why two people can both be honest and both be certain and still contradict each other.

It is also worth being clear about what governs the middle of that chain, because it is not the index file. A page gets cited when it is retrievable, extractable, verifiable and consistent, and an index file touches the first of those four and none of the others.

A great deal of the confusion here is nomenclature. Four files come up in the same conversation, and they do different jobs for different readers with different consequences when they are wrong.

Robots.txt is permission. Every major crawler has honoured it for decades and the failure mode is that you block a reader you wanted. Sitemap.xml is enumeration, the search engines document how they read it, and the failure mode is slower discovery. AGENTS.md instructs a coding agent working inside your repository, it is read from a checkout rather than over HTTP, and the failure mode is that the agent makes the wrong change to your code. Llms.txt is preference, its adoption by any reader is unclear, and the failure mode is wasted fetches or nothing at all.

Three of those four are settled and documented by the parties who read them. The fourth is the one everybody argues about, and the argument is possible precisely because nobody who reads it has documented what they do with it.

Worth reading the llms.txt specification itself before repeating anything about it, because a surprising share of the claims attributed to it are not in it. It defines a file format and its intent. It does not promise a citation outcome, and it does not claim adoption it has not got. The trade press write-up of the proposal is similarly careful, describing it as proposed rather than as standard. Most of the overclaiming happened downstream of both.

If your reason for shipping an llms.txt is that coding agents will use it to find your reference pages, there is one behavioural study on that question and it points somewhere you may not expect.

An empirical study of how coding agents read documentation instrumented 557 agentic coding sessions, yielding 94,813 development events including 3,033 documentation interactions, alongside 33,097 agentic pull requests and 690,260 classified file-level change records. Instruction files and the agents' own working notes accounted for 60.5 percent of all documentation interactions. Classical technical documentation accounted for 10.6 percent. API references accounted for 1.3 percent.

Read those three numbers in order. The artefact getting nearly two thirds of the attention is a file in the repository, written by or for the agent, that has nothing to do with your documentation domain. The reference page you spent a quarter improving is the last of the three by a factor of eight.

The study also reports that the assumed loop of read the docs, then write the code, barely appears. The adjacent transition probability between a documentation consultation and a code edit is 0.002, and no explicit documentation-based validation sequence was observed at all. Consultation was self-initiated 70.2 percent of the time rather than triggered by a failure, and among pull requests that changed both code and documentation, code was touched first 4.7 times more often.

None of that says your documentation is worthless. It says that the specific story used to justify machine readable documentation files, in which an agent consults your reference before writing against your API, is not what these traces show. If you want the version of this question that applies to a human buyer rather than a coding agent, we have written separately about LLM SEO, what it is and what the work looks like and about the questions developer tool buyers ask AI.

Nobody has published a controlled result for any of the three engines, so the honest answer is that nobody knows, including us. Everything above is context. What follows is the answer to the title, and that answer is a test design rather than a number, because the number does not exist yet.

Start from the version most teams run, so it is clear what is wrong with it. You add an llms.txt on a Tuesday. Three weeks later somebody asks whether it worked. You open a tracker, look at a visibility score for the month before and the month after, see it up four points, and the answer becomes yes. That procedure is worthless, and it is worthless for two independent reasons that compound.

The first reason has nothing to do with your llms.txt and everything to do with the instrument.

We ran the cheapest possible version of this check on our own work. Twelve money queries, five identical repeats each, one model, one day, nothing changed between runs. Sixty calls, all sixty succeeded. Seven of the twelve queries changed outcome between identical asks. Not between Tuesday and Friday. Between two asks a few seconds apart using the same words.

That is an instability rate of 58.3 percent, and it is the number that makes a single before-and-after reading meaningless. If more than half your queries disagree with themselves under repetition with nothing changed, then a reading taken once before a docs change and once after is two draws from a noisy distribution, and the difference between them is mostly the width of that distribution.

We then built the null. Twenty thousand random splits of that same query set, with no intervention applied at all, recomputing the arm difference under each fake assignment. The null centred on zero, mean plus or minus 0.0016, standard deviation 0.215.

The zero mean is the design's kill test and it passed: an instrument that manufactures a difference out of nothing would not centre there. The 0.215 is the bar. It is how big a difference looks when nobody did anything, which is the number a real result has to clear, and doubling it gives a conventional threshold around 0.43. We have written the long version of this argument in why a single AI visibility score is noise, and the full procedure in how to track AI visibility, the procedure and the proof.

Apply that to the llms.txt question directly. A four point move on a monthly visibility score, with one reading on each side, is comfortably inside a distribution that a set with our instability produces while sitting completely still. It is not weak evidence. It is not evidence.

The second reason is worse, because it is invisible and it fires even when the first reason is handled.

Suppose you do everything right. You freeze a query set, you hold half back, you run a proper comparison, and you get a null result. It is tempting to report that as evidence llms.txt does nothing. Before you do, ask whether the design you ran could have detected a real effect if one had been there.

Computed from the variance in our own step-zero run, a design of 12 queries by 5 samples has a minimum detectable lift of 51.1 percentage points. Twenty by twenty reaches 21.4. Thirty by thirty reaches 11.4. Forty by forty reaches 9.9.

Now put a plausible effect size next to that ladder. If llms.txt does anything for citations, it is not going to move citation share by fifty percentage points. A generous guess is single digits to low double digits, and that is a guess rather than a measurement. Which means every design on that ladder except the last one is structurally incapable of seeing it, and a null from any of them is inconclusive rather than negative.

This is the trap that makes the whole topic feel unresolvable. Teams run underpowered tests, get nulls, and conclude the file does nothing. Vendors run uncontrolled tests, get movement, and conclude the file works. Both groups are reading noise, in opposite directions, with equal confidence. The sample-size arithmetic behind those floors is worth reading in full, and we have written it up separately in prompt tracking and the sample size it needs.

Note the shape of our own honesty here. We publish no causal lift figure for llms.txt, and we are not able to, because the pilot design we have run is roughly four times underpowered against the design that would reach 9.9 points. Publishing a lift off it would be publishing a number the design could not have produced.

Here is what a test worth running looks like. None of it is novel statistics. It is the ordinary apparatus of a controlled measurement, applied to a file.

Freeze the question set first. Around forty unbranded buyer questions, written down before anything ships and not edited afterwards. Never questions carrying your own brand name, because those are already won and cannot move. Never questions you picked after seeing which ones looked promising.

Baseline with repeats, not once. Ask every question several times. Five is a floor rather than a target at an instability rate like ours. The point of the repeats is that they turn each query into a proportion with a denominator instead of a boolean, and a proportion is the only thing you can put an error bar on.

Split the pages, at random, and hold half back. Half your documentation pages get listed in the index. Half get nothing, for the entire window. This is the step everybody skips and it is the step that does all the work, because it subtracts whatever the system did on its own during those weeks. Without it you are comparing your work against the calendar.

Change one thing. The index. Not the index plus a documentation rewrite plus a new navigation in the same window. Two changes in one window buy you one uninterpretable result, and the temptation to ship them together is strongest exactly when the window is expensive.

Re-read, then shuffle the labels. Compute the difference between arms. Then discard the arm labels, reassign them at random at the query level, recompute the difference, and repeat a few thousand times. Where your observed difference sits inside that distribution is the answer. Not whether it cleared a threshold somebody else chose.

Four checks before you believe your own reading. Did the instrument stay identical across the window, same wording, same repeat count, same engine, same model version string. Did the arms fail equally, because an asymmetric shock voids the comparison. Is the difference outside your own shuffled null. And could the design have seen a smaller effect, because if it could not, a null is inconclusive rather than negative.

If that sounds like a lot of apparatus for one file, it is, and that is the honest reason nobody has published this result. It requires deliberately not listing half your pages for a quarter, which is hard to defend internally and harder to keep when the quarter goes badly. That difficulty is the whole explanation for the evidence gap. It is not that the question is hard to think about. It is that the answer costs a quarter of restraint. This is the same discipline behind measuring answer engine optimization lift, and it is the part of the work that does not demo.

What the test costs, in calls rather than in feelings

Since the honest recommendation involves a quarter, it is worth pricing so the decision is a real one.

Take the design that could actually see a plausible effect: 40 queries by 40 samples, one engine. That is 1,600 calls per reading. Three baseline readings before anything ships gives 4,800. Twelve weekly readings across a ninety day window gives 19,200. Total 24,000 calls on one engine, and engines multiply before anything else does, so covering two doubles it to 48,000.

Those are metered calls landing on somebody's card, plus the storage, plus a scheduler somebody maintains when a provider changes a response shape. That is the real reason this result has not been published by anyone, and it is a better explanation than assuming the field is careless.

It also tells you when not to run it. If the decision behind the question is worth less than a quarter of measurement and the 24,000 to 48,000 metered calls above, do not run the arm. Ship the file, watch the logs, and hold the question open honestly rather than manufacturing an answer with a design that cannot produce one. Half-running this test is worse than skipping it, because it produces a number that looks like evidence and is not.

And one boundary on the arithmetic itself: those figures are derived from a stated design rather than measured on a running programme. Multiply them out yourself before quoting them. Converting calls to money depends on your provider and your model, and we are deliberately not asserting a single dollar figure here that would be wrong for most readers.

You will be shown llms.txt results by vendors and by consultants, and you now have enough to grade any of them without running a single call yourself. Five questions do almost all of the work, and the fourth one decides most cases on its own.

Five things to look for. Was the query set frozen before the change, unbranded, and published so you can read it, or is it unstated. Were there several readings per query with the repeat count reported, or one. Was the comparison a held out arm of comparable size, or the same pages in an earlier month. Was there a stated result that would have refuted the claim, written before the run. And what is the thing being called: an effect on one surface in one window, or proof that llms.txt works.

The fourth is the one that decides it. A test with no failing result defined in advance was never a test. There is a good public argument on exactly this point, made by showing that the evidence offered for llms.txt survives intact if you swap the filename for a nonsense one. That is a falsification test rather than an opinion, and an argument that cannot fail is not an argument. We take the same side on method while declining the conclusion, because "the evidence is bad" and "the file is worthless" are different claims and only the first is established.

Whatever it does, these are outside its reach, and confusing any of them for its job is how a team ends up disappointed by a file that was never going to help.

It cannot grant access. That is robots.txt, your CDN bot rules and your WAF. A listed page behind a block stays blocked, and the listing does not argue with the block.

It cannot make a page readable. If the text only appears after JavaScript runs, listing the URL changes nothing about what a non-rendering fetcher receives.

It cannot compel a fetch. It is a suggestion. No provider has committed to honouring it, and the Ahrefs finding that nothing probes for files that do not exist means it is not even a discovery mechanism for a site nothing already visits.

It cannot fix a wrong answer. A model repeating a parameter name you removed two versions ago is a training and retrieval problem. Adding a file that lists your pages does not reach into model weights and does not force a fresh retrieval on any particular question. If your problem is stale answers about your API, the relevant lever is content freshness and third-party corroboration, and we have looked at how quickly cited sources turn over in half of AI citations are under 13 weeks old.

It cannot be measured by its own presence. The file existing is not a result. It is the thing whose effect you have not measured yet. This is the most expensive of the five, because it is the one that closes the ticket.

Every discussion of machine readable documentation eventually reaches somebody saying that AI crawlers do not execute JavaScript, so a client-rendered documentation site is invisible. That is true as a statement about crawlers. On this end of the market it appears to be a solved problem, which was the most surprising result in our probe.

Thirty eight of the forty documentation entry points were fetchable, and 38 of 38 returned their content in the HTML with no JavaScript executed. Median visible text was 11,198 characters. The thinnest page returned 2,945 characters and the richest returned 77,152. Not one came back as a shell.

A uniform result across every member of a set is usually the instrument rather than the finding, so this one needed a control before we would report it. The control was a client-rendered application fetched by the same code path, which returned 10 characters of visible text. The check could have failed and did not. Two sites are excluded rather than counted: one documentation URL returned 404, and one returned 403 to our user agent, which is a block rather than an absence.

The practical reading is that if you are a documentation-heavy company on a mainstream documentation platform, you have probably already cleared this bar without deciding to, because the platforms pre-render. It still takes one curl to know rather than assume, and it is upstream of everything else in this post, because a page a fetcher cannot read is not helped by being listed in an index.

Pulling every failure mode in this post into one place, each paired with the specific step that would have caught it before it cost anything. Three of the eight were observed directly in our own forty-site probe, and two more come from another practitioner's scan.

The pattern across all eight is that the early ones are cheap to prevent and invisible to detect, while the late ones are expensive to prevent and obvious once you know. The soft 404 costs one line in a health check and will otherwise sit there for a year. The held out arm costs a quarter of restraint and cannot be added afterwards, because you cannot retroactively not-change half your pages.

Probably yes, as a cheap experiment rather than as a fix, and here is the recommendation broken into five parts with a confidence level attached to each rather than smoothed into one confident voice. The ordering matters more than any single item.

Ship the file. High confidence, low stakes. It costs an afternoon, 34 of the 40 documentation sites we probed already carry one, no published evidence suggests harm, and the downside if it does nothing is an afternoon. Ship it with a stated stopping point rather than as a fix, and do not let its presence close a ticket about AI visibility.

Serve a markdown mirror if your platform does it for free. Medium confidence. The one real navigation experiment favours it, the party running that experiment sells the product, and 25 of our 40 already do it. If it is a checkbox, tick it. If it is a month of engineering, that month has better uses.

Read your robots.txt as four decisions today. High confidence, and this is the one we would actually prioritise. Thirty two of thirty seven sites are letting a rule written for search crawlers decide for four AI agents. Twenty minutes of work, and unlike the index file it has a known mechanism and a known failure mode.

Look at your access logs monthly. High confidence. Filter by user agent, count requests for the file, and you have observed the one link in the chain that is directly observable. Almost nobody does this and it is nearly free.

Run the held out arm if the decision is worth a quarter. This is the only option that can come back negative, and it is the only one that produces a number rather than a belief. It is also genuinely expensive in organisational patience rather than in money.

What we would not do is spend an argument on this file. It is cheap and unproven, which is a perfectly ordinary place for a piece of infrastructure to sit while the evidence catches up. The failure is not shipping it or skipping it. The failure is calling a deployment a result, and then building a quarter of work on top of a number nobody measured.

If you want the causal answer rather than the deployment picture, that is the work behind answer engine optimization as we run it, and it starts with a frozen query set rather than with a file.

The limits of everything above, stated plainly, because a post whose central argument is that other people overclaim owes the reader its own boundary in the same detail. There are four of them and each one narrows a claim made earlier.

Our probe measures configuration and nothing else. Forty sites, one day, six requests each. It cannot see a retrieval, a ranking or a citation. Every number in it describes what a server returned to curl on 2026-09-02, and any sentence in this post that goes further than that is reasoning rather than measurement, labelled as such.

The sample is a convenience sample. It over-reports adoption relative to the web because it is drawn from prominent documentation properties, and the direction of that bias is known even though its size is not.

We have no causal result. Not for llms.txt, not for anything else. Our instrument passed its kill test and our pilot was underpowered, and both of those facts are published rather than implied.

The two log studies we cite are somebody else's, tagged published throughout, and we did not re-run either of them. Our reconciliation of them is an argument about sampling, not a measurement, and it is falsified if somebody demonstrates the two populations were comparable.

And the honest state of the question, one week before this was written, was put better by a practitioner than by any vendor page ranking for the term. He had read everything published on it, could not answer it, and named the exact observation that would settle it for him.

That is the reader this post is written for, and the answer he deserves is not a confident yes or a confident no. It is that the question is answerable, that answering it costs a quarter of restraint, and that until someone spends that quarter the correct posture is to ship the cheap thing and stop calling it evidence.