Making your docs agent-readable, and what agents read

There is a sentence going around developer marketing that sounds like advice and works like a slogan. Make your docs agent-readable.

It is repeated by platform vendors, by the cross-vendor convention's own site, by documentation tools, by consultants, and increasingly by the people who run engineering organisations. It has the shape of something you can act on. Write a file, describe your repository, and the coding agents that now sit between a developer and your product will understand you better.

The trouble is that the sentence carries at least four separate claims, they need four different kinds of evidence, and only the cheapest of the four has been established anywhere we can check.

So we went and measured the cheap one properly, because it is the one nobody had put a number on for this specific audience, and because a number you can check beats a sentence you cannot.

On 2026-09-12 we queried 40 named developer tool, CLI, MCP server and AI infrastructure repositories through the GitHub contents API and recorded what each one actually serves at its root. Not what its vendor recommends. Not what a survey says people intend to do. What is in the repository right now.

28 of the 40 carry an agent instruction file, which is 70 percent. 23 carry AGENTS.md, 23 carry CLAUDE.md, and 18 carry both at once. 31 carry a CONTRIBUTING.md written for human contributors, and 9 of those carry nothing equivalent for the reader that now arrives first.

40 of 40 resolved, so the 12 carrying nothing, 30 percent of the sample, are a measurement rather than a hole in the probe.

That last point is worth pausing on, because it is the difference between a count and a guess. A repository that does not exist and a repository with no agent file produce output that looks identical unless the code distinguishes them. We declared four controls before the sample ran and all four fired, including a negative control against a repository that does not exist, which returned HTTP 404 and was excluded from the denominator rather than counted as a zero.

Read the advice literally and it says nothing at all. Read it the way people act on it and it says four things, in ascending order of what it would take to believe them.

The first claim is that the file exists and is fetchable. This is cheap to establish, it is what we measured, and it is the only one in the list with a public number attached for this audience.

The second is that an agent actually reads it during a working session. Some vendors state that their agents do. Third-party samples have been reported in community threads and we have seen those reports. We did not verify them, we did not run them, and we are deliberately not restating their figures here as though we had. Reporting that something was reported is a different act from reporting a finding.

The third is that reading it changes what the agent then does. This needs a held out arm over the same task set with one variable moved. We could not find one published on a named public sample.

The fourth is the one every recommendation is actually about, even when it does not say so. It is that publishing the file changed whether a model cites you, recommends you, or gets your API right. That needs a frozen query set, a held out arm, a fixed window, and a null distribution to judge the difference against. Nobody has published that, and that includes us.

The gap between claim one and claim four is not a technicality. It is four steps of inference, and most of the writing on this subject makes all four in a single sentence while standing on evidence for the first.

Claim one, the file exists

This is the claim we can settle and the only one with a number in this post.

It is worth stating why even this rung needs care. The naive version of the probe asks GitHub for a file path and records whether the response was a 200. That version silently counts a repository that has been renamed, deleted or made private as a repository with no agent file, because both return a 404 through that route. On a sample of 40, 2 or 3 such cases would move the headline by 5 to 7 percentage points, and nothing in the output would show it.

So the probe reads the repository's root tree instead, matches file names case-insensitively against a declared list, and records a repository whose tree cannot be read as unresolved rather than as zero. 40 of 40 resolved, and README.md was present on 39 of them, which is 97.5 percent against a 90 percent sanity floor. The probe made 42 root tree requests, 40 for the sample and 2 for the controls, plus one extra call per repository that carried a .github directory. That is why the 12 carrying nothing can be reported as a finding rather than as a shrug.

The negative control is the one doing the work. It asks a question the positive control cannot: when the probe sees nothing, is that because there is nothing, or because it looked in the wrong place. A positive control proves the matcher can fire. Only a negative control proves that a zero means zero.

Claim two, an agent reads it

Here the evidence stops being ours.

Vendors publish that their agents read particular files, and those statements are worth what a vendor statement is worth, which is a description of intent rather than a measurement of behaviour in your repository. Community threads have reported third-party samples of what agents read during real sessions, with figures attached. We have read those threads. We did not run the study, we cannot see its method, and we are not going to restate its numbers here as though the act of quoting them made them ours.

This is a deliberate choice and it costs us something. Restating a striking third-party figure would make this piece feel more substantial. It would also mean that if the figure is wrong, we have laundered somebody else's error through our own credibility, and the whole argument of this post is that the laundering is the problem.

What we will say is the structural point, which does not depend on any particular study. A file being read is not the same as a file being followed, and both are invisible from outside the session. If you want to know whether your own agent reads your own file, the observation is available to you and not to us. Put a distinctive string in the file, give the agent a task that would surface it, and see whether it comes back. That is a one afternoon experiment on your own repository and it beats any survey.

Claim three, reading it changes the action

This is where the chain gets genuinely hard, and where the sceptical argument lands hardest.

An agent that reads your file and then does what it would have done anyway has read your file. The behaviour you care about is counterfactual, which means it cannot be observed from a single run. You need the same task attempted with and without the file, enough times to see past the variance, and coding agents are not deterministic.

That last point is underrated. We have measured repeat variance on a different question and found that 7 of 12 identical queries flipped outcome across 5 repeats on the same day with nothing changed, across 60 calls of which 60 succeeded. If a similar variance holds for agent task completion, and there is no reason to assume it does not, then a handful of before-and-after runs will produce a confident-looking result that is entirely noise.

Claim four, it changed a citation or a recommendation

This is the claim being sold, usually without being stated.

It is also the furthest from the evidence. Between your file and a model recommending your SDK sit the agent's read, the agent's decision, the training or retrieval pipeline of whatever answers the eventual question, and the query itself. Each one is a place the effect can vanish.

We sell work in this area and we still have no result on this claim. What we have is a answer engine optimization work built on a held out arm and a published account of how badly a small design can mislead you. That is the honest inventory.

The most useful thing the probe found was not the adoption rate. It was the overlap.

23 repositories carry AGENTS.md, which is 57.5 percent. 23 carry CLAUDE.md, the same 57.5 percent. Those two numbers being identical is a coincidence of this sample and not a law, but the third number is the one that matters. 18 carry both.

That is 45 percent of the whole sample, and 64 percent of the 28 repositories that have adopted the practice at all. The majority of projects doing this thing are doing it twice.

Two files, describing one repository, written at different times, usually by different people, with nothing in the repository asserting that they agree. When they disagree, the behaviour you get depends on which vendor's agent happened to run.

This is not a hypothetical failure. It is the exact complaint a platform CEO made publicly on 2026-08-25, and it travelled far enough that the same grievance topped a subreddit the following day. The engagement figures sit on the embed below, re-resolved through our own API rather than quoted from a screenshot.

Read what that is evidence of, and what it is not. It is a first-hand account of an operational problem from somebody running a large engineering organisation. It is not a measurement that either file changes a model's output. Those are different claims and this post keeps them apart deliberately.

What our probe adds is the frequency. The same complaint reached 1,303 upvotes and 252 comments when it was carried onto Reddit the next day. It is not one company's unusual situation. It is the condition most adopting repositories are in.

The failure mode is mundane and it is the one you can actually fix this week. A test command changes. Somebody updates the file their own agent reads. The other file keeps the old command. Six weeks later an agent runs the stale one, fails, and starts guessing.

31 of the 40 carry a CONTRIBUTING.md, which is 77.5 percent. 25 carry a docs directory, 62.5 percent. 39 carry a README, 97.5 percent.

These projects are not careless about documentation. They are among the better documented projects in their segments, which is part of why they were chosen, and several of them follow the canonical community definition of docs-as-code closely.

9 of them, 22.5 percent, carry a CONTRIBUTING.md and no agent instruction file of any kind. They have written down, carefully, how a human contributor should set up, build, test and submit. They have written nothing for the reader that now arrives first and does not read prose the same way.

That is the cleanest gap in the dataset, and it is the one that costs an afternoon rather than a quarter.

12 repositories, 30 percent of the sample, carry no agent instruction file at all, and it would be easy to write that up as a scolding. It is not one.

Several of the twelve are single-purpose SDKs, where the repository is one client library with one obvious entry point and a generated reference. The question a coding agent has to answer in that repository is much smaller, and the README may genuinely carry it. Others are large infrastructure projects with strong conventions and long-standing contributor documentation.

We are reporting a count, not a grade. What the count establishes is that adoption is a majority and not a consensus, which is a different and more useful fact than either "everyone does this" or "nobody does this".

The advice usually stops at write a file, which is where it becomes unactionable. Our probe cannot tell you what to put in it, because it counted presence rather than contents, and the prior art on contents is better read directly than summarised here. What we can offer is the shape of the problem, derived from what an agent is doing when it arrives.

An agent entering a repository has a budget and a question. The budget is context, which is finite and usually smaller than your documentation. The question is almost always operational rather than conceptual: how do I build this, how do I test it, what am I not allowed to touch, where does this kind of change go.

That is a different question from the one a README answers. A README is written to persuade a person that the project is worth their time and then to orient them, which is how the docs platforms themselves describe the practice. An agent is already committed and needs to act.

So the useful content is the content a new engineer would have to ask somebody for on their first day, and would not find written down anywhere.

The exact build command, including the flags that are not optional. Not the idealised one from the README, the one that actually works on a clean checkout.

The test command, and how to run a single test rather than the whole suite. An agent that can only run the entire suite will either burn ten minutes per iteration or skip testing.

The parts of the repository that are generated, and therefore must not be edited by hand. This is the single highest-value line in most instruction files, because an agent editing a generated file produces a change that looks correct, passes review, and disappears on the next build.

The conventions that are enforced and the ones that are merely common. An agent cannot tell the difference by reading the code, and it will faithfully reproduce whatever it saw most recently.

Where a given kind of change belongs, when the repository has more than one plausible home for it.

What to do when something fails, specifically. An agent that hits a failing build with no guidance will start changing things, and the things it changes first are usually the assertions.

Notice that every one of those is a fact about your repository that is true whether or not a model ever reads it, and that most teams have never written down. That is the argument for writing the file that does not depend on any causal claim, and it is the argument we actually believe.

What to leave out

Length is a cost here in a way it is not in a README. Context spent on your file is context not spent on your code.

Leave out the pitch. An agent is not evaluating whether to adopt you.

Leave out anything already obvious from the repository structure. If the directory is called tests, the file does not need to say that tests live in tests.

Leave out aspiration. A file describing the conventions you intend to adopt is worse than no file, because it will be followed.

Leave out anything that changes often and is not load-bearing, because every such line is a line that will drift and then mislead.

The three segments do not behave the same

Our forty repositories span three segments and they did not adopt at the same rate, which is worth a word of caution about our own headline.

Developer tools and APIs, the largest group, includes several client SDKs. A single-purpose client library has a smaller surface for an agent to get lost in, and the README often does carry the operational answer. Several of the twelve carrying nothing sit here, and we would not call that a gap without opening each one.

CLIs and MCP servers skew toward adoption, which is unsurprising given that this group builds the tooling agents run on and is closest to the convention as it forms.

AI infrastructure is the group where both files show up together most readily, which is also where the maintenance burden lands hardest, since these projects move fastest.

We are not reporting per-segment percentages, and the reason is a discipline rather than an omission. 40 repositories split 3 ways gives cells of roughly 13, where a single project moves the number by 8 points. A percentage computed on thirteen items invites a comparison the sample cannot support. The aggregate is the number this sample can carry.

Here is the part that most writing on this topic skips, so it is worth being explicit about our own position on it.

Our probe sits on rung one. We measured presence across a named sample with declared controls. That is a real result and it is a small one.

We did not measure whether any agent read any of those files. We did not measure whether a repository carrying one produces better agent output than one that does not. We did not measure citations, recommendations, or anything a buyer would call a result.

If that reads as a strange thing for a company selling AI-visibility work to publish, it is the whole position. The reason to trust a number is that the person publishing it tells you what it cannot support.

The reason rung one cannot carry rung four is that there is a chain between them, and every link is a separate empirical question.

You publish the file. Something fetches it. Something uses it to decide what to do. The action changes. The citation or the recommendation changes.

We can see the first link. Vendors can see the second and some of them state it. Nobody has published the third or the fourth on a public sample, and the fifth is the one being sold.

A claim about the last link resting on evidence from the first is not a lie. It is an inference, and inferences of that length are usually wrong in ways that are expensive to discover late.

The sceptical position deserves to be quoted rather than paraphrased, because it is the strongest argument on this topic and it is not being answered anywhere in the ranking set.

We think that argument is right about the claim and wrong about the chore, and the rest of this post is the case for that split.

There is a second-order cost that the advice never mentions, and it showed up in the corpus the day before we ran the probe.

A model release now ships with guidance to go back and revisit your instruction files. That is a reasonable thing for a platform to publish. It also means these files are not write-once, and every repository carrying two of them pays that cost twice, forever, on somebody else's release schedule.

The convergence argument is worth noticing here. A cross-vendor file name exists precisely so that one file can serve every agent. On our sample, adoption of the shared name and adoption of the single-vendor name are exactly level. Standardisation has not reduced the number of files. It has added one.

A repository can carry a perfect instruction file and still be unreadable to the thing you are hoping will read it, for reasons that have nothing to do with the file.

Access comes before content

If your documentation lives on a site that blocks the crawlers behind the assistants your buyers use, nothing you write in any file matters. That is a robots and infrastructure question, it is checkable in an afternoon, and it fails silently: the page returns 200 to you and nothing to them.

This is the most common version of the problem we see, and it is invisible from inside the company because every internal check passes. Nobody notices that the thing which is supposed to read the page never asked for it.

Check what your own logs say about which agents fetch what, before spending a quarter on content aimed at them.

Structure is read differently from prose

A model extracting an answer from your page is doing something closer to lifting a fragment than to reading. Content that survives that operation tends to share a shape: a direct answer near the question, a table where a table is warranted, headings that state the question rather than gesture at a theme, and units and dates attached to numbers rather than implied by context.

The head term behind this piece measures 320 searches a month at a keyword difficulty of 7, inside a cluster of roughly 1,010 a month. None of that is new advice and none of it is specific to agents. It is the same discipline that makes a page usable by a person in a hurry, which is one reason it is safe to follow while the causal question is open.

Examples do more work than description

An agent copies. This is the most reliable observation anyone has about how these systems behave in a repository, and it does not require a study to accept, because it is visible in any session.

That means a correct, current, runnable example is worth more than a paragraph describing what an example would look like. It also means a stale example is actively dangerous in a way stale prose is not, because it will be reproduced faithfully rather than read sceptically.

If you do one thing to your repository this month and you are choosing between writing an instruction file and fixing the three examples that no longer run, fix the examples.

Error messages are documentation now

When an agent hits a failure it reads the error, and the error is frequently the only documentation it consults at that moment. An error that names the fix is worth a page of prose that nobody loads.

This one sits outside most people's model of documentation entirely, and it is the place where a developer tool company has the most leverage and the least competition.

This is the most common response from teams who have invested in documentation, and it deserves a straight answer rather than a sales one.

Good documentation and agent-readable documentation overlap heavily, and the overlap is most of the value. The practitioner field has been sceptical about whether the docs-as-code promise held since well before agents were a reader, and that scepticism is worth reading on its own terms. A page with a clear direct answer near the top, correct runnable examples, real units on its numbers and headings that name the question is better for both readers. If you have that, you have done the majority of the work and the marginal gain from a dedicated file is smaller than the advice implies.

Three places the overlap breaks down are worth knowing about.

The first is that good documentation is frequently organised as a narrative, and narrative is the structure that survives extraction worst. A concept page that builds an idea across four sections is excellent for a person reading it once and close to useless to something lifting a fragment, because no single fragment carries the answer. That is not an argument for abandoning the narrative. It is an argument for putting a direct answer near the question and then developing it, which costs one paragraph.

The second is that documentation is usually written for the reader who is deciding, and an agent inside your repository has already decided. The tone, the framing and the ordering are all tuned for a question your reader is no longer asking. That mismatch is invisible when you read your own docs, because you read them as the author.

The third is the stale example problem, and it is the one that does real damage. A human reader who hits an example that no longer runs concludes the docs are out of date and works around it. An agent reproduces it, ships it, and the failure surfaces somewhere else entirely with no trace back to the cause. The asymmetry means an example that is merely old is a different category of risk than it used to be.

None of that requires a new file. It requires a pass over the things you already publish with a different reader in mind, which is cheaper than most teams assume and does not depend on any claim about citations.

A short checklist for the pass

Take your 5 most trafficked documentation pages and, for each one, check whether the first hundred words answer the question in the heading. If they set up a question instead of answering it, the page will lose every extraction it enters.

Check whether every code sample on those pages runs against the current version. Not whether it looks right. Run it.

Check whether your numbers carry units and dates in the text rather than in surrounding context. A figure that means something only because of the paragraph above it will arrive stripped of that paragraph.

Check whether the operational facts a new engineer needs are written down anywhere at all. Build, test, run one test, what is generated, where things go. If they live in somebody's head, the file this post is about is a good place to put them and the README is a better one.

Check what your logs say about which agents fetch those pages. This is the only item on the list that can invalidate the other four, and it is the one nobody runs first.

That checklist is deliberately unexciting. It contains no lever and promises no lift, and every item on it is independently useful whether or not the causal claim about agent files ever gets settled. Which is, in the end, the shape of the recommendation this whole post arrives at.

A post arguing for restraint should say which of the arguments against it land.

It is not falsifiable the way we are describing

The sharpest version of the sceptical case is that a held out arm on citation outcomes is close to impossible to run honestly, because the systems change under you. Retrieval pipelines update, models are replaced, and a six week window contains several of those events.

This is true and it is a real limit. It is an argument for shorter windows, larger query sets and permutation tests rather than for giving up, but anyone claiming a clean causal result on a six month window should be asked what changed upstream during it.

Presence counts are easy to misread

Ours included. Forty repositories chosen for being well known is a sample of the adopting edge. If somebody reads our 70 percent as the state of GitHub, we have misled them, and the phrasing throughout this post is an attempt to make that reading hard.

The practice may be good for reasons unrelated to models

We think this is right and we have said so twice. The clarity benefit is real, immediate, and does not depend on the causal claim. It is also the only benefit currently supported by anything.

Two files may be correct for a while

The strongest objection to our own main recommendation. If one vendor's agent reads only its own file, then carrying both genuinely maximises coverage, and collapsing to one costs you something real today in exchange for a consistency benefit.

We still think one source with a pointer is the better trade, because the failure mode of divergence is silent and the failure mode of a pointer is loud. But a team that carries both deliberately, with a check that they agree, has not made a mistake. A team that carries both because nobody noticed has.

None of this is hard to test. It is just that nobody has published the test.

Freeze the set first. Whether you are testing agent behaviour or citation outcomes, decide the task set or query set before you look at any result, and write it down. A set you adjust after seeing the data is not a set, it is a story.

Split it and hold an arm out. Apply the change to one half only. The half you leave alone is what tells you whether the difference you see is the change or the week.

Fix the window before you start. Decide how long you will measure for, in advance, and then measure for that long whatever the interim numbers look like.

Judge the difference against a null distribution rather than against zero. This is the step people skip and it is the one that does the work. Shuffle your own labels a few thousand times with no intervention applied and see how large a difference appears anyway. Our own run used 20,000 random splits. On small samples that number is usually much larger than people expect, which is why so many confident results are noise.

We have run that shape of test on a different question and published the measurement it produced, including the part where our own design turned out to be underpowered. A 12 query by 5 repeat run had a minimum detectable lift of 51.1 percentage points, roughly 4 times underpowered for the effect anyone cares about, and 7 of 12 queries flipped outcome across identical repeats on the same day with nothing changed. A permutation test over 20,000 random splits of the same query set, with no intervention applied at all, produced a null difference centred on zero, with a mean within 0.0016 and a standard deviation of 0.215.

Those two results are ours, they are published, and they are about measurement volatility rather than about agent files. We cite them here as the yardstick by which any claim on this topic should be judged, including anybody's claim about AGENTS.md, and not as evidence that agent-readable docs do or do not work.

Everything above is a count of other people's repositories, which is interesting and is not your situation. The version that matters takes about twenty minutes and answers a question only you can ask.

Start with what you serve. List your repository root and record which of the instruction files are present. If two are present, diff them, and specifically diff the four lines that drift first: the install step, the build command, the test command and the list of generated paths that must not be hand-edited. In our experience those four carry most of the divergence, because they are the lines that change when tooling changes and the lines nobody thinks of as documentation.

Then check the dates. A file last touched eleven months ago in a repository that has changed its test runner since is not a neutral artifact. It is a confident instruction to do the wrong thing.

Then look at what is missing rather than what is stale. Take the last 5 issues a new contributor opened that turned out to be environment problems rather than bugs. Every one of those is a line that belongs in the file, and it is a line you can write from memory because you answered it already.

That exercise produces a better file than any template, because it is derived from the questions your repository actually generates rather than from a generic list of headings.

Then measure the thing you actually sell

If you sell developer tools and APIs, the reason this topic reached your roadmap is probably not agent task completion. It is that buyers now ask an assistant which tool to use, and you want to be the answer.

That is a different measurement and it should not be conflated with this one. An agent reading your repository and an assistant recommending your product to a stranger are two different systems, reached by two different paths, and improving one does not obviously move the other. Your repository file is read by an agent already inside your codebase, which usually means somebody already chose you.

The recommendation question is answered somewhere else entirely: in the pages the model retrieves when somebody asks which tool to use, most of which you do not own. Treating your AGENTS.md as a lever on that is the category error this post is trying to prevent, and it is an easy one to make because both live under the heading of being readable by AI.

Measure them separately. Ask which queries your buyers actually type, freeze that set, and record what gets cited today, before you change anything. A baseline you did not take is a baseline you will invent later.

What a baseline actually requires

The temptation is to run the queries once, screenshot the answers and call that the before state. That produces a number that will not survive contact with the after state, for a reason we have measured directly rather than assumed.

Run the same query set 5 times on the same day against the same model with nothing changed, and a meaningful share of the outcomes will differ. On our own step-zero run, 7 of 12 queries flipped outcome across 5 identical repeats, which is 58 percent of the set. That is not a failure of the model and it is not a bug in the measurement. It is the variance that is always there and that single-shot measurement hides.

A baseline therefore needs repeats, and the number of repeats needed depends on how large an effect you are hoping to detect. A design that can only detect a 50 point swing will report nothing for a 10 point improvement and you will conclude the work failed.

This is the part of the job that gets skipped, and skipping it is why so much AI visibility reporting shows a line going up that nobody can attribute to anything.

One number in the probe is easy to skim past and it is arguably the most actionable thing in the dataset.

39 of the 40 repositories carry a README at the root. 1 does not, and it is a very large infrastructure project that organises its entry points differently.

Nothing else in the sample comes close to that. The agent instruction file sits at 28. CONTRIBUTING sits at 31. A docs directory sits at 25. Only 4 carry a copilot-instructions file, which is 10 percent.

The README is the only file you can rely on being present, being found, and being read, and it is the one file in the repository that predates this entire conversation. Which raises an uncomfortable question for the advice this post is examining: if your operational facts are not in the README, and the reader only reliably loads the README, what exactly is the instruction file buying you.

The honest answer is that it depends on whether the thing reading your repository looks for the instruction file by name, and that is a property of the agent rather than of your repository. Some do. It is not universal, the convention is three months into a standards argument, and a compiler project spent September debating whether to adopt it at all, in a thread that reached 240 upvotes and 61 comments.

So a defensible position, and one our sample cannot refute, is to put the operational facts in the README where everything finds them and to use the instruction file for the agent-specific material that would be noise for a human reader. That splits the content by audience rather than duplicating it by filename, and it degrades gracefully if the convention changes again.

We are not claiming this is better. We are pointing out that the sample contains a much stronger signal about the README than about either instruction file, and that nobody arguing about AGENTS.md against CLAUDE.md seems to have noticed the file that beats both on availability by a wide margin.

Where the convention actually is

It is worth being precise about the state of play, because the advice is often written as though it were settled.

A cross-vendor file name exists and has real adoption, at 23 of our 40. A single-vendor file name has exactly the same adoption in this sample. Neither has won. 18 projects are hedging by carrying both, which is a rational response to an unsettled convention and also the source of the defect we spent half this post on.

In the last three weeks a compiler project opened a public debate about adopting the cross-vendor name, a platform CEO threatened to ban an agent over which file it reads, and a model vendor published guidance telling developers to revisit their files. That is what a convention looks like while it is forming, and it means any recommendation phrased as a permanent answer is overstating its confidence.

The practical read is that the cost of being wrong here is low, because a pointer file is one line and can be reversed in a minute. Treat it as a reversible decision and stop spending meetings on it.

What this means if you also run a documentation site

Most of the companies asking this question have two surfaces, a repository and a documentation site, and they are frequently treated as one problem because both involve writing things down.

They are different problems with different readers and different failure modes.

The repository is read by an agent that is already working inside your code, usually because somebody already chose you. The operative question is whether it can act correctly. The failure mode is a wrong command confidently followed.

The documentation site is read, if it is read at all, by a retrieval pipeline answering somebody else's question, often before you have been chosen. The operative question is whether a fragment of it survives extraction and arrives attached to your name. The failure mode is silence, and silence is much harder to notice than a wrong command.

A team that fixes its AGENTS.md and leaves its documentation site unreadable to the crawlers behind the assistants has improved the experience of people who already use it and changed nothing about whether anybody new hears about it. That is a perfectly reasonable thing to do on purpose. It is a bad thing to do by accident while believing you addressed the second problem.

The measurement that tells you which problem you have is not complicated. Ask the questions your buyers ask, record what gets cited, and check separately whether your own logs show the relevant agents fetching your documentation at all. If the fetches are not happening, no amount of writing changes the outcome, and that is the cheapest thing on this entire list to check.

The one claim we will make without a caveat

Everything in this post that touches effect has been hedged, because the evidence does not support more.

Here is the exception. If your repository carries two instruction files that disagree with each other, you have a defect, right now, independent of whether any model ever reads either one. It is producing wrong behaviour for whichever agent reads the stale one, it is invisible in code review because neither file is code, and it affects the majority of the repositories in our sample that adopted the practice at all.

You do not need a controlled study to act on that. You need a diff.

There is a structural reason these files go stale that has nothing to do with discipline.

Your repository changes on your schedule. The convention for what agents read changes on somebody else's, and in the last month alone that included a new cross-vendor file gaining traction, a compiler project opening a debate about whether to adopt it, and a platform telling every developer to revisit their existing file for a new model.

Those two clocks are not synchronised and never will be. A team that treats the instruction file as a document written once at the start of a project is guaranteeing that it will be wrong, and the only question is when somebody notices.

The practical consequence is that this should be owned the way a build config is owned rather than the way a README is owned. It has an owner, it is reviewed when the things it describes change, and the review is triggered by the change rather than by a calendar.

That is a small process claim and it is the one recommendation in this post we would make with no evidence at all, because it is true of every file that describes a moving system.

It is worth naming the overlap, because the public argument is noisier than the actual disagreement.

Both sides agree the file is cheap. Neither side claims it carries a meaningful downside beyond the context it consumes and the maintenance it demands.

Both sides agree that writing it forces you to understand your own repository. The sceptical video that prompted that thread, 23,332 views at the time of writing, makes this point explicitly and treats it as the one genuine benefit. The vendor guidance makes the same point in more flattering language.

Both sides agree that nobody has published the controlled result. The disagreement is entirely about what to do in its absence.

That is a narrower dispute than the volume suggests, and it resolves cleanly once you separate the chore from the claim. Do the chore, which is cheap and independently useful. Suspend the claim, which is unevidenced and is the thing being sold.

One more measurement, because it changes what this page is for.

We probed the head term for this piece 5 times on 2026-09-12. An AI Overview fired on 5 of 5 usable trials, with a median of 11 references when it fired. We were cited 0 times out of 5. A second term returned only 3 usable trials of 20 attempted, below the 5-trial floor, so no claim is recorded for it.

That number cuts in two directions at once and it is worth saying both out loud.

It caps the traffic case. This is a query Google increasingly answers itself, so a page ranking on it will convert fewer clicks than the search volume suggests. Anyone selling you a traffic forecast on this term should be asked whether they checked.

It is also the entire reason to write. 11 sources are being pulled into that answer and none of them is us. A query with an AI Overview is a citation target rather than a traffic target, and the thing that gets pulled into a citation set is usually the page carrying the number nobody else has.

There is prior art on counting here, and it is good. A community post surveying what the hundred largest GitHub repositories put inside their AGENTS.md files reached 155 upvotes. It counts file CONTENTS. We counted file PRESENCE across a named sample in three specific segments. Those are different questions and we would rather say so than let a reader assume we are second in line on the same one. The rest of our work on this cluster sits under AI citations for developer tools and APIs.

Diff your two files today, if you have two. That is the defect you can find by reading your own repository, it does not depend on anybody's causal study, and on our sample it affects the majority of projects that adopted the practice.

Collapse to one source. Keep the other name as a one line pointer so you do not trade a consistency problem for a coverage problem while the vendors are still converging.

Put the file under the review your code gets. A stale instruction file is worse than no instruction file, because it is confidently wrong at the exact moment somebody trusts it.

Then, if you want to know whether any of it did anything, hold an arm out. Until somebody does that and publishes it, every claim on this topic including ours is standing on rung one.

Since the whole post argues for holding an arm out, it is only fair to say what that looks like concretely rather than leaving it as an instruction.

Start by writing down the question in a form that can come out wrong. Does adding an agent instruction file to this repository change the rate at which a coding agent completes a defined task set without human correction. That is answerable. Does it improve developer experience is not.

Build the task set from real work rather than from imagination. 20 to 40 tasks drawn from closed issues and merged pull requests, each with a definition of done that another person could check. Freeze it and write it down before running anything.

Split it. Half the tasks run against the repository as it is, half against the repository with the file added, and the assignment is made before you see any result.

Fix the window and the repeat count in advance. Every task run several times, because the variance is the thing that will fool you, and a single run per arm is the design that produces confident nonsense.

Then compute the null before you look at the difference. Shuffle the arm labels a few thousand times over your own results with no intervention applied and see what spread of differences appears from nothing. That distribution, not zero, is what your observed difference has to beat.

Two things usually happen when a team runs this for the first time. The effect they expected does not clear the null, and the exercise surfaces four or five genuine repository defects that were costing them far more than the file ever could. The second outcome is the one worth paying for, and it is available whether or not the first one arrives.

If that sounds like a lot of apparatus for a markdown file, it is. That is the actual finding. The claim is small, the honest test is large, and the gap between them is why the sentence has been repeated for a year without anybody closing it.

We publish measurement work, so the standard we hold other people's numbers to is the standard this page has to meet.

That meant three decisions readers can check. We did not restate a third-party figure we had not verified, even where it would have strengthened the piece. We declared our controls before running the sample rather than describing them afterwards. And we labelled every row of every table in this post with where the number came from, including the rows where the answer is that nobody knows.

The last one is the one that matters most, and it is why the tables here carry a provenance tag per row rather than a single source line at the bottom. A table with one citation underneath it tells you somebody had a source. A table where each row says measured, published or unknown tells you which claims survive if that source turns out to be wrong.

We would drop the sceptical framing tomorrow given a study with a frozen task set, a held out arm, a fixed window and a null distribution, run on a public sample, showing that adding or editing an agent instruction file changed what a model did or cited. The effect could be large or small; what would matter is that it was measured rather than asserted.

We would also update this page if our own sample turned out to be unrepresentative. 40 repositories is enough to say something about widely used developer tool projects and not enough to say something about GitHub. If somebody runs the same probe across a random draw of 10,000 repositories and gets a different adoption rate, theirs is the better number and we will say so here.

We would also want to hear about a negative result, and we would publish one. A well designed test showing that agent instruction files change nothing measurable would be genuinely useful to every team currently budgeting for this work, and it is a much easier study to run than the positive version because it does not require the effect to exist. The reason nobody has published it is not that it would be unwelcome. It is that a null result is harder to get attention for, which is a problem with incentives rather than with the question.

One more thing would change our mind about the recommendation rather than the claim. If the convention settles, and one file name wins outright across the agents your buyers use, then the split brain argument that occupies half this post stops applying and the advice collapses to write the file. That may happen within a year. It has not happened yet, and our sample is the evidence for that rather than an opinion about it, because twenty-three against twenty-three is what an unsettled convention looks like when you count it instead of asking people.

Until then the honest position is the boring one. Write the file, because it costs an afternoon and it forces you to state your own build command in one place. Collapse the duplicate, because that is a real defect with a real fix. Do not buy a citation forecast from anyone who has not held an arm out, and do not sell one either.