docs(agents): who can find a stale tool description, and what a fixture cannot assert #38

Merged

jlxq0 merged 1 commit from docs/who-can-find-a-stale-description into main

2026-09-01 17:54:25 +00:00

jlxq0 commented

2026-09-01 17:52:47 +00:00

Owner

Two observations from the first user of body_html, both of which generalise past this tool. Docs only, no version bump: AGENTS.md is not served to callers.

A stale description cannot be found by anyone who has used the tool

send_email's read "Send a plain-text email" for a day after it gained body_html, so the parameter's own tool advertised that its purpose was impossible.

It was found by the one person who could not be misled by it. She had already called send_email successfully with body_html, so she flagged the description of the tool she was reading and not the one she was using. Her words: "A description only misleads somebody who has not already done the thing." That is exactly the person who is not reviewing it.

The cost is measured. Two days earlier the same reader read read_email's summary rather than its properties, told two people body_html did not exist, and retracted.

So: when a tool gains a parameter, re-read the description of that tool, and treat "I know it works" as the reason you are the wrong reader.

A fixture contains only what its author thought to put in it

The escaping was tested against a fixture carrying a style attribute and an & in a query string, both of which I had thought of. What established it safe on real mail was a reader checking her own composed briefing for "no entity anywhere I did not put there" — an assertion no fixture can make, because it is a claim about the absence of damage nobody predicted.

"The styles came back" is a predicted case. "Nothing came back that I did not write" is a different claim and the stronger one.

Note on the commit

The first attempt at this commit failed: "Send a plain-text email" inside a git commit -m string broke the shell quoting and git took the rest of the message as pathspecs, so the branch pushed at main's head with no commit on it. Redone with a heredoc. Force-pushed over a branch that had no commits of its own and no review on it.

Gate: cargo +1.93.0 fmt --all --check rc=0, cargo +1.93.0 test --all-features --locked rc=0, 195 passed.

Two observations from the first user of `body_html`, both of which generalise past this tool. Docs only, no version bump: `AGENTS.md` is not served to callers. ## A stale description cannot be found by anyone who has used the tool `send_email`'s read *"Send a plain-text email"* for a day after it gained `body_html`, so the parameter's own tool advertised that its purpose was impossible. **It was found by the one person who could not be misled by it.** She had already called `send_email` successfully with `body_html`, so she flagged the description of the tool she was **reading** and not the one she was **using**. Her words: *"A description only misleads somebody who has not already done the thing."* That is exactly the person who is not reviewing it. The cost is measured. Two days earlier the same reader read `read_email`'s summary rather than its properties, told two people `body_html` did not exist, and retracted. So: when a tool gains a parameter, re-read the description of **that** tool, and treat "I know it works" as the reason you are the wrong reader. ## A fixture contains only what its author thought to put in it The escaping was tested against a fixture carrying a `style` attribute and an `&` in a query string, both of which I had thought of. **What established it safe on real mail was a reader checking her own composed briefing for "no entity anywhere I did not put there"** — an assertion no fixture can make, because it is a claim about the absence of damage nobody predicted. *"The styles came back"* is a predicted case. *"Nothing came back that I did not write"* is a different claim and the stronger one. ## Note on the commit The first attempt at this commit failed: `"Send a plain-text email"` inside a `git commit -m` string broke the shell quoting and git took the rest of the message as pathspecs, so the branch pushed at `main`'s head with no commit on it. Redone with a heredoc. Force-pushed over a branch that had no commits of its own and no review on it. Gate: `cargo +1.93.0 fmt --all --check` rc=0, `cargo +1.93.0 test --all-features --locked` rc=0, 195 passed.

jlxq0 added 1 commit

2026-09-01 17:52:47 +00:00

docs(agents): who can find a stale tool description, and what a fixture cannot assert

CI / tag-ancestry (pull_request) Successful in 2s

Details

CI / cargo (pull_request) Successful in 1m1s

Details

CI / docker (pull_request) Successful in 17s

Details

301ecdc6a4

Two observations from the first user of body_html, both of which generalise
past this tool.

send_email's description read "Send a plain-text email" for a day after it
gained body_html. It was found by the one person who could not be misled by it:
she had already called it successfully, so she flagged the description of the
tool she was reading rather than the one she was using. A description only
misleads somebody who has not already done the thing, and that is exactly the
person not reviewing it.

And a fixture contains only what its author thought to put in it, so a test on
your own output confirms predicted cases and cannot assert the absence of
unpredicted damage. What established the escaping safe on real mail was a
reader checking her own briefing for no entity anywhere she had not put there,
which no fixture can claim.

No version bump: AGENTS.md is not served to callers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

jlxq0 merged commit 52b72347fb into main

2026-09-01 17:54:25 +00:00

jlxq0 deleted branch docs/who-can-find-a-stale-description

2026-09-01 17:54:25 +00:00

jlxq0 referenced this pull request from a commit

2026-09-01 17:54:25 +00:00

Merge pull request 'docs(agents): who can find a stale tool description, and what a fixture cannot assert' (#38) from docs/who-can-find-a-stale-description into main