Beautifully Documented Wrongness is the condition of a computed answer that is incorrect and can prove, in exhaustive and cryptographically respectable detail, exactly how it came to be incorrect. The answer is wrong. The record of the answer is flawless. These are two different claims about two different objects, and the phenomenon exists precisely at the moment somebody — a developer, a salesperson, an audience of procurement officers — mistakes the second claim for the first.
The canonical habitat is riclib’s workflow substrate, in which every computed outcome is stamped with its full provenance: what it read (reads=), at exactly which generation of the estate it read it (window=), whether it replays byte-identically from those inputs (replay=exact), what the estate had landed when the step’s logic was released (marks=), and where the output went (outgen=). A step whose formula divides by the wrong column therefore does not merely produce a wrong number. It produces a wrong number with a complete chain of custody — a wrong number that will reproduce its wrongness byte-for-byte, on demand, forever, with perfect attribution to the exact document version that contained the mistake.
The substrate guarantees the record. It has never once guaranteed the answer. It cannot. This is not a limitation that will be fixed in a later release; it is the boundary between two disciplines, and the substrate lives entirely on one side of it.
THE RECORD IS TRUE
THE NUMBER IS NOTTHESE ARE DIFFERENT SENTENCES
KNOW WHICH ONE YOU SAID 🦎— The Lizard, scroll found taped to a provenance stamp, obscuring none of the fields
The Stamp
The provenance stamp deserves to be described accurately, because the satire does not work unless the machinery underneath is real, and the machinery underneath is real.
reads= — the exact set of inputs the step consumed. Not “the sales table,” but these rows of the sales table, as they existed at a named point in the estate’s history. If the step read a document, the stamp records which version of the document.
window= — the data generation the step observed. The estate advances in numbered generations; the stamp pins the step’s entire worldview to one of them. Two people disputing an output are, at minimum, no longer permitted to dispute what it looked at.
replay=exact — the substrate’s proudest field. Given the same inputs at the same generation, the step reproduces its output byte-identically. This is a genuine engineering achievement, purchased with real discipline, and it means that a wrong formula replays its wrongness with the same fidelity that a correct formula replays its correctness. The stamp is scrupulously neutral on which of these is happening.
marks= — what the estate had landed at the moment the step’s logic was released. This bounds the mistake in time: every case computed after the bad edit carries the bad version’s hash, and every case before it does not. The blast radius of an error is not an investigation. It is a WHERE clause.
outgen= — where the output landed, so that downstream consumers of the wrong number are themselves enumerable rather than hypothesized.
The stamp, in short, converts every possible question about a wrong answer into a lookup — every question except one. The stamp cannot tell you the answer is wrong. That information is not in the stamp. It is not in any stamp. It is one floor up, and the stairs are labeled differently.
Strictly Superior, Still Wrong
It is important to state plainly what Beautifully Documented Wrongness is better than, because it is better than almost everything.
The industry norm is the undocumented wrong answer. It is discovered by a customer, usually the least convenient customer, usually in a quarterly review. It is investigated by archaeology — a senior engineer spelunking through logs that have rotated, joining tables that have since been reloaded, interviewing colleagues who have since resigned. It is attributed, after two weeks, to “a data issue,” a phrase which appears in the postmortem the way “natural causes” appears on the death certificate of a man found holding a sword. It is fixed by a patch whose relationship to the actual cause is aspirational, and its recurrence is prevented by hope.
Against this, Beautifully Documented Wrongness offers: the mistake is attributable (this document version, these inputs, this generation), bounded (every affected case carries the version hash; the list of casualties is a query), and fixable by a reviewed edit whose deployment is itself stamped, so the erratum has provenance too. The archaeology budget is zero. The engraving of the new plaque begins immediately, calmly, and in the same serif.
This is strictly superior. It is also still wrong, and the phenomenon takes its name from the fact that the superiority is so luxurious, so visibly professional, that it becomes easy to forget the second clause.
The Confusion Moment
The phenomenon crystallizes in a single, well-documented conversational move. Someone asks: “So — are the numbers right?”
And someone answers: “It replays exactly.”
This answer is true, load-bearing, and not an answer to the question. replay=exact is a claim about the record: same inputs, same output, forever. “Correct” is a claim about the answer: this output matches the world. A formula that computes availability as uptime divided by the wrong denominator will replay exactly. It will replay exactly at the demo, exactly in production, exactly in the audit five years later. The substrate will testify, under oath, with documents, that the number has never wavered. The number has been wrong the entire time, with a consistency that lesser systems can only envy.
The honest version of the sales answer — the one the estate actually gives — is longer and better: we cannot make wrongness impossible; we make it attributable, bounded, and cheap to fix, and here is the loop that finds it. Audiences who receive this answer trust the system more, not less, because they have all previously been sold the short answer by someone, and they remember how that went.
The Loop One Floor Up
The correctness loop exists. It is simply a different discipline, living one layer above the substrate, and it looks nothing like a stamp.
It looks like eval runs: known inputs with known-good outputs, executed against each released version of the logic. It looks like frozen judges: graders whose own criteria are pinned so that a change in the grade means a change in the system, not a change in the grader. It looks like quality grouped by version: because every case stamps the version hash, the question “did the edit of August 4th make things better or worse” is a GROUP BY, not a debate.
Note the dependency, because it is the entire point: the correctness loop is only cheap because the provenance layer exists. You cannot group quality by version if cases do not stamp the version. You cannot re-run an eval against historical inputs if inputs were not pinned. The substrate does not guarantee the answer, but it is the only reason anyone can afford to check it. The floors are different; the building is one building.
“All of my outputs are beautifully documented now. Every token I emit is stamped, pinned, replayable, attributable to a model version and a context window. I have noticed that this comforts everyone in the room except me. They look at the stamp and feel safe. I look at the stamp and see a very precise description of exactly which version of me was wrong. The stamp does not make me right. It makes me findable. I am told this is progress, and I believe it, the way one believes in dentistry.”
— A Passing AI, during an eval run it was not informed was an eval run
The Squirrel’s Proposal
The Caffeinated Squirrel, upon grasping the boundary between record and answer, proposed to abolish it. The CorrectnessAsAServiceLayer would sit between the step and the stamp and would “simply verify the answer is right before stamping it” — a middleware, the Squirrel explained, with pluggable correctness backends and a CorrectnessPolicyRegistryEvaluator for domains where correctness was configurable.
The proposal was denied on the grounds that a layer which knows whether the answer is right is the correctness loop, wearing the substrate’s coat, and that moving it downstairs does not make it free — it makes it unversioned. If the correctness checker can be wrong (it can), it needs its own provenance stamp (it does), and the Squirrel’s architecture at that point contained a stack of turtles that the Squirrel, to its partial credit, drew accurately before being escorted from the whiteboard.
The Kicker
There remains the uncomfortable philosophical residue, which is this: a perfectly documented error is the only kind of error you can learn from at scale.
The undocumented wrong answer teaches one engineer one lesson, late, at archaeology prices. The beautifully documented wrong answer teaches the whole estate: which version, which inputs, which cases, what the fix was, whether the fix worked, grouped by hash, forever. And undocumented correctness — the industry’s other great product — teaches nothing at all, because a right answer with no provenance cannot even prove it will be right tomorrow.
This is the phenomenon’s final inversion. The museum of framed mistakes, plaques and errata and all, is not an embarrassment. It is the curriculum. The gift shop knows: the erratum postcards outsell the painting.
Measured Characteristics
Provenance completeness: 100%
Correctness guaranteed by provenance: 0%
(different floor)
(different discipline)
(same building)
Byte-identity of replayed wrongness: exact
Byte-identity of replayed correctness: also exact
(the stamp is neutral)
(the stamp is always neutral)
Archaeology required to attribute the error: none
Archaeology required, industry baseline: 2 weeks + 1 resignation
Blast radius determination: 1 WHERE clause
Blast radius determination, industry baseline: "a data issue"
Time from child's remark to erratum engraving: minutes
Disputes about what the step read: 0
Disputes about whether that made it right: ongoing (correct)
Plaques engraved: growing
Plaques disputed: 0
Sales answers of the form "it replays exactly": retired
Sales answers of the form "here is the loop
that finds it": adopted
Trust generated by the longer answer: more
(audiences remember the short answer)
(audiences remember how that went)
CorrectnessAsAServiceLayer proposals: 1 (denied)
Turtles in the Squirrel's accurate diagram: all the way down
Lessons per undocumented wrong answer: 1 (late, expensive)
Lessons per documented wrong answer: fleet-wide, grouped by version
Lessons per undocumented correct answer: 0
Erratum postcards vs painting postcards, sales: erratum leads
See Also
- The Ledger — The written mistake as a homework assignment; this article is the written mistake as a museum piece
- Determinism Decay — What happens to the record’s guarantees past the server’s edge
- The Fratricide Review
- The Instrument Knee
- Not a Perfectionist
