docs: production-readiness standard, read every conformance iteration

The bar for shipping PureXML in an IDE: correctness total and characterized,
false positives at zero, not "passes the suite." Four stoppers (reject no valid
input; accept no faulty input; characterize correctness via differential + fuzz,
no silent under-rejection debt; interactive safety: bounded resources, located
recoverable errors) plus a per-iteration checklist and definition of done.

Wired into the per-iteration protocol (schema-validity-burndown.md) and the
session Read First list (CLAUDE.md) so it is read every iteration.
This commit is contained in:
Mihaela Mihaljevic 2026-06-14 16:28:03 +02:00
parent 87cc83d0ce
commit 5aeb2aca4c
3 changed files with 63 additions and 0 deletions

View File

@ -23,6 +23,7 @@ Non-negotiables:
- `docs/rules/cross-platform.md`
- `docs/rules/testing.md`
- `docs/rules/verification.md`
- `docs/production-readiness.md` (the bar for shipping this in an IDE; read every conformance iteration)
Confirm rule loading with `rules-loaded`.

View File

@ -0,0 +1,57 @@
# Production readiness standard (read this every iteration)
PureXML is going into an IDE as the authority on whether a developer's XML and
XSD is correct. The bar is not "passes our test suite." It is correctness,
total and characterized, with the false-positive direction held at zero. Read
this before and after every schema-validity (or any conformance) iteration and
hold the change to it.
## The four stoppers (each one blocks production on its own)
1. **Rejecting valid input (false positives).** The tool must never reject a
correct schema or document. `valid-schemas-rejected` (#148) and
`valid-instances-rejected` (#146) must reach 0. In an editing loop the first
wrong red mark on work the developer knows is correct destroys trust in every
other diagnostic, and they disable the validator. This is the worst-felt
failure and the one to guard hardest: under-rejection is recoverable,
over-rejection is a non-starter.
2. **Silently accepting faulty input (false negatives).** `invalid-schemas-accepted`
(#145) and `invalid-instances-accepted` (#147) must reach 0. The tool exists
to catch the developer's mistakes; a missed error behind a green light ships
broken work, and a false negative the user cannot see is worse than a crash.
3. **Uncharacterized correctness.** Every count is measured against one finite
corpus (the XSTS subset). Real XML and XSD are unbounded. "Passes the suite"
is not "is correct." Production needs a differential harness against a trusted
reference validator over a large real-world corpus, plus fuzzing for hangs and
crashes. And no silent debt: every case the implementation excludes must be
named and bounded, never a "documented under-rejection" that emits nothing.
4. **Interactive safety.** An IDE revalidates on keystrokes against possibly
hostile input. There must be a proven worst-case time and memory bound (a cap
is a band-aid, not a bound), every diagnostic must be located (line and
column), and validation must be recoverable (report all errors, not the
first).
## Per-iteration checklist (apply on EVERY change)
- Does this change reject any valid input? If yes, that is a false positive:
stop and fix before merge. The valid-rejected buckets (#146, #148) may never
rise, and their target is 0, not "held at a baseline."
- Does it leave a silent gap? If it under-rejects, the exclusion must be NAMED
and bounded in code and changelog, and tracked as debt to close, never shipped
silently.
- Is it verified against the XSTS differential AND an adversarial over-rejection
critic before merge?
- Are any new diagnostics located (line/col) and recoverable?
- Is it durable: committed, fast-forward merged, pushed to the remote, and the
memory mirror committed and pushed? "Merged locally" is not done.
## Definition of done (the whole goal, not one iteration)
All four buckets at 0 on the FULL XSTS (not the subset); a differential + fuzz
harness so correctness is bounded, not sampled; every diagnostic located and
recoverable; a proven resource bound so interactive use cannot hang; and every
deliverable durable. Until at least false positives are 0, PureXML is a strong
engine but not an authority to put in front of a developer.

View File

@ -78,6 +78,11 @@ Each iteration targets one self-contained rule family with a clean root cause.
## Per-iteration protocol (the PR-critic-loop)
**Read `docs/production-readiness.md` first, every iteration, and hold the change
to its four stoppers and checklist.** The target for all four XSTS buckets is 0,
both directions; valid-rejected is not an acceptable baseline. Durability
(commit, ff-merge, push, mirror) is part of done.
For each cluster, on a fresh branch off `main`:
1. **Scope & measure.** Identify the cluster's cases in