diff --git a/CLAUDE.md b/CLAUDE.md index 7157f18..17ea177 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,6 +23,7 @@ Non-negotiables: - `docs/rules/cross-platform.md` - `docs/rules/testing.md` - `docs/rules/verification.md` +- `docs/production-readiness.md` (the bar for shipping this in an IDE; read every conformance iteration) Confirm rule loading with `rules-loaded`. diff --git a/docs/production-readiness.md b/docs/production-readiness.md new file mode 100644 index 0000000..d4f3cc1 --- /dev/null +++ b/docs/production-readiness.md @@ -0,0 +1,57 @@ +# Production readiness standard (read this every iteration) + +PureXML is going into an IDE as the authority on whether a developer's XML and +XSD is correct. The bar is not "passes our test suite." It is correctness, +total and characterized, with the false-positive direction held at zero. Read +this before and after every schema-validity (or any conformance) iteration and +hold the change to it. + +## The four stoppers (each one blocks production on its own) + +1. **Rejecting valid input (false positives).** The tool must never reject a + correct schema or document. `valid-schemas-rejected` (#148) and + `valid-instances-rejected` (#146) must reach 0. In an editing loop the first + wrong red mark on work the developer knows is correct destroys trust in every + other diagnostic, and they disable the validator. This is the worst-felt + failure and the one to guard hardest: under-rejection is recoverable, + over-rejection is a non-starter. + +2. **Silently accepting faulty input (false negatives).** `invalid-schemas-accepted` + (#145) and `invalid-instances-accepted` (#147) must reach 0. The tool exists + to catch the developer's mistakes; a missed error behind a green light ships + broken work, and a false negative the user cannot see is worse than a crash. + +3. **Uncharacterized correctness.** Every count is measured against one finite + corpus (the XSTS subset). Real XML and XSD are unbounded. "Passes the suite" + is not "is correct." Production needs a differential harness against a trusted + reference validator over a large real-world corpus, plus fuzzing for hangs and + crashes. And no silent debt: every case the implementation excludes must be + named and bounded, never a "documented under-rejection" that emits nothing. + +4. **Interactive safety.** An IDE revalidates on keystrokes against possibly + hostile input. There must be a proven worst-case time and memory bound (a cap + is a band-aid, not a bound), every diagnostic must be located (line and + column), and validation must be recoverable (report all errors, not the + first). + +## Per-iteration checklist (apply on EVERY change) + +- Does this change reject any valid input? If yes, that is a false positive: + stop and fix before merge. The valid-rejected buckets (#146, #148) may never + rise, and their target is 0, not "held at a baseline." +- Does it leave a silent gap? If it under-rejects, the exclusion must be NAMED + and bounded in code and changelog, and tracked as debt to close, never shipped + silently. +- Is it verified against the XSTS differential AND an adversarial over-rejection + critic before merge? +- Are any new diagnostics located (line/col) and recoverable? +- Is it durable: committed, fast-forward merged, pushed to the remote, and the + memory mirror committed and pushed? "Merged locally" is not done. + +## Definition of done (the whole goal, not one iteration) + +All four buckets at 0 on the FULL XSTS (not the subset); a differential + fuzz +harness so correctness is bounded, not sampled; every diagnostic located and +recoverable; a proven resource bound so interactive use cannot hang; and every +deliverable durable. Until at least false positives are 0, PureXML is a strong +engine but not an authority to put in front of a developer. diff --git a/docs/schema-validity-burndown.md b/docs/schema-validity-burndown.md index 4423bd7..e425bd8 100644 --- a/docs/schema-validity-burndown.md +++ b/docs/schema-validity-burndown.md @@ -78,6 +78,11 @@ Each iteration targets one self-contained rule family with a clean root cause. ## Per-iteration protocol (the PR-critic-loop) +**Read `docs/production-readiness.md` first, every iteration, and hold the change +to its four stoppers and checklist.** The target for all four XSTS buckets is 0, +both directions; valid-rejected is not an acceptable baseline. Durability +(commit, ff-merge, push, mirror) is part of done. + For each cluster, on a fresh branch off `main`: 1. **Scope & measure.** Identify the cluster's cases in