NodetteHQ/PureXML
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases 1 Wiki Activity
main
PureXML/docs/production-readiness.md
Mihaela Mihaljevic 5aeb2aca4c docs: production-readiness standard, read every conformance iteration 
The bar for shipping PureXML in an IDE: correctness total and characterized,
false positives at zero, not "passes the suite." Four stoppers (reject no valid
input; accept no faulty input; characterize correctness via differential + fuzz,
no silent under-rejection debt; interactive safety: bounded resources, located
recoverable errors) plus a per-iteration checklist and definition of done.

Wired into the per-iteration protocol (schema-validity-burndown.md) and the
session Read First list (CLAUDE.md) so it is read every iteration.
2026-06-14 16:28:03 +02:00

3.2 KiB
Raw Permalink Blame History

Production readiness standard (read this every iteration)

PureXML is going into an IDE as the authority on whether a developer's XML and XSD is correct. The bar is not "passes our test suite." It is correctness, total and characterized, with the false-positive direction held at zero. Read this before and after every schema-validity (or any conformance) iteration and hold the change to it.

The four stoppers (each one blocks production on its own)

  1. Rejecting valid input (false positives). The tool must never reject a correct schema or document. valid-schemas-rejected (#148) and valid-instances-rejected (#146) must reach 0. In an editing loop the first wrong red mark on work the developer knows is correct destroys trust in every other diagnostic, and they disable the validator. This is the worst-felt failure and the one to guard hardest: under-rejection is recoverable, over-rejection is a non-starter.

  2. Silently accepting faulty input (false negatives). invalid-schemas-accepted (#145) and invalid-instances-accepted (#147) must reach 0. The tool exists to catch the developer's mistakes; a missed error behind a green light ships broken work, and a false negative the user cannot see is worse than a crash.

  3. Uncharacterized correctness. Every count is measured against one finite corpus (the XSTS subset). Real XML and XSD are unbounded. "Passes the suite" is not "is correct." Production needs a differential harness against a trusted reference validator over a large real-world corpus, plus fuzzing for hangs and crashes. And no silent debt: every case the implementation excludes must be named and bounded, never a "documented under-rejection" that emits nothing.

  4. Interactive safety. An IDE revalidates on keystrokes against possibly hostile input. There must be a proven worst-case time and memory bound (a cap is a band-aid, not a bound), every diagnostic must be located (line and column), and validation must be recoverable (report all errors, not the first).

Per-iteration checklist (apply on EVERY change)

  • Does this change reject any valid input? If yes, that is a false positive: stop and fix before merge. The valid-rejected buckets (#146, #148) may never rise, and their target is 0, not "held at a baseline."
  • Does it leave a silent gap? If it under-rejects, the exclusion must be NAMED and bounded in code and changelog, and tracked as debt to close, never shipped silently.
  • Is it verified against the XSTS differential AND an adversarial over-rejection critic before merge?
  • Are any new diagnostics located (line/col) and recoverable?
  • Is it durable: committed, fast-forward merged, pushed to the remote, and the memory mirror committed and pushed? "Merged locally" is not done.

Definition of done (the whole goal, not one iteration)

All four buckets at 0 on the FULL XSTS (not the subset); a differential + fuzz harness so correctness is bounded, not sampled; every diagnostic located and recoverable; a proven resource bound so interactive use cannot hang; and every deliverable durable. Until at least false positives are 0, PureXML is a strong engine but not an authority to put in front of a developer.