PureXML/docs/xsts-deviations.md

# XSTS deviation worklist

This partitions the W3C XML Schema Test Suite (XSTS, 2006-11-06 archive)
deviations by **root cause and kind**, so the burn-down is driven by
understanding each rule rather than by chasing the count. First measured
2026-06-13; baselined and ratcheted in `Tests/XSTSSuiteTests.swift`.

## What the oracle measures

The runner compares PureXML's verdict to the suite's expected validity. A
schema is "accepted" iff `Schema.Document(source)` does not throw; an instance
is "accepted" iff `validate` returns no errors. The four counts:

| Kind | Meaning | Direction |
|---|---|---|
| valid schema rejected | legal XSD that PureXML refused to compile | too strict |
| invalid schema accepted | illegal XSD that PureXML compiled anyway | too lenient |
| valid instance rejected | valid document PureXML flagged | too strict |
| invalid instance accepted | invalid document PureXML passed | too lenient |

The oracle is binary and coarse: "invalid schema accepted" conflates *we have a
bug* with *we do not implement this validity constraint by design*. The point of
this document is to separate those.

Only tests with a settled expectation are counted. The suite marks contested
entries `<current status="queried">` (disputed in W3C Bugzilla, the expected
validity itself in question, e.g. `\p{Lu}*` over two uppercase letters one of
which is an astral math capital); agreeing or disagreeing with a contested
expectation is not meaningful, so the runner skips them. Excluding the queried
entries removed about 70 apparent deviations that were never bugs.

## Baseline (settled expectations; synced 2026-06-16)

| Kind | First measurement | Now |
|---|---|---|
| valid schema rejected | 75 | 1 |
| invalid schema accepted | 2467 | 264 |
| valid instance rejected | 604 | 173 |
| invalid instance accepted | 582 | 155 |

Counts are against *settled* W3C expectations (queried/disputed entries excluded).

Current ratchet (2026-06-19): `valid schema rejected = 0`,
`invalid schema accepted = 43`, `valid instance rejected = 0`,
`invalid instance accepted = 31`. The historical table above is retained as the
published baseline snapshot; the active burn-down is in
`docs/plans/2026-06-18-false-negative-tail.md`.

XSTS is a deliberately adversarial, exhaustive corpus; mature validators
(Xerces, libxml2) do not pass it fully. The goal is not a green number. It is to
fix the deviations that are genuine correctness bugs affecting real documents,
and to document the remainder as spec-justified, deliberate exclusions.

## libxml2 differential triage (2026-06-26)

With the corpus runnable, the open tail (10 invalid-schemas-accepted, 14
invalid-instances-accepted) was triaged against libxml2 (`xmllint --schema`), the
package's stated reference. A case libxml2 also accepts is one where matching the
suite means being stricter than libxml2 (Xerces/.NET territory): a deliberate
scope choice with false-positive risk. A case libxml2 rejects is a genuine gap
whose fix aligns PureXML with its reference rather than exceeding it.

Schema side (10 cases):

| Case | libxml2 | Reading |
|---|---|---|
| `particlesZ030_d` | rejects | Reference-aligned bug, now fixed: a `simpleContent` restriction relaxing a base `required` attribute to `optional` (derivation-ok-restriction.2). invalid-schemas-accepted 10 -> 9. |
| `elemZ026`, `particlesV020` | accept too | Closed 2026-07-05: NameAndTypeOK clause 1 now pairs the DECLARED element references (`Particle.declaredReference` preserves the pre-substitution-expansion identity), so a restriction that swaps a head reference for a substitution-group member is rejected. Stricter than libxml2, spec-grounded (section 3.9.6 Elt:Elt clause 1 has no substitution allowance). The identical construct `particlesZ008` carries a settled `valid` expectation contradicting elemZ026's settled `invalid`; Z008 is reclassified as spec-divergent (excluded), with the full adjudication in `Tests/XSTSSuiteTests.swift`. |
| `particlesIk011` | accept too | Closed at the 9 -> 8 step: NameAndTypeOK over a union base applies cos-st-derived-ok, so a separately-declared union whose members merely subset the base union's members is not a valid derivation. |
| `particlesIk025`, `particlesIk027` | accept too | Disclosed divergence, kept in the count: the restricting type derives from a union MEMBER's base, and PureXML accepts per cos-st-derived-ok clause 2.2.4; the XSTS `invalid` expectation reads the clause more literally (derivation must pass through the anonymous member itself). See the ratchet history in `Tests/XSTSSuiteTests.swift`. |
| `particlesHa161`, `particlesM034` | accept too | Closed 2026-07-05: RecurseAsIfGroup against a NON-repeating base choice ({max occurs} = 1) now applies the literal section 3.9.6 pairing; the effective-total-range approximation is kept only for a repeating choice, where the particlesZ001 ambiguity actually lives (counts accumulated across repetitions). Ha161: a derived `a{0,1}` is not a pointwise NameAndTypeOK restriction of the branch `a{1,1}`. M034: a derived `d1+` cannot map into `(d1*, d2+)` because the required `d2` is unmapped and not emptiable (its settled-valid twin M035, `d2+`, still passes). Stricter than libxml2. |
| `particlesIg004` | accept too | Reclassified spec-divergent (excluded, see `Tests/XSTSSuiteExclusions.swift`): section 3.3.2 maps `block="#all"` and the explicit `substitution extension restriction` to the SAME {disallowed substitutions} set, so NameAndTypeOK clause 5's superset holds and the `invalid` expectation has no spec path; jointly unsatisfiable with the settled Ig001/002/003 (equal sets, valid) and Ig005 (`#all` covers the triple, valid). |
| `schN12` | (multi-document) | Policy-dependent deviation, kept in the count (the schA7 precedent). A chameleon redefine of a no-namespace group whose redefining document is `elementFormDefault="qualified"`: the redefined group's locals land in ns-b while the originals are no-namespace, so src-redefine clause 6.2.2's restriction obligation fails NameAndTypeOK clause 1. PureXML does not enforce 6.2.2, and cannot uniformly: mgO013's settled-VALID schema also violates 6.2.2 (an `all{e1(1,1)}` redefining `all{e1(0,0)}`), so any uniform enforcement rejects a settled-valid schema (a valid-schema-rejected false positive, the hardest floor). The two settled expectations admit no single conformant policy; PureXML keeps the FP-safe one. |

Instance side (sampled): `idZ010`, `mgO013`, `particlesZ001`, `reT17`, `reT38`
are rejected by libxml2 (reference-aligned). `reT17`/`reT38` are now fixed: they
are the `\D` half of the Ethiopic-digit `\d` bug. XSD pins the regex repertoire at
Unicode 3.2, where `U+1369`..`U+1371` are `Nd` (reclassified to `No` later), but
`XSDCategory`'s per-code-point override table omitted them, so `\d` used the modern
`No`. Pinning the nine to `Nd` makes `reT17`/`reT38` reject (`invalid-instances-accepted`
14 -> 12) and the matching `\d`-valid cases `reS17`/`reS38`/`reZ004` pass without
their former suspect-test exclusion; this is the complete `Nd` 3.2-vs-now delta, and
`\w` is unaffected. `reZ006i` (the `\c` case) is also now fixed: `\i`/`\c` were wired
to the parser's XML 1.0 Fifth Edition name check, but XSD defines them against the
Second Edition (Appendix B) classes; the new `XSDNameCharacter` uses the Appendix B
ranges, so the Fifth-Edition-only combining mark `U+0346` no longer matches `\c`
(`invalid-instances-accepted` 12 -> 11). The remaining intricate cases: `particlesZ001`'s
rule is marked ambiguous by the suite itself (RecurseAsIfGroup); `mgO013` is an
`xs:all` under `redefine`; `idZ010` turns on keyref namespace resolution with
non-absolute namespace URIs. The multi-document `sch*` / `targetns` cases need
per-schema pairing before a verdict.

`anyURI_a004_1339.i` is reclassified as a contested test, not a deviation
(`invalid-instances-accepted` 11 -> 10, now excluded like the suite's `queried`
entries). The suite pairs a `valid` schemaTest with an `invalid` instanceTest, but
the two are mutually inconsistent: the schema's `include`/`import`/`redefine` have
unresolvable schemaLocations, which XSD 1.0 §4.2.1 makes "not an error" (no
inclusion performed; the empty `redefine` imposes no resolution requirement per
`src-redefine` clause 1), so the schema is conformantly valid, and the only
components the instance exercises (`root`/`bar`/`st`/`att`) are fully locally
defined with every value byte-identical to one of the type's six anyURI
`enumeration` members, so the instance is valid. There is no spec path to
valid-schema + invalid-instance. Two independent adversarial analyses failed to
refute this; libxml2 is no authority (it non-conformantly fails to compile the
schema on the unresolvable include). Matching the suite would reject a valid
instance whenever an include cannot be resolved, a false positive an IDE hits
constantly. Recorded in `XSTSSuiteTests.specDivergentInstances`.

The `sch*` schema-location cluster, dissected per case (an adversarial analysis
separated the contested from the genuine):

- `schA2.i`, `schA5.i`: **contested**, reclassified (`invalid-instances-accepted`
  10 -> 8). Each group's own documentation says the inline `xsi:schemaLocation`
  hint "is ignored" and the instance "must conform to" the collection schema; under
  exactly that behavior libxml2 validates the instance (schA2 against `a.xsd`, schA5
  against `a.xsd`+`b.xsd`), so the `invalid` expectation contradicts the group's own
  documentation. Reaching `invalid` needs the instance's hint to override a schema
  already loaded for that namespace, which XSD 1.0 §4.3.2 makes optional (strategy 1)
  and which would be a direct IDE false positive. Excluded in `specDivergentInstances`.
- `schA7.i`: **genuine, policy-dependent deviation, left in the count.** Here the
  `invalid` IS spec-grounded: with hints ignored, the instance's `b:e1` (namespace
  `ns-b`) has no declaration in the `a.xsd`-only collection and is demanded by
  `root`'s default-`strict` `##any` wildcard, so §3.10.4 makes it invalid. PureXML
  accepts it only by honoring the `ns-b` hint to gap-fill the wildcard, the other
  conformant §4.3.2 strategy. A real divergence in schema-location policy, not a
  suite bug; not excluded.
- `schU3.i`, `schU4.i`, `schU5.i`: **genuine under-rejection in an underspecified
  area, deferred.** These are circular `redefine` chains where one attribute group
  (`c-g1`) is redefined to conflicting types (`int` via one document, `string` via
  another) across an include/redefine cycle. The suite expects the `int` reading, so
  the instance's `c1="a"` is invalid; PureXML resolves to the laxer type and accepts.
  libxml2 does not support this at all (it rejects the schema: "Multiple redefinition
  of the same component is not supported"), so matching the suite means out-resolving
  the reference on a construct whose composition order in a cycle the spec does not
  pin down. Tracked as deferred scope, not excluded.

`mgO013.i` is reclassified as contested (`invalid-instances-accepted` 7 -> 6).
A `redefine` defines `group = all{e1(1,1)}`, redefining a base `all{e1(0,0)}`
without referencing itself. `src-redefine` clause 6.2.2 makes a no-self-reference
group redefine a restriction obligation, and `all{e1(1,1)}` is not a valid
restriction of `all{e1(0,0)}` (Occurrence Range OK fails: derived max 1 > base
max 0), so enforcing it rejects the schema, a valid-schema-rejected false positive
against the `valid` schemaTest. Yet §4.2.2 makes the referencing type resolve to
the redefined `all{e1(1,1)}`, against which the single-`e1` instance is valid. The
`valid` schema and `invalid` instance expectations are mutually exclusive; libxml2
accepts both. An adversarial analysis found no third path. The identity-field
case `idZ010` (a `key`/`keyref` field selecting a non-simple `anyType` node) was a
genuine bug and is fixed (reject per cvc-identity-constraint.3).

`particlesZ001.i` is reclassified as contested (`invalid-instances-accepted` 5 -> 4).
`Derived` complexContent-restricts `Base`, and the instance is valid against
`Derived`'s own content model. cvc-complex-type (§3.4.4) validates an instance
against the element's own `{content type}` and never re-runs Particle Valid
(Restriction) / RecurseAsIfGroup (§3.9.6, a schema-assembly constraint), so
schema-valid + instance-invalid is unreachable. The manifest documentation itself
labels the governing rule ambiguous; libxml2 accepts both. The identity-field
`idZ010` (a `key`/`keyref` field selecting a non-simple `anyType` node) and the
`xsi:type` namespace case `targetns00101m` (an unprefixed `xsi:type` resolves in
the instance namespace context, not by bare local name) were genuine bugs and are
fixed.

Takeaway: on the schema side, libxml2 parity is reached except the one case now
fixed; the remaining nine are a deliberate choice to exceed the reference. On the
instance side the Ethiopic-digit `\d`/`\D`, `\i`/`\c`, identity-field `idZ010`, and
`xsi:type` `targetns00101m` cases are now fixed (reference-aligned); `anyURI_a004`,
`schA2`, `schA5`, `mgO013`, and `particlesZ001` are shown contested. The remaining
instance deviations are `schA7` (a genuine schema-location policy choice) and
`schU3/4/5` (circular `redefine` composition, which libxml2 does not implement);
both are deliberate, not clean bugs.

## Category A: genuine correctness bugs

### Fixed (thirty-four root causes, ~2258 deviations cleared)

list-datatype length facets (#146) · union pattern/enumeration facets · duration
partial-order facets · QName length non-constraining · the full XSD `\p{Is...}`
Unicode block set · element-level `block` against `xsi:type` · empty-element
default/fixed values + enumeration-restriction-replaces · `block="substitution"`
on substitution-group heads · substitution-group closure stops chaining through
members whose own `block` contains `substitution` · `\w`/`\W` XSD definition · document-scoped
xs:ID/xs:IDREF uniqueness and resolution · `<xs:attribute ref>` resolution ·
mixed-content type with no element model · built-in `xsi:type` resolution gated
by substitution validity (the XSD Part 2 derivation lattice), which also closed
the two gaps it exposed (a simple-typed element rejecting stray attributes;
identity fields comparing in value space when the node carries an `xsi:type`) ·
compile-time constraining-facet definition validity (length-family facets are
`nonNegativeInteger`, co-occurrence and range order) · value-bound facet
validity (`min`/`maxInclusive`, `min`/`maxExclusive`, `enumeration` values valid
in the base space; inclusive/exclusive exclusions; bound ordering) ·
substitution-group heads typed `xs:anySimpleType` only admitting simple and
simpleContent members · global element refs carrying declaration metadata into
Particle-Valid-Restriction · `gMonth` `--MM--` lexical form · schema `id` attribute validity (`xs:ID`: NCName and
unique within the document). The schema-validity work brought invalid schemas
accepted 2461 to 43 (schema-document structural validity: content model + complexType content shape, attribute values, name/reference + pattern-regex lexical validity, and reference resolution, against the schema-for-schemas, plus simpleType variety constraints on list itemType and union memberTypes, plus targetNamespace scope matching on local schema references, plus substitution group exclusions based on element final and schema finalDefault attributes, duplicate schema container validation across separate files with identical content, schema element order, import constraints validation, and the later tail fixes); see `schema-validity-burndown.md`.
Plus the measurement fix (exclude W3C-disputed `status="queried"` entries).

### Remaining, root-caused (the hard tail)

Each is a substantial, self-contained study, not a one-locus cluster fix:

| Cluster(s) | ~Count | Root cause (diagnosed) | Difficulty |
|---|---|---|---|
| `particlesOb/Je/Jf`, related | ~30 | `<xs:any>` defaults to `processContents="strict"`; the matched element's declaration lives in a *separate* schema document referenced only by the instance `xsi:schemaLocation` (no `<import>`). Needs multi-document schema loading + `xsi:schemaLocation` honoring, plus strict-wildcard semantics and Particle-Valid-Restriction. | large (new capability) |
| `reI` | ~10 | `pattern` values containing literal tab/newline: XML attribute-value normalization (tab/newline to space) versus the value's `whiteSpace` facet. The "right" pattern text depends on that interaction. | subtle |
| `test`, `elemT` | ~65 | heterogeneous grab-bags (MS "Additional"/element tests): many distinct minor causes, not one root cause. Each case needs individual triage. | per-case grind |
| `idF`, `idc` remainder | ~14 | identity-constraint selector/field XPath edge cases beyond the implemented core. The value-space comparison via instance `xsi:type` and built-in `xsi:type` resolution now land; what remains is the declared-type-driven value-space comparison (needs PSVI type assignment, since the field's declared simple type is not threaded into identity validation) and selector/field XPath edges. | moderate |
| `wildG`, `stZ`, `addB` | ~25 | wildcard-in-group composition, simple-type edge cases, attribute-default edges. | mixed |

The clean, single-root-cause, high-leverage fixes are exhausted. Reaching zero
on the instance buckets requires the multi-document-schema capability (the
largest single lever, ~30 cases) and per-case work on the grab-bags; neither is
a quick fix, and rushing them would violate the project's no-shortcuts ethic.

## Category B: schema-for-schemas validity (unimplemented by design)

The bulk of the 2467 "invalid schema accepted". PureXML compiles schemas
permissively: it reads the components it understands and ignores structurally
illegal content rather than validating the schema document against the
schema-for-schemas. Examples found:

- `ctB`: `<complexType>Annotation information</complexType>` (bare text content) is accepted.
- `notatF`: `<notation>` placed inside `<restriction>` (illegal position) is accepted.
- `idA/idB/idC`: malformed identity-constraint definitions accepted.
- `annotation/annotB`: illegal annotation placement accepted.
- `addB`, `stF`, `ctG`, `ctH`: assorted illegal component structures accepted.

| Cluster(s) | Count |
|---|---|
| `ctB/ctG/ctH` (complexType structure) | ~183 |
| `idA/idB/idC` (identity constraints) | ~118 |
| `notatF` (notation placement) | ~66 |
| `annotation/annotB` | ~62 |
| `addB`, `stF`, and the long tail | remainder of 2467 |

**Status (revised):** now being implemented as the general schema-for-schemas
mechanism, not a per-rule pile. The content-model pass has landed (`id` validity,
facet-definition validity, and the `allowedChildren` structural table), driving
invalid-schemas-accepted from 2461 to 1313. The remaining structural work (the
component **attribute model**, child order, particle semantics) continues under
`schema-validity-burndown.md`. A too-lenient compiler does not corrupt
valid-document validation, but rubber-stamping invalid schemas is the largest
conformance gap, so it is being closed rather than excluded.

## Category C: arguable / needs investigation

- `test`, `elemT` (invalid instance accepted, ~86): not yet sampled; classify before fixing.

## Category D: deliberate interpretations of spec-ambiguous points

These are resolved, not open: the spec underspecifies the point and the suite's
own contributors disagree, so a single behavior cannot satisfy all of them. We
pick the defensible reading and accept the residual deviation.

- **QName length facets are non-constraining.** XSD 1.0 Datatypes 4.3.1 defines
  the unit of length for string (characters), anyURI (characters), binary
  (octets), and list (items), but not for QName, so it is implementation-defined.
  NIST (36+ cases) and Xerces treat `length`/`minLength`/`maxLength` on QName as
  always satisfied; Microsoft (4 cases: `QName_length001/003`, `QName_minLength003`,
  `QName_maxLength001`) treats them as character counts and expects rejection.
  These are directly contradictory (NIST marks a 15-character value valid under
  `length=1`; MS marks `foofo` invalid under `length=4`). We chose
  non-constraining, matching Xerces and the larger NIST set, because a QName's
  value is an abstract (namespace, local-name) pair with no defined character
  length. Cost: the 4 MS cases remain in "invalid instances accepted". Fixing
  the 36+ NIST false rejections (valid documents wrongly rejected) is the more
  important direction.

- **RecurseAsIfGroup for a repeating element against a repeating choice
  (`particlesZ001`).** A derived `element{0,unbounded}` restricting a base
  `choice{0,unbounded}(element, any)`. The Microsoft test marks it valid, and its
  own documentation states the reason: "Particle Derivation OK (Elt:Choice --
  RecurseAsIfGroup) rule is ambiguous." Under the literal §3.9.6 rule (which
  Xerces follows) the element is wrapped as a one-member `{1,1}` synthetic group
  whose member keeps the element's `{0,unbounded}` occurrence, and that member
  must be a valid restriction of a `{1,1}` choice branch; `{0,unbounded}` does not
  fit `{1,1}`, so the strict reading rejects it. We follow the strict reading
  (matching the reference implementation), accepting this one residual "valid
  schema rejected". Adopting the lenient reading would require a special case that
  cannot be expressed without over-accepting genuinely invalid restrictions on
  PureXML's content-model representation (an attempt to do so regressed roughly
  thirty other cases). Tracked on #165. The sibling `particlesV003` (Sequence:Choice
  MapAndSum) was a genuine bug, since fixed.

## How a fix lands

Pick a Category A cluster, read 2-3 cases (schema + instance + expected) against
the spec text, fix the owning code, add a focused regression test, re-run
`swift test -c release --filter XSTS`, confirm the relevant count drops, and
ratchet the baseline in `Tests/XSTSSuiteTests.swift` down to the new value.
Tracking issues: #145 (invalid schema accepted), #146 (valid instance
rejected), #147 (invalid instance accepted), #148 (valid schema rejected).

## Historical autopilot accounting (2026-06-18): labelled-suite invalid-schemas-accepted = 74

At this earlier snapshot, the `XSTSSuiteTests` ratchet
(settled-expectation labelled subset) stood at
**0 valid-schemas-rejected, 74 invalid-schemas-accepted, 0 valid-instances-rejected,
133 invalid-instances-accepted**. False positives have been held at 0 across the
whole campaign (the hard gate). The remaining 74 were the hard tail at that
snapshot. They partitioned as:

### Fixable debt (genuine bugs, but intricate and false-positive-prone)

| Cluster | ~Count | Root cause | Why deferred |
|---|---|---|---|
| `schL`, `schM`, `attgZ` | ~12 | redefine RESTRICTION: a redefined group/attributeGroup must be a valid *restriction* of the original (src-redefine.6.1.2/7.2.2); a redefined attributeGroup may not drop or widen an attribute. | reuses `ParticleRestriction` + attribute-restriction (the most false-positive-prone machinery); needs the original resolved across containers. Fresh focus. |
| `particlesIk` (011/025/027) | ~3 | NameAndTypeOK union membership: a restricting element's type must derive from a *member* of the base element's union type. | deliberately bypassed earlier to avoid a false positive (`string` is a valid member of `union(decimal\|string)`); chain-walk cannot model union membership. |
| `wildI`, `wildZ` | ~4 | Unique Particle Attribution for WILDCARDS: two overlapping `<any>` (or `any` vs element) make a non-deterministic content model. | `ContentModelDeterminism` is the most false-positive-prone area; wildcard overlap must be added carefully. |
| `schN`, `schI`, `schZ011`, `schO`, `schZ006`, `schN10` | ~12 | chameleon / cross-namespace include-import-redefine, self-pointing redefine, duplicate-in-redefine. | cross-document namespace semantics; intricate. |
| `particlesEa/Fb/Ha/Hb/K/M/V`, `elemZ`, `ctZ`, `stZ`, `addB`, `test`, `xsd*.e` | ~28 | heterogeneous particle-restriction edges, element derivation/substitution, anySimpleType edges, adhoc grab-bags. | per-case triage; several need PSVI/model enrichment. |

### Irreducible by design / disclosed (document, do not "fix")

| Cluster | ~Count | Reason |
|---|---|---|
| `particlesZ` (RecurseAsIfGroup / MapAndSum ambiguities, e.g. Z001) | a few | the test's own documentation calls the rule ambiguous; matches the Xerces strict reading (Category D). |
| `reK`, `RegexTest_*` | ~3 | the pattern engine deliberately tolerates `.unsupported`/`.badEscape` categories to avoid rejecting valid-but-unsupported patterns (a disclosed under-rejection, not a defect). |

### Process note

Three autopilot rules this session (redefine self-reference, identity-constraint
xpath prefix, redefined attributeGroup self-reference count) each passed an
adversarial critic pass before merge; the critic caught three genuine false
positives the W3C suite and the differential both missed (two namespace-collision,
one nesting-boundary), each fixed before merge. Any rule that resolves names or
walks references MUST namespace-gate and bound its walk, and MUST go through the
critic loop. The intricate clusters above are best executed supervised, one at a
time, with that loop.