A complex or simple type may not derive, transitively through its base, from itself. Such schemas were accepted. The check walks the base chain built from each type's complexContent/simpleContent extension/restriction base (and a simpleType's restriction base), following an edge only when the base resolves to the schema's own target namespace. Three exceptions keep it from over-rejecting: element content that recurses (an element of the type's own type) is not a derivation and is not followed; a type redefined in xs:redefine legitimately names itself as its base, so redefined types end the chain; and the ur-types (anyType/anySimpleType/anyAtomicType, which the schema-for-schemas bootstraps with self-referential definitions) end the chain. The target-namespace gate prevents a local type whose base is a same-local-name type in another namespace from being mistaken for a self-cycle (found by an adversarial critic; the engine resolves by local name). XSTS: invalid schemas accepted 519 to 516 (addB101, elemM003, xsd007.e now rejected), no other bucket raised. Union/list and genuine cross-namespace derivation cycles are a disclosed under-rejection.
9.5 KiB
Schema-validity burndown
The standing plan for closing the largest XSTS conformance gap. Autopilot
contract: follow this document iteration by iteration, each as its own PR with
a critic pass, until invalid-schemas-accepted is driven down to its
irreducible tail. Do not deviate to new features while this is open.
The problem (measured 2026-06-13, 2006-11-06 archive)
| Bucket | Count | Reading |
|---|---|---|
| valid-schemas-rejected | 72 -> 4 | production target is 0: rejecting a correct schema is the most user-hostile defect (#148). Particle-restriction over-rejection fixed: pointless maxOccurs=0 particles removed, content-free restriction of an EMPTY base, empty-derived emptiable-base, and (#163) the pairwise check over the normalized content model with occurrence subsumption applied only to same-kind pairings (cross-kind element/group, group/element, group/wildcard match occurrence inside, by effective total range / member). #164 resolved element-ref namespaces by prefix binding (the particlesJj/Jk/Jm/Jn/Js, particlesQ/R cluster, -28). Content-free (maxOccurs=0) derived groups and substitution-group declaration order closed particlesW006/elemZ027_a; the W3C MapAndSum rule (Sequence:Choice, #165) closed particlesV003. Remaining 1 (particlesZ001): a documented spec ambiguity (RecurseAsIfGroup; the reference rejects it), not a defect. |
| invalid-schemas-accepted | 2461 -> 516 | we rarely catch a bad one (facet, default/fixed value, particle-restriction name+type, id, structural content-model, UPA determinism, content order, identity-constraint XPath-subset, wildcard-namespace, final/block, simpleType content, group/wildcard and group/element cardinality, element-ref namespace, sequence/choice MapAndSum, circular type derivation landed). |
| valid-instances-rejected | 233 -> 180 | (instance side; #164's element-ref namespace fix is shared with instance validation, -34) |
| invalid-instances-accepted | 160 |
One-sentence diagnosis: the validator runs its instance-validity rules but not its schema-validity rules. It compiles what it understands and ignores the rest, so ~18% of the invalid-schema corpus is accepted silently.
Source split of the 2461: 2323 Microsoft, 138 Sun. The Microsoft mass spreads across every category (DataTypes 703, ComplexType 262, IdentityConstraint 221, SimpleType 139, ModelGroups 137, Attribute 124, Particles 118, Wildcards 97, Element 96, Notations 84, Group 82, Regex 74, Schema 61, AttributeGroup 52, Additional 39, Annotations 30, Errata 4). No single root cause; this is a campaign of distinct schema-component constraints.
Where the checks belong
Schema-validity findings are reported at compile time, through
PureXML.Validation.XSDSchema.consistencyErrors(...) (called from
SchemaDocument.Document.init), which already throws SchemaError.inconsistent
when a named type violates a rule. New rule families are added there (or in a
sibling validation composed into it), never at instance-validation time. The
existing pattern: collect ALL findings, report them together.
Priority order (highest leverage first, measured)
Each iteration targets one self-contained rule family with a clean root cause.
- Facet-definition validity (XSD Part 2 §4.3). Mostly done (2461 -> 1789).
Landed: length-family facet values must be
nonNegativeInteger(totalDigitsapositiveInteger);lengthexcludesminLength/maxLength;minLength<=maxLength;fractionDigits<=totalDigits. Value-bound facets (min/maxInclusive,min/maxExclusive) andenumerationvalues must be valid in the base value space;InclusiveexcludesExclusiveon each side; lower bound may not exceed (nor, if exclusive, equal) the upper. (The bound-value check exposed and fixed agMonth--MM--lexical gap.) Still open in this family: facet applicability per base (fractionDigitsonly on decimal-derived,lengthonly on string/binary/list, etc.);whiteSpacemay not weaken an inherited value; facet repetition. Inherited-facet co-occurrence (across derivation steps) is deliberately not checked yet (current pass reads only the local restriction). idattribute validity (xs:ID). Done (1789 -> 1688). Theidattribute on any XSD component is xs:ID: each value must be a valid NCName and unique within the schema document. Walks the schema-document tree, skippingappinfo/documentationforeign content and treating only the unprefixedidas the component identifier. Broad reach (idA-idK, attgA, attB, ctA).- Component-name uniqueness. Done (909 -> 891). Global type (one space for
simpleType+complexType), element, attribute, model-group, attribute-group names
unique; identity-constraint names unique per document; keyref
referresolves to a key/unique (SchemaNameUniqueness.swift). - Resolvable references. Done (1092 -> 1051).
type/base/itemType/memberTypestypes,element/attribute/group/attributeGroupref, and elementsubstitutionGroupmust resolve to a declared component or a built-in (SchemaReferences.swift), matched by local name, skipped whenimport/include/redefinemay supply externals. Still open: keyrefreferresolution; the context-sensitive applicability rules (formon a global decl,refexcludingname/type). - Facet applicability per base (the open facet-family tail): which facets
a base admits (
fractionDigitsonly on decimal-derived,lengthonly on string/binary/list, etc.), and a fixed facet may not be changed. - Schema-for-schemas structural validity. Content-model pass done
(1688 -> 1313, -375). A data-driven
allowedChildrentable (the schema-for- schemas child content model) rejects disallowed children, multi/misplacedannotation, and identity constraints missing selector/field, via one general mechanism (SchemaStructure.swift). Attribute value validity also landed (enumerated attributes,minOccurs/maxOccurs; 1313 -> 1190), plus name/reference lexical validity (namean NCName;type/base/ref/itemType/refer/substitutionGroupQNames; 1190 -> 1129), pluspatternregex validity (compile each pattern; reject only unambiguous syntax errors, tolerating engine gaps; 1129 -> 1092), plus complexType content-model shape (one simpleContent/complexContent, exclusive with model groups/attributes; one model group; 1051 -> 997), plus particle minOccurs <= maxOccurs (997 -> 977), plus named-group exactly-one-compositor (977 -> 956), plus attribute applicability (allowed-attributes-per-component table + ref/name/type exclusion; 956 -> 925), plus Element Declarations Consistent (same-name elements in one content model share a type; 925 -> 909). Still open: attribute applicability (which attributes each component admits, e.g.formis prohibited on a global declaration,refexcludesname); child order beyond the leading annotation; and particle / model-group semantic rules (Unique Particle Attribution, Particle-Valid-Restriction completeness).
Per-iteration protocol (the PR-critic-loop)
Read docs/production-readiness.md first, every iteration, and hold the change
to its four stoppers and checklist. The target for all four XSTS buckets is 0,
both directions; valid-rejected is not an acceptable baseline. Durability
(commit, ff-merge, push, mirror) is part of done.
For each cluster, on a fresh branch off main:
- Scope & measure. Identify the cluster's cases in
/tmp/xsts-failures.txt; read a sample of the actual invalid schemas to pin the exact rule(s). State the rule in one sentence (first principles). - Implement at the owning layer (compile-time consistency check). Make impossible states unrepresentable where it helps; no special-case bandaids.
- Unit tests for the rule: a valid schema passes, each invalid form is rejected with a located, specific message. Swift Testing.
- Gates (all must pass):
bash scripts/check-style.sh,bash scripts/check-namespacing.sh,swiftformat . --config .swiftformat --lint,swiftlint --config .swiftlint.yml --strict,swift build,swift test,bash scripts/check-wasm.sh. - XSTS ratchet.
XSTS_ROOT=/private/tmp/xsts/xmlschema2006-11-06 swift test -c release --filter XSTS.invalid-schemas-acceptedmust fall; no other bucket may rise (hard gate). Update the pinned baselines inTests/XSTSSuiteTests.swiftto the new numbers. - Critic pass (mandatory before merge). Spawn an independent reviewer over
git diff main...HEAD: correctness (does the rule over- or under-reject? spec citation), removed-behavior, cross-file impact, and altitude (is this a bandaid or the real rule?). Treat findings adversarially; fix every real one and re-run gates + XSTS. A finding that the rule is too aggressive (a valid schema now rejected) is a release blocker. - Document & commit. CHANGELOG entry with the before/after XSTS numbers;
update this file's priority table and
docs/xsts-deviations.md. Commit<type>(schema): …, then fast-forward merge tomain(local only; never push, per policy). Delete the branch.
Invariants
- The ratchet only falls. A rising bucket is a regression and blocks the merge.
- Disclosed limits are fine; silent debt is the violation. Until the schema side
is solid the README must state the conformance numbers and what is not yet
checked, and the package stays
0.x. - Apple/Swift-native only; no external SwiftPM deps; macOS + Linux + WASI.