The two monolithic XSD content/identity rules only satisfied the OpenAPIKit
validation idiom at the wrong granularity: no per-constraint removability,
no isolable failing/near-miss tests. Split them into 32 named rules, each
finding tagged at its origin and cached per XSDContext so the walk still
runs once per document (the #101 decision against duplicating XSD's
recursive type resolution still holds). Wired the streaming path's ID/IDREF
and identity checks through the same framework instead of calling the
validators imperatively, and added the unwrapAndLookup combinator the rule
already named as required.
Also includes the mandated repo-wide swiftformat/swiftlint pass.
NameAndTypeOK clause 1 (XSD 1.0 section 3.9.6, Elt:Elt) pairs the declared
element declarations and requires the same {name} and {target namespace};
no clause grants substitution an allowance. The substitution-group
expansion rewrites the compiled term (an abstract head is dropped, members
join a synthetic choice), so the checker previously compared the expanded
names and a head-for-member swap looked name-identical.
Particle.declaredReference now preserves the pre-expansion identity of an
element-ref particle, and the restriction check pairs the declared
references before the structural pairing table (extracted as
structuralValid). Closes XSTS elemZ026 and particlesV020:
invalid-schemas-accepted 8 -> 6, valid buckets unchanged at zero.
Stricter than libxml2, which does not implement this edge; elemZ026 is
pinned in the differential baseline accordingly.
particlesZ008, byte-for-byte the same construct as elemZ026 but carrying
a settled valid expectation contradicting elemZ026's settled invalid, is
reclassified as a spec-divergent schema test with the adjudication
recorded in the suite (exclusion tables split into
Tests/XSTSSuiteExclusions.swift for file length).
Separately attributed: the differential baseline gains ten further pinned
entries (addB070a, addB108, addB177, attO025, elemZ027_c, errC008,
groupH021v, notatB005, notatE002, particlesEb040) that reproduce with
this change stashed. They are pre-existing drift from earlier main-line
fixes, not effects of this commit. Baseline 231 -> 242, differential
green.
Two constructs are flat in the schema document, so they escape the parser's element-nesting maxDepth (256) yet make a content model arbitrarily deep once compiled: a <xs:group ref> chain and a complexContent derivation chain (the definitions are siblings). The particle compiler inlines each group reference and composes each derivation once per link, overflowing the stack on a long chain. The expanded depth is now bounded by one counter (XSDContext.expansionDepth, incremented at each compositor level, group-reference expansion, and derivation step) against maximumContentModelExpansionDepth (512); a model past the bound is rejected with a located error, not crashed. The counter lives in a value-type context so it tracks depth not width (no width false positive), and the rejection is harvested into the schema's errors (no silent truncation). Verified conformance-neutral over the full W3C XSTS corpus (all four buckets unchanged), the libxml2 determinism differential unchanged, and the full suite; an adversarial critic found the derivation-chain gap, which this fixes with the same counter.
Step 1 of the GSS content-matcher build (docs/design/counted-content-automaton.md).
A multiply-referenced `<xs:group>` expands to one copy-on-write-shared `Particle`
(GroupParticleMemo), but `Group`/`Term` are value types with no exposed identity,
so a content-model compiler cannot tell two references apart and inlines the group
per reference (the 2^K build the GSS matcher will close).
Group now carries a `definitionID: Int?`, stamped from a per-compile counter on
GroupParticleMemo when an expansion is first compiled, so every reference sharing
the memo key reuses the same stamped term and distinct definitions differ. It is
metadata only: Group is not Equatable/Hashable, so it perturbs no determinism or
restriction comparison, and it is nil for an inline group.
FP-neutral by construction (no consumer yet). Full in-repo suite green; the XSTS
buckets are unchanged (valid-schemas-rejected 0, invalid-schemas-accepted 8,
valid-instances-rejected 0, invalid-instances-accepted 14); swiftformat/swiftlint
clean. Step 2 (a call/return content-model build keyed on this id, alongside the
inlining oracle) consumes it.
Compiling a schema with deeply nested, multiply-referenced model groups was
exponential: groupReferenceParticle inlined each <xs:group ref> into the Particle
content model, so g0=(g1,g1), g1=(g2,g2), ... built 2^K particles (depth 14 =
0.21s, depth 20 ~13-18s). The expansion is now memoized per (group definition,
cycle-visiting set) in a per-compile GroupParticleMemo shared across the value
copies XSDContext.scoped/visiting make, so sibling references share it and the
build drops to O(groups) (depth 40 now ~0.016s; previously infeasible).
The shared content term is reused by value (copy-on-write), so the compiled model
is identical and validation is unchanged: the XSTS gate holds (invalid-schemas-
accepted 15, valid-schemas-rejected 0, instance buckets unchanged) and the
streaming-vs-tree fuzz differential still passes. This closes the schema-compile
half of the interactive-safety group blowup (companion to the positionCap
determinism fix); a depth-40 compile test guards against regression. Full suite,
style, namespacing, and WASM all pass.
An xs:all group's own minOccurs="0" makes the GROUP optional (it occurs zero
or one times), not each member optional. The parser forced every member's
minOccurs to 0 whenever the all-group was optional, so a non-empty but
incomplete optional all-group (a required member missing) was wrongly accepted,
and a complexContent restriction that relaxed such a required member also slipped
past the particle-restriction check (the zeroed minOccurs made it look already
optional). Members now keep their declared occurrence; the absent-group case is
handled at validation time by skipping the required-member check only when the
all-group contributes no children at all. Applied identically on the tree and
streaming paths (both call allStructureErrors). A required all-group is
unchanged. Catches mgZ001 (instance) and particlesK006 (schema). XSTS invalid-
instances-accepted 21 to 20 and invalid-schemas-accepted 18 to 17, both valid-
rejected buckets held at 0.
The instance-validity derivation and substitution tables were keyed by bare
local name, so two same-local-name types or elements in different namespaces
collided. In multi-document validation this let one schema's block/derivation
facts shadow another's, and it blocked honouring blockDefault (whose re-land
widened that collision into an over-rejection).
Key the instance subsystem by namespaced identity {ns}local instead. The shared
DerivationTables keeps its bare-keyed fields for the schema-validity machinery
(Particle-Valid-Restriction and the consistency rules, which resolve QNames to
bare names and walk the same backbone) and gains parallel ns* fields built in
the same pass, resolving each container's components through its target
namespace and prefix bindings. Only the instance consumers
(blockedSubstitutionError, notDerivedSubstitutionError, abstractTypeError, the
substitution-group closure, and referenceParticle) read the ns* tables, via a
derivationKey(fromReference:) that reduces a type:{ns}local reference, a bare
name, or an element ref to a {ns}local backbone key. A key miss stays permissive
(no constraint, substitution allowed), so the re-keying cannot over-reject.
With the namespaced backbone in place, blockDefault is re-landed FP-free: a
schema's blockDefault supplies the {disallowed substitutions} of every element
and complex type without an explicit block (own block overrides; block="" clears;
intersected with the methods each component admits), so blockDefault="extension"
bars an extension substitution and "#all" bars all, without one namespace's
defaults shadowing another's.
The pure re-keying preserves single-document behaviour exactly (the corpus is
overwhelmingly single-namespace, where {ns}local and bare coincide): all four
XSTS buckets held at vsr=0 / isa=69 / vir=0 / ivi=89 after re-keying alone.
blockDefault then drove invalid-instances-accepted 89 to 81, no false positive
(valid-instances-rejected and valid-schemas-rejected held at 0), no other bucket
moved. An adversarial review exercised same- and cross-namespace substitutions,
no-targetNamespace keys, simple-type restriction, transitive substitution groups,
and blockDefault override/over-block cases, and the prior multi-document
cross-namespace blockDefault over-rejection is now accepted while a same-namespace
blocked substitution is still rejected (precise, not merely permissive).
A restricting element may not be nillable unless the base element it restricts
is also nillable. The element particle (Term.element) now carries a nillable
flag, populated from the element's nillable attribute.
XSTS invalid-schemas-accepted 144 to 142, no other bucket moved.
Particle Valid (Restriction) now requires a restricting element's {disallowed
substitutions} to be a superset of the base element's: a restriction may block
more substitutions, never fewer. The element particle (Term.element) carries a
block set, populated from the element's block attribute or the schema's
blockDefault.
XSTS invalid-schemas-accepted 179 to 163, no other bucket moved.
Consolidates the in-progress #146 work; valid-instances-rejected reaches 4
on the 2006-11-06 XSTS archive (buckets now 1/262/4/147).
Regex: match Unicode scalars rather than grapheme clusters; the empty
pattern matches only the empty string; pin the category repertoire to
Unicode 3.2, the version the corpus is consistent at, so code points added
in 3.2 (Tagalog, Cyrillic Komi) are word characters while later additions
(the Tamil digit zero) stay unassigned and match \D. New XSDCategory and a
3.2 assignment table derived from DerivedAge, replacing the old Unicode31
files.
Content model: assess each child against the particle the content automaton
matched, not a by-name lookup (#180). Two same-named particles keep their
own fixed values, and an element matching a skip wildcard is no longer
validated against its global declaration. Selection is made reproducible
when an ambiguous schema slips past the UPA check.
XSTS runner: compile and merge every schemaDocument in a schemaTest, not
just the first; this also surfaced one previously-masked invalid-instance
acceptance (recorded in the baselines).
Also folds in attribute-group anyAttribute inheritance, extension wildcard
union, namespace-qualified type tables for multi-document
xsi:schemaLocation, per-container elementFormDefault, strict and lax
attribute wildcard validation, built-in type resolution, and xs:redefine
simpleType overrides.
