The two monolithic XSD content/identity rules only satisfied the OpenAPIKit
validation idiom at the wrong granularity: no per-constraint removability,
no isolable failing/near-miss tests. Split them into 32 named rules, each
finding tagged at its origin and cached per XSDContext so the walk still
runs once per document (the #101 decision against duplicating XSD's
recursive type resolution still holds). Wired the streaming path's ID/IDREF
and identity checks through the same framework instead of calling the
validators imperatively, and added the unwrapAndLookup combinator the rule
already named as required.
Also includes the mandated repo-wide swiftformat/swiftlint pass.
The counted content-model automaton represents its active set as
(state, counter-vector) configurations bounded by states times the product of
the per-counter saturation caps. An unbounded counter already saturates at its
minimum (every higher count is matching-equivalent). A finite maximum the input
cannot reach is equivalent: the active set increments a counter at most to its
cap, the cap is at most inputLength+1, so when maximum > inputLength+1 the value
can never reach maximum, belowMaximum is always satisfied, and every value at or
above the minimum is indistinguishable. Saturate such a counter at its minimum
too, so a huge but finite maxOccurs on a nested counter no longer multiplies the
active set by an inputLength factor per nesting level.
A document that did not terminate within a minute against the Z035 worst case
(nested sequences/choices with maxOccurs up to 1e11) now validates in linear
time. No accept/reject decision changes: the full XSTS suite holds at
0/35/0/26, and an adversarial no-caps reference-matcher differential found no
false positive or false negative, including the nullable-body case (where the
builder makes canExit unconditional, so the counter value is matching-irrelevant)
and the occurrence boundary. Guarded by ContentMatcherWorstCaseTests.
Partial: a model with several moderate (input-reachable) maxOccurs nested deeply
still enumerates a large active set; the general polynomial bound (a position-
indexed DP over the counting automaton) remains under #192.
The counted content automaton clamped every occurrence counter to a single
inputLength + 1 ceiling, so a content model with nested occurrence bounds let
the epsilon-closure grow toward inputLength raised to the counter count. The
XSTS particlesQ005 instance (about 1000 elements against
sequence{1,1000}(e1{1,unbounded})) drove that to roughly 16M configurations per
closure and hundreds of millions of hash operations, so validation did not
terminate in reasonable time.
Each counter now saturates at min(maximum, inputLength + 1), and an unbounded
counter at its minimum: once the minimum is reached every higher count is
matching-equivalent, because canExit only tests the minimum and belowMaximum is
vacuous, so collapsing them is exact. The active-configuration set is then
bounded by the content model rather than by the input length.
particlesQ005 goes from a multi-minute near-hang to about two seconds with
identical results. The 1448 unit tests pass and the full XSTS suite still
settles at 0/43/0/31, and now completes instead of hanging.
A restricting element may not be nillable unless the base element it restricts
is also nillable. The element particle (Term.element) now carries a nillable
flag, populated from the element's nillable attribute.
XSTS invalid-schemas-accepted 144 to 142, no other bucket moved.
Particle Valid (Restriction) now requires a restricting element's {disallowed
substitutions} to be a superset of the base element's: a restriction may block
more substitutions, never fewer. The element particle (Term.element) carries a
block set, populated from the element's block attribute or the schema's
blockDefault.
XSTS invalid-schemas-accepted 179 to 163, no other bucket moved.
Consolidates the in-progress #146 work; valid-instances-rejected reaches 4
on the 2006-11-06 XSTS archive (buckets now 1/262/4/147).
Regex: match Unicode scalars rather than grapheme clusters; the empty
pattern matches only the empty string; pin the category repertoire to
Unicode 3.2, the version the corpus is consistent at, so code points added
in 3.2 (Tagalog, Cyrillic Komi) are word characters while later additions
(the Tamil digit zero) stay unassigned and match \D. New XSDCategory and a
3.2 assignment table derived from DerivedAge, replacing the old Unicode31
files.
Content model: assess each child against the particle the content automaton
matched, not a by-name lookup (#180). Two same-named particles keep their
own fixed values, and an element matching a skip wildcard is no longer
validated against its global declaration. Selection is made reproducible
when an ambiguous schema slips past the UPA check.
XSTS runner: compile and merge every schemaDocument in a schemaTest, not
just the first; this also surfaced one previously-masked invalid-instance
acceptance (recorded in the baselines).
Also folds in attribute-group anyAttribute inheritance, extension wildcard
union, namespace-qualified type tables for multi-document
xsi:schemaLocation, per-container elementFormDefault, strict and lax
attribute wildcard validation, built-in type resolution, and xs:redefine
simpleType overrides.
Complete the element-vs-element type clause of Particle Valid (Restriction):
the restricting element's type must be, or derive by restriction from, the base
element's type. Previously only names were compared, so an unrelated, wider, or
ur-type element type was accepted under restriction.
Enrich the content-model particle: Term.element carries the element type's
derivation identity (the resolved type name, anyType for an absent type, nil for
an inline anonymous type). Record named simpleType restriction derivations in the
derivation table alongside complex ones. The element-vs-element check walks the
user restriction chain and then the built-in lattice to decide derivation by
name; an ur-type base, a union base (membership is not a restriction chain), and
an inline anonymous type are permitted, so no valid restriction is rejected.
XSTS: invalid schemas accepted 616 to 594, invalid instances accepted 165 to 160
(more invalid schemas rejected up front), valid schemas rejected held at 72,
valid instances rejected held at 214.
Running the full W3C XSTS archive to completion exposed three crash and hang
classes, now fixed with regression tests, which unblocked the #129 first
measurement.
- Content-model and regex NFA builders now cap total state count (2^20),
degrading further occurrence/quantifier repetition to star, so nested
high-maxOccurs schemas no longer OOM (was 8 GB on the MS Particles set).
- Content-model validation advances one active state-set per child instead of
re-walking the prefix each time, removing the O(children^2) spin.
- Regex class subtraction [abc-[b]] no longer traps on a reversed range, and a
reversed range such as [z-a] is rejected cleanly.
XSTS first measurement baselined: 75 valid schemas rejected, 2467 invalid
schemas accepted, 604 valid instances rejected, 582 invalid instances accepted
across 14383 groups. The suite runs green in 8s, opt-in via XSTS_ROOT.
Refs #129, #122.
The W3C XML Schema Test Suite (2006-11-06, ~13k groups; W3C Document
License, never vendored) runs opt-in via XSTS_ROOT: schemaTests must
compile or be rejected per their expected validity, instanceTests
validate against their group's schema, hrefs resolve against the
referencing file, and include/import locations from disk relative to
the schema. Disagreement counts are asserted exactly; baselines are
pending the first measurement (scheduled 2026-06-14), and failures
are written to /tmp/xsts-failures.txt for the burn-down. Building the
runner immediately caught a denial of service: maxOccurs in the
trillions (groupF009v) OOM-killed the process because the content
automaton unrolled occurrence counts literally; unrolling is capped
at 16384 with larger repetitions treated as unbounded, the libxml2
posture. Roadmap mermaid records the closed behavioral-parity set
(#141-#144, collation), the #139 performance epic, and XSTS moving
to runner-in. Advances #129.
Add xs:any and xs:anyAttribute wildcards with namespace and
processContents constraints (skip/lax/strict), and resolve the
list built-in datatypes (IDREFS, ENTITIES, NMTOKENS) to their
item types so each list item validates.
WildcardNamespace models ##any/##other/enumerated URI sets;
Wildcard.admits gates undeclared attributes and wildcard-matched
children. ContentNFA.follow exposes the follow set for completions.
Closes#77
Slice 3 of #94. From the compiled schema, compute exactly what is allowed at an
element, for autocomplete and "what's missing".
- ContentNFA.follow(after:) returns the precise follow-set: the element-name
labels the content automaton can accept next after the children already
present, plus whether the content may legally end there.
- Schema.Completions reports the allowed next elements, the complete flag (false
means a required child is still expected), a wildcard flag, and each declared
attribute with its required and present status.
- CompletionEngine resolves an element's declared type along a coding path (the
global declaration, then each parent's content model) and computes the
completions; Schema.Document.completions(at:in:) is the entry over a ranged
tree.
Tests pin the follow-set as the children accumulate (name then qty then the
optional note, complete flipping true), the attribute required/present status,
and resolution for a nested element by its coding path.
Add ComplexValidator validating an element against a ComplexType: attribute
uses (required, typed, unknown-attribute rejection) and the content model
(empty, simpleContent, elementOnly, mixed). Particles compile to a Thompson
NFA over element names honoring minOccurs/maxOccurs with sequence and choice
compositors; all groups are validated order-independently by counting. Child
elements are validated recursively against their declared types.
Closes#9
