particlesZ001 declares Derived as a complexContent restriction of Base and an
instance whose <elem> (type Derived) content is annotation + four elements. The
suite pairs a valid schemaTest with an invalid instanceTest, but the two are
mutually exclusive for a consistent conformant processor, and the manifest's own
documentation labels the governing rule, Particle Derivation OK (RecurseAsIfGroup),
ambiguous.
cvc-complex-type (§3.4.4) validates an instance against the element's own
{content type}: for <elem> that is Derived = sequence(annotation?, element*),
which the instance satisfies, so the instance is valid. Particle Valid
(Restriction) / RecurseAsIfGroup (§3.9.6) is a schema-assembly constraint that is
never re-run during instance validation, so there is no path from an accepted
schema to an invalid instance. The only way to reach the suite's invalid is to
reject the schema as an invalid restriction, which is a valid-schema-rejected
false positive and would still fail the suite's own valid schemaTest. libxml2
accepts both schema and instance, agreeing with PureXML.
Exclude particlesZ001.i via specDivergentInstances with the spec-grounded
justification in code and docs/xsts-deviations.md. invalid-instances-accepted
5 -> 4; full XSTS in release now 0/8/0/4, both false-positive buckets at 0. No
library behaviour change. Confirmed by an adversarial analysis that found no
spec-coherent reading yielding valid-schema + invalid-instance.
mgO013 redefines a model group `group` to `all{e1(1,1)}`, redefining a base
`all{e1(0,0)}` (e1 forbidden) without referencing itself. The suite pairs a
`valid` schemaTest with an `invalid` instanceTest, but the two are mutually
exclusive for a consistent conformant processor.
src-redefine clause 6.2.2 makes a no-self-reference group redefine a restriction
obligation: the redefined group must be a valid restriction of the original.
`all{e1(1,1)}` is not a valid restriction of `all{e1(0,0)}` (Particle Valid
(Restriction), Occurrence Range OK: derived max 1 > base max 0), so enforcing
6.2.2 rejects the schema, a valid-schema-rejected false positive against the
schemaTest's own `valid` verdict. Yet §4.2.2 makes the referencing type's group
reference resolve to the redefined `all{e1(1,1)}`, against which the single-e1
instance is valid. There is no third path: reaching instance-invalid requires
rejecting the schema. libxml2 accepts both, agreeing with PureXML and the
schemaTest. An adversarial analysis confirmed no reading yields valid-schema +
invalid-instance.
Exclude mgO013.i via specDivergentInstances with the spec-grounded justification
in code and docs/xsts-deviations.md. invalid-instances-accepted 7 -> 6; full XSTS
in release now 0/8/0/6, both false-positive buckets at 0. No library behaviour
change: matching the suite would introduce the false positive the production bar
forbids.
The sch* schema-location cluster, dissected per case rather than swept. An
adversarial analysis separated the contested tests from the genuine deviations.
schA2.i and schA5.i are contested and now excluded (invalid-instances-accepted
10 -> 8). Each group's own documentation states the inline xsi:schemaLocation
hint "is ignored" and the instance "must conform to" the collection schema, but
under exactly that prescribed behavior the instances are valid: libxml2 validates
schA2.xml against its a.xsd and schA5.xml against its a.xsd+b.xsd collection. So
the suite's invalid expectation contradicts its own documentation. Reaching
invalid requires letting the instance's hint override a schema already loaded for
that namespace, which XSD 1.0 §4.3.2 makes optional (strategy 1: do nothing when a
schema for the namespace is already known) and which would be a direct false
positive for an IDE: an instance file forcing rejection against the project's own
schema. PureXML's policy (honor hints but keep the collection's declarations on
conflict) is conformant.
schA7.i is NOT excluded. The adversarial pass showed its invalid IS spec-grounded:
with hints ignored, the instance's b:e1 (namespace ns-b) has no declaration in the
a.xsd-only collection and is demanded by root's default-strict ##any wildcard, so
§3.10.4 makes it invalid. PureXML accepts it only by honoring the ns-b hint to
gap-fill, the other conformant §4.3.2 strategy. A genuine policy-dependent
deviation, left in the count and documented.
schU3.i/schU4.i/schU5.i are also left in the count, documented as a deferred
under-rejection in an underspecified area: circular redefine chains where an
attribute group is redefined to conflicting types across an include/redefine
cycle. The suite expects the int reading (so c1="a" is invalid); PureXML resolves
the laxer type and accepts. libxml2 does not support the construct at all (rejects
the schema: multiple redefinition not supported), so matching the suite means
out-resolving the reference on a construct whose cyclic composition order the spec
does not pin down.
Full XSTS in release now 0/8/0/8, both false-positive buckets at 0. No library
behavior change; reasoning recorded in code and docs/xsts-deviations.md.
The W3C XSTS group anyURI_a004_1339 pairs a `valid` schemaTest with an `invalid`
instanceTest, but the two expectations are mutually inconsistent, so PureXML's
acceptance of the instance is spec-correct rather than an under-rejection.
The schema's include, import, and redefine carry unresolvable schemaLocations
(ftp://, gopher://, mailto:). XSD 1.0 §4.2.1 makes an unresolvable include "not
an error" (no inclusion performed), the import likewise, and the empty redefine
imposes no resolution requirement (src-redefine clause 1 gates that on
non-annotation children), so the schema is conformantly valid, agreeing with the
schemaTest. The only components the instance exercises (root, bar, st, att) are
fully locally defined, and every instance value is byte-identical to one of the
type's six anyURI enumeration members, so a strict assessment of the document
element is valid. There is no spec path to valid-schema + invalid-instance.
libxml2 is no authority here: it non-conformantly fails to compile the schema on
the unresolvable include, so it disagrees with the suite's valid-schema verdict
too. Matching the suite's invalid-instance expectation would mean rejecting a
valid instance whenever an include cannot be resolved, a false positive an IDE
editing loop hits constantly, which the production bar forbids.
Exclude it from the counts via specDivergentInstances, mirroring how the suite's
own status="queried" entries are skipped, with the spec-grounded justification
recorded in code and docs/xsts-deviations.md. invalid-instances-accepted 11 -> 10;
full XSTS in release now 0/8/0/10, both false-positive buckets at 0. Two
independent adversarial analyses failed to refute the instance's validity.
XSD 1.0 (Datatypes Appendix F) defines the regex escape \i as the characters
that may begin an XML Name (Letter | '_' | ':') and \c as the characters that
may appear in one (NameChar), citing the XML 1.0 Second Edition productions:
the enumerated, Unicode-2.0-era character classes of XML 1.0 Appendix B
(BaseChar, Ideographic, CombiningChar, Digit, Extender). \i and \c were wired
to PureXML.Parsing.XMLCharacter, which correctly implements the Fifth Edition
Name production used to parse live XML, but the two genuinely differ: the Fifth
Edition admits whole blocks where the Second Edition has gaps, so \c
over-accepted, e.g. the combining mark U+0346 (COMBINING BRIDGE ABOVE), which
Appendix B's CombiningChar excludes (its first range ends at U+0345).
Add XSDNameCharacter, transcribed clean-room from the Appendix B grammar
(identical across editions one through four) and stored as sorted, non-
overlapping code-point ranges with binary-search membership; the parser's own
name check is untouched. The two tables reproduce the W3C XSTS \i/\c
enumerations (reZ005, reZ006: 34516 and 35122 code points) exactly and in both
directions, so the more restrictive definition rejects no spec-valid input:
there is no false-positive surface.
Verified over the full W3C XSTS corpus in release: invalid-instances-accepted
12 -> 11 (reZ006i, whose value is U+0346, now rejects), both false-positive
buckets at 0. Full suite (1834 tests) green; an adversarial critic brute-forced
the binary search over the whole code-point space (0 mismatches), confirmed the
tables are sorted and non-overlapping and equal the oracle sets, and that the
parser's Fifth Edition path is left intact. Found by triaging the invalid-
instance tail to a single boundary code point (U+0345 valid vs U+0346 invalid)
and reading the XML 1.0 Appendix B grammar.
XSD 1.0 fixes the regular-expression repertoire at Unicode 3.2 (Datatypes
Appendix F), where U+1369..U+1371 (ETHIOPIC DIGIT ONE..NINE) are general
category Nd, each carrying a decimal value; a later Unicode reclassified them
to No. XSDCategory derives \d (== \p{Nd}) from the running platform's category,
pinned to 3.2 by an assigned-in-3.2 gate plus a per-code-point override table,
but the table omitted these nine, so \d used the modern No: a valid \d-constrained
Ethiopic digit was wrongly rejected (cardinal-rule false positive) and a
\D-constrained one wrongly accepted.
Pin the nine to Nd. This is the complete set of code points that were Nd at 3.2
and are not today; the reverse direction is empty (every later-Nd point was
unassigned at 3.2 and already reports Cn), verified by diffing the 3.2 and current
Unicode Nd sets. \w is unaffected: an Ethiopic numeral is category N either way,
neither punctuation, separator, nor other.
Verified over the full W3C XSTS corpus in release: invalid-instances-accepted
14 -> 12 (reT17, reT38, the \D cases, now reject), and the matching \d-valid cases
reS17/reS38/reZ004 no longer need a suspect-test exclusion and pass as ordinary
agreements, with both false-positive buckets at 0. Full suite (1830 tests) green;
an adversarial over-rejection critic re-derived the Nd delta bidirectionally and
confirmed no false-positive surface. A regression-pinning test exercises the
production regex path, including the boundary that U+1372 (No even at 3.2) is
not \d.
* perf(schema): graph-structured-stack content matcher, sharing each xs:group (alongside)
The per-instance content-model build inlines every xs:group reference, so a
multiply-referenced nested group (g0=(g1,g1), g1=(g2,g2), ...) explodes to 2^K
states and reset actions: a depth-16 doubling chain already takes seconds, depth
20+ hangs. This compiles the same particle to a counted Recursive Transition
Network instead, where each definitionID group is compiled once and reached by a
call, and simulates it with a graph-structured stack.
ContentRTNBuilder mirrors ContentNFABuilder exactly except that a group with a
definitionID becomes a call into its shared, compiled-once fragment (memoized by
id; the fragment's accept is a return). The callee's counters are deliberately not
propagated to the caller: every counter is reset on its own particle's entry edge,
which is on the path from the callee's start, so a shared fragment resets itself on
every re-entry without the caller re-listing it. Re-listing was the residual
blow-up: it nested the reset-action lists to 2^K even when the states stayed O(K).
ContentRTNMatcher runs configurations (state, counters, gssNode): a call saves the
caller's counters on a GSS node interned by (returnState, savedCounters, position)
and a return restores them, so the shared callee never mixes counter state across
its callers; parents are a set and a popped node replays its pop to a parent added
later (the GLL completeness detail). The build drops to O(groups): a depth-24
doubling chain (2^24 inlined states) compiles to under 480 states in under a
millisecond.
It is gated by a differential (ContentRTNDifferentialTests) against the trusted
inlining matcher ContentNFA as the oracle: the two agree on the accept verdict and
the matched particle over the hand model cases, the named-over-wildcard tie-break,
the shared-group doubling chains that exercise call and return, and a 4000-case
fuzz of nested models with occurrences, wildcards, and shared groups, plus a bound
assertion. Built alongside the inlining matcher and not yet wired into validation,
so no behaviour change and the XSTS buckets are unchanged; the switch-over is gated
on this differential.
* perf(schema): add the streaming/located surface to the GSS matcher, lock it differentially
The content-model validator reads two things off the inlining matcher: matchedParticles
for per-child attribution, and the streaming startStates/step/admissible/isComplete for
located diagnostics (which child breaks the model, and the "expected a, b" hint). The GSS
matcher only had matchesWhole/matchedParticles, so it could not yet replace the inlining
matcher. This adds the streaming surface: admissible works because the closure already
expands call states into the callee's label states, so the follow set has the same shape
as ContentNFA's.
The differential gate now locks the whole surface, not just the accept verdict: driving
both matchers child by child, the first rejected child (step is nil) agrees, the
admissible expectation set agrees at the start and after every accepted child, and
isComplete agrees once the sequence is consumed. The fuzz now stamps group references the
way the parser does, a unique definitionID per distinct group (so a singly-referenced
named group also takes the call path, not only doubled groups), a shared id only for
genuinely shared structure, and some anonymous inline groups, over occurrences and
wildcards. Still alongside, not wired; no behaviour change, XSTS buckets unchanged.
* perf(schema): complete the GSS matcher surface with follow(after:), verified
follow(after:) is the last ContentNFA method the GSS matcher lacked: the completion
path (XSDCompletions) reads it for the acceptable-next labels and whether the content
may end. It composes the streaming surface, and the differential now locks its allowed
set and complete flag to ContentNFA too. The RTN matcher is now a complete drop-in for
the ContentNFA surface the validator and completions read, so the switch-over is a
mechanical builder swap gated on XSTS neutrality. Still alongside; no behaviour change.
* docs(design): record GSS Steps 1-3 done, the switch as a corpus-gated mechanical swap
Updates the counted-content-automaton Step 4 to the current state: the GSS matcher and
its full differential gate are built and green in-repo, and the production switch is now
a three-site builder swap blocked only on the XSTS neutrality measurement, which must run
on a machine that has the corpus. Records the exact swap sites and the gate so the corpus
machine can execute it mechanically.
* perf(schema): validate and complete on the GSS matcher, killing the last 2^K content-model hang
The three call sites that built the inlining content automaton once per validated
element, ComplexValidatorContentModel.sequenceStructureErrors (located structure
errors), ComplexValidator.validateParticle (per-child particle attribution), and
XSDCompletions (acceptable-next labels), now build ContentRTNBuilder and run
ContentRTNMatcher. A schema whose nested <xs:group> references inline to 2^K positions
(the g{i}=(g{i+1},g{i+1}) doubling shape) no longer makes each keystroke-time validation
of an element exponential in the nesting depth: the automaton is built in O(groups).
Validating a short instance against a depth-40 doubling schema (2^40 inlined positions),
the shape an editor sees mid-typing, now completes in milliseconds with a located error
instead of hanging; a new end-to-end test pins it.
Verified conformance-neutral on the full W3C XSTS corpus in release: valid-schemas-rejected
and valid-instances-rejected stay 0, invalid-schemas-accepted and invalid-instances-accepted
unchanged at 8 and 14. The located diagnostics, per-child attribution, and completions are
behaviour-identical by the surface differential (ContentRTNDifferentialTests), which locks
the verdict, matched particle, rejection index, admissible set, isComplete, and follow to
the inlining ContentNFA. ContentNFABuilder/ContentNFA are retained as that oracle.
* docs(design): mark GSS content-matcher Step 4 done, XSTS-neutral in production
Records that the three production sites are switched to the graph-structured-stack
matcher and the switch was measured conformance-neutral on the full W3C XSTS corpus in
release (all four buckets held). Notes the local corpus path and that ContentNFABuilder
is kept as the differential oracle rather than deleted.
* test(schema): compare full matched-particle attribution; bound a cyclic group id
From the adversarial review of the GSS matcher:
- The differential compared only the matched-particle kind (element/wildcard), so it
could not observe the type/value metadata matchedParticles carries. Every fuzz and
hand element now carries a per-name fixed value constraint and the comparison projects
it, so the gate verifies the metadata is carried identically through call/return, not
only that the kind agrees. A name maps to one constraint, so the comparison does not
depend on the lowest-state-index tie-break (which only distinguishes same-name
particles, a UPA violation the compiler rejects before instance validation); the
matchedParticles doc now records that caveat.
- ContentRTNBuilder.groupFragment memoizes after compiling, so a (malformed) cyclic
definitionID would recur without bound, the interactive-safety hazard this matcher
removes. definitionID groups are acyclic by construction (the parser breaks cycles at
element boundaries, the inlining builder relies on the same), but bound it anyway:
a cycle degrades to a terminating empty fragment instead of recurring.
No production behaviour change; the differential, full suite, and the full XSTS corpus
in release (16/16, all four buckets held) stay green.
The previous section settled the algorithm and the bound proof; this makes the
build turnkey rather than exploratory. It is the full scope of the reverted RTN
attempt (whose group sharing was correct) with the explicit return stack replaced
by the graph-structured stack:
- Step 1: a group-definition identity (`Group.definitionID`, stamped from the
existing `GroupParticleMemoKey` in `XSDParserParticles`), FP-neutral on its own.
- Step 2: a `ContentRTNBuilder` alongside the trusted inlining `ContentNFABuilder`,
compiling each definition once via `call`/`return` epsilon edges, counters still
per-configuration and reset on entry so a shared fragment does not alias.
- Step 3: the GSS simulation: a node `(returnState, parents-set)` interned per
`(returnState, inputPosition)`, configuration `(state, counters, gssNode)`, with
call interning and pop fan-out, deduplicated on the configuration key.
- Step 4: the differential gate against the inlining oracle (XSTS + a
nullable-nested-group fuzz, verdict AND matched-particle equal), then the switch
and a perf assertion on GSS/configuration COUNTS (not a wall-clock timeout, the
reverted attempt's lesson).
Each step is its own PR; steps 1-2 keep the oracle authoritative, step 4 is the
only behaviour change and is gated to equivalence before it lands. Design only.
Closes the proof gap that sank the reverted RTN matcher. The counted automaton
removed the occurrence-magnitude blow-up and the compositional determinism check
removed the determinism group-sharing blow-up, but the per-instance matcher build
(ContentNFABuilder.group) still expands a multiply-referenced group once per
reference, so a doubling chain g0=(g1,g1)... builds 2^K states and hangs on the
first validate. That is the last place stopper 4 is a hang, not a proven bound.
The reverted pushdown rewrite compiled the group DAG to call/return edges
(O(groups) program) but kept EXPLICIT return stacks, so a nullable doubling chain
made the start epsilon-closure reach ~3^depth distinct stacks: the exponential
moved from the build to the per-keystroke closure. It was verdict-correct yet
bounded nothing, and was reverted.
This section derives the fix: a graph-structured stack (the GLL/Earley device).
A GSS node (returnState, parents-set) shares stack suffixes, so configurations are
keyed on (state, counters, gssNode) and the live set is bounded by O(states), not
by the number of distinct stacks. The nullable doubling chain that produced
3^depth explicit stacks produces O(K) GSS nodes. The section states the size and
termination bounds (counters clamped to N+1, non-recursive group DAG), the
deterministic-model O(N*S*D) match, and the same alongside-then-differential-gated
rollout that made the compositional-determinism change FP-safe, with a perf
ASSERTION on configuration/GSS counts rather than a wall-clock limit (the reverted
attempt's lesson). Design only; no code change. Status: DESIGNED, not implemented.
Element stored its children inline and Node was `indirect`, so a Node's
recursive layout made building, comparing, hashing, and releasing a deeply
nested tree recurse one native frame per level and overflow the stack
(signal 10) -- on a constrained stack such as a Swift Task, as shallow as a
few thousand deep.
Element now holds its children behind a copy-on-write reference
(ElementStorage), so Node is a fixed-size value (no longer indirect) and:
- `==` and `hashValue` walk the tree iteratively through explicit work
stacks (the synthesized conformances recursed); equal trees still hash
equally.
- ElementStorage's `deinit` flattens the subtree into a heap worklist
before releasing, draining only storage it solely owns so a copy-on-write
sibling keeps its subtree.
- `children` is a computed copy-on-write view with a `_modify` accessor, so
in-place mutation (the parser's append loop) stays amortized O(1).
Value semantics are unchanged; Element/Node remain Equatable, Hashable,
Sendable. A 50000-deep tree builds, compares, hashes, and releases in a
fraction of a second; 1782 tests pass. Completes the value-Node half of #341.
Triaging the 24 open XSTS failures against libxml2 (the stated reference)
separates genuine reference-aligned bugs from stricter-than-libxml2 scope
decisions. On the schema side, 9 of 10 are accepted by libxml2 too (Xerces-
level strictness, false-positive-prone); only particlesZ030_d was a real
gap, now fixed. The instance tail's reference-aligned cases are intricate
and need per-case analysis. Guides the burndown toward bugs, not non-bugs.
The W3C XML Schema Test Suite (2006-11-06) drives the opt-in conformance
runner but is never vendored: the W3C Document License permits
redistribution only as the complete unmodified archive, and the corpus
is about 200 MB. This script downloads the official archive on demand,
verifies its SHA-256, and extracts it to the path XSTS_ROOT expects.
Idempotent and portable (shasum or sha256sum). Points the onboarding doc
at it.
xs:unique/xs:key/xs:keyref were quadratic over a wide list of targets.
The duplicate check compared each value tuple against every tuple already
seen (and a keyref against every referenced key), and the error-location
helper rescanned the parent's whole child list to position each target.
Key tuples in a Set by their raw string when field equality provably
reduces to a raw string comparison (whitespace-preserving lexical types,
no colon, which is most ids); value-space types (numeric, boolean, date),
whitespace-collapsing types, and QName-bearing values keep the exact
pairwise comparison, so which documents pass is unchanged. Position each
target through a per-parent, per-name index built once.
A 16000-item xs:unique document drops from about 4.0 s to about 0.12 s
(~34x), and the cost is now linear. The full local suite and the W3C
XSTS datatype/identity conformance corpus are unchanged.
firstInDocumentOrder() computed a document-order key even for a zero- or
one-node set. The key walks a node up to the root, taking its index among each
parent's children, which is a linear scan of a wide parent. string(@x) and
value-of take the string-value of a (usually single) node-set through this
path, so an XSLT transform over a wide flat fan-out rescanned the wide parent
once per extraction, turning the transform quadratic.
For a set of zero or one node the document order is irrelevant: the node is
trivially first. Return it directly without computing the key, the same
count <= 1 early-out sortedByDocumentOrder() already has.
A transform reading a couple of attributes and child values per element over a
20k-element document drops from ~2.3s to ~0.65s (~3.5x). The output is
unchanged: the full suite, the Apache Xalan XSLT gold-output corpus, the W3C
XPath/XSLT/EXSLT tests, and the WASM build stay green, and FirstInDocumentOrderTests
pins the single-node fast path and the multi-node document-first selection.
The bulk-copy escape loop still iterated Characters and compared each to '&',
'<', '>' (a grapheme comparison per character), the dominant remaining serialize
cost for content that needs escaping. Every escaping trigger is an ASCII marker,
or in ASCII-only mode a non-ASCII lead byte, so find the next one with
value.utf8.firstIndex(where:) and decode no grapheme for the run between markers.
The marker byte always lands on a grapheme boundary: an ASCII byte is its own
scalar and forces a cluster break (so '&', '<', '>', the quote, and whitespace
are never mid-cluster), and after a grapheme-aligned runStart the first byte at
or above 0x80 is a lead byte. So value[marker] and value[runStart ..< marker]
stay valid, and a multi-scalar cluster escapes as a unit (Swift rounds a
mid-grapheme index to its boundary, so the base letter is neither dropped nor
duplicated). The output is byte-identical.
Measured serialize ~23% faster on the 20k-item generated corpus via a same-state
A/B (~2.8x to ~2.2x slower than libxml2). Verified by EscapingBulkCopyTests
(the oracle differential extended with combining-mark, ZWJ, and flag clusters),
the Apache Xalan gold-output and C14N suites, the full suite, the WASM build,
and an adversarial grapheme-alignment review.
Escaping.text and Escaping.attribute rebuilt a value that needed escaping one
character at a time: result += String(character) for every unescaped character,
plus the escaped form for each marker. Re-escaping a text node that holds a
single '&' rebuilt the whole value grapheme by grapheme, and the per-character
String construction and append dominated the serialize path for entity-bearing
content.
Copy each run between escapable characters in a single append and rewrite only
the markers. The no-escaping-needed fast path (return the value unchanged) is
untouched. The output is byte-identical: matching is per-Character so multibyte
graphemes copy whole, and the escaped forms are the same.
Measured serialize ~7% faster on the 20k-item generated corpus via a same-state
A/B (~3.0x to ~2.8x slower than libxml2). Verified by EscapingBulkCopyTests (a
per-character-oracle differential with multibyte cases), the Apache Xalan
gold-output and C14N canonicalization suites, the full suite, the WASM build,
and an adversarial review that included a 400k-input fuzz.
parseTree ran TreeNode(parse(...)), which allocates two full trees: the parser
builds an immutable value Node tree, then TreeNode.init(_:) walks it to build
the mutable reference tree. The parsed tree is the deliverable, so building the
value tree first is pure overhead.
Parser.buildTree is the direct counterpart to build: the same iterative event
loop with the same element stack and the same well-formedness guards (an
unmatched end tag throws unexpectedEndOfInput; a document with no element throws
emptyDocument), but it accumulates TreeNode children per frame and adopts them
when an element closes, so the tree is allocated once. The produced tree is
identical: same kinds, names, attributes, values, child order, and parent links.
Measured parse ~8% faster on the 20k-item generated corpus via a same-state A/B
(6.1x to ~5.5x slower than libxml2). Verified by ParseTreeDirectBuildTests (a
structural differential against the convert-from-Node path, plus throw-parity
and a serialized round-trip), the full suite and the W3C XML conformance,
RELAX NG, and Xalan corpora (which all build trees through parseTree), the WASM
build, and an adversarial equivalence review.
EntityExpander.expand located the next '&', '<', and ';' with String.firstIndex
over graphemes, so every literal byte of a run was decoded into a grapheme just
to look for a one-byte marker. All three markers are ASCII, so a UTF-8 index is
a valid string index: scan over raw.utf8 instead, and the found index still
subscripts the string directly (the marker byte is never a continuation byte,
so its index lands on a scalar and grapheme boundary).
The decoded text is unchanged, including runs where the marker follows
multibyte UTF-8: a precomposed letter, a combining sequence, or an astral
character all leave the marker on a grapheme boundary.
Measured parse ~5.5% faster on the 20k-item generated corpus via a same-state
A/B (6.5x to ~6.1x slower than libxml2). Verified by EntityDecodeBulkTests
(extended with four multibyte-boundary cases), the W3C XML conformance corpora,
the Xalan XSLT gold-output corpus, the full suite, and the WASM build.
Building the mutable TreeNode tree from a parsed value Node ran append for every
child. append -> insert detaches the child from any previous parent, rejects a
self/ancestor cycle, and inserts into a children array that grows element by
element. The children the conversion passes come from element.children.map and
are freshly built and parentless, so all three guards are no-ops and the
incremental growth rebuilds an array that was already prepared.
TreeNode.init(adopting:) adopts the prepared array directly and wires each
child's parent. The tree is identical: the same children in the same order with
the same parent links. The adopting initializer is used only by the value-Node
conversion; every public factory keeps the detaching, cycle-checking append path
for externally supplied children that may already live in a tree.
Measured parse ~12% faster on the 20k-item generated corpus via a same-state
A/B (7.7x to ~6.5x slower than libxml2). Verified by TreeNodeAdoptingInitTests
(parent-wiring and post-adoption mutability), the W3C XML conformance corpora,
the Xalan XSLT gold-output corpus, the full suite, the WASM build, and an
adversarial equivalence review.
QualifiedName(_:) ran firstIndex(of: ":") over the whole name, grapheme by
grapheme, on every element and attribute. Most names have no prefix, so the
common case scanned the entire name to find no colon at all.
The byte scanner already visits every byte of an all-ASCII name, so have
takeASCIIName record the offset of the first colon (one comparison per byte),
and split at that known offset in QualifiedName(ascii:colonOffset:). The
unprefixed name (no colon) is then built with no scan; a prefixed name splits
at the recorded offset. The names are identical to the firstIndex path for
every colon placement: none, leading (:local), trailing (prefix:), middle,
doubled (::), and multiple (first colon wins). The hint is confined to the
all-ASCII fast path; non-ASCII names and every other construction site keep
QualifiedName(_:).
Measured parse ~12% faster on the 20k-item generated corpus via a same-state
A/B (8.5x to ~7.7x slower than libxml2). Verified by QualifiedNameColonHintTests
(colon-placement differential), the W3C XML conformance corpora, the Xalan XSLT
gold-output corpus, the full suite, the WASM build, and an adversarial
equivalence review.
EntityExpander.expand rebuilt every reference-bearing text run one character at
a time: result.append(character) per character, with a per-character charge
against the amplification budget. A text node holding a single & was copied
grapheme by grapheme, which is the bulk of the per-text-node cost in a document
whose content uses entities.
Scan to the next '&' or '<' and append the literal run in one operation,
charging the budget by the run length. The decoded text, the CDATA-verbatim
shielding, and the billion-laughs budget enforcement are unchanged: the same
overrun throws at the same input, since charging a run of length L against
budget B fails exactly when L > B, the same condition the per-character guard
enforced.
Also return early from recordReferenceContentFindings when there are no DTD
content models (the common no-DTD case): it otherwise scanned each text run for
'&' and rendered the enclosing element's qualified name before the model lookup
that would fail anyway.
Measured parse ~5% faster on the 20k-item generated corpus via a same-state A/B
(9.06x to ~8.5x slower than libxml2). Verified by the W3C XML conformance
corpora, the new EntityDecodeBulkTests six-case decode differential, the full
suite, the Xalan XSLT gold-output corpus, and the WASM build, with an
adversarial equivalence review of the rewrite.
The descendant and attribute walks tested every candidate node through a
closure that called matchesTree / matchesAttribute with the step's NodeTest and
the evaluation namespaces dictionary:
descendants(of: tree) { matchesTree($0, step.test, on: step.axis, context.namespaces) }
That passes the namespaces dictionary (and the NodeTest's associated string) by
value on every call, so a //name-style query over a wide document carried a
dictionary retain/release on each of tens of thousands of nodes, on the
traversal that dominates evaluation.
compiledTreeTest, compiledAttributeTest, and compiledNodeTest resolve the parts
that do not change as the walk moves (the axis principal kind, whether eval-time
bindings are in effect, the prefix-wildcard shape of the name) exactly once and
return a tight per-node closure capturing only the local name. The selected
nodes are identical to the unfused matchesTree / matchesAttribute / matches.
Measured xpath ~35% faster on the 20k-item generated corpus via a same-state
A/B, from ~2.0x to ~1.3x slower than libxml2. The full suite, the Apache Xalan
XSLT gold-output corpus, and the WASM build stay green. XPathCompiledNodeTestTests
is an exhaustive differential asserting each compiled closure equals the
function it specializes across every node, test, axis, and binding.
An EvaluationContext carried the variable bindings, the function library, the
namespace bindings, and the budget by value. Re-focusing the context onto each
node of a node-set (focused(on:position:size:), called once per node when a
predicate is applied) copied all three dictionaries every time, even though
none of them changes as the focus moves.
Bundle the focus-independent state in an Environment reference type. The
context now holds the node, position, size, and an Environment reference;
re-focusing copies the three integers and shares the environment by reference
instead of copying and reference-counting three dictionaries. The values and
behavior are identical.
Measured xpath ~16% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000). The full XPath suite, the Apache Xalan XSLT
gold-output corpus, and the WASM build stay green.
The attribute axis built an XPath node for every attribute of the element
and then filtered by the step's node test. A predicate like [@name='value']
runs that per matched element, so for a wide document it wrapped many
attributes only to discard all but one, and each wrap retains the owner
element and copies the attribute's qualified name and value.
Apply the same fusion the descendant axes use: test the raw attribute during
the walk and wrap only the ones that pass. matches delegates, for an
attribute node, to a new matchesAttribute (the attribute axis always carries
the attribute principal kind, so the reduction is exact), and the attribute
walk applies it as a keep filter. The node-set, declaration order, and
uniqueness are identical to build-then-filter.
Measured xpath ~20% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000); xpath is now ~2x libxml2, down from ~13x at
the start of this work. XPathFusedAttributeTests covers every node-test kind
through the fused path, and an adversarial equivalence review found no query
whose result changes. Full suite, Apache Xalan XSLT gold output, and the WASM
build stay green.
The descendant and descendant-or-self axes built the full subtree node list
and then filtered it by the step's node test. For a //name query over a wide
document most of the wrapped nodes were discarded immediately, and each wrap
retains a tree node whose weak parent forces the slow reference-counting path.
Run the node test against the raw tree node during the walk, so a rejected
node is never wrapped. matches(_:_:on:namespaces:) now delegates, for a tree
node, to a new tree-node matchesTree (one source of truth, exercised by the
full XPath suite), and the descendant walk applies it as a keep filter. The
node-set, document order, and uniqueness are identical to build-then-filter;
only non-matching nodes are no longer materialized.
Measured xpath ~11% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000). XPathFusedDescendantTests covers every
node-test kind through the fused path, and an adversarial equivalence review
found no query whose result changes. Full suite, Apache Xalan XSLT gold
output, and the WASM build stay green.
`descendants(of:)` recursed on wrapped XPath nodes and, at every node, built
a fresh array for that node's subtree and concatenated it
(`result.append(contentsOf: descendants(of: child))`). That is an array
allocation and copy per node, roughly O(n x depth) work over the subtree, and
the parse profile showed it dominating XPath evaluation for a `//`-style query
over a wide document.
Recurse on the raw tree node instead, appending into one shared inout
accumulator. The result is the identical node-set in the identical document
order; only the way the array is built changes, from O(n x depth) to O(n).
Measured xpath ~46% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000). Full suite, the Apache Xalan XSLT gold-output
corpus, and the WASM build stay green.
A path that is a single forward-axis step from one context node returns its
nodes already in document order with no duplicates, so the document-order
sort is redundant. AxisNavigation produces the structural forward axes
(child, descendant, descendant-or-self, self, following, following-sibling)
in document order, deriving sibling order from the same children array the
order key compares, so the two can never disagree. This is the common shape
after //x compiles and fuses to a single descendant::x step.
The attribute axis (whose order interleaves with namespace declarations), the
namespace axis (produced in dictionary order), and every reverse axis
(nearest-first) are excluded and still sorted. Measured xpath ~7% faster on
the 20k-item generated corpus via a same-state A/B, identical results.
XPathSingleStepOrderTests pins document order for each included axis, that the
reverse axes still normalize, and that multi-step paths still sort. An
adversarial order-correctness review found no query whose order or uniqueness
the change breaks.
A path expression sorted its result into document order through orderUnique,
which first ran a Set-based de-duplication. But evaluateSteps already returns
a duplicate-free node-set: each step de-duplicates across its context nodes,
or uses an axis whose per-context results cannot overlap. So that Set copied
every node for nothing, and each copy retains the wrapped tree node, whose
weak parent reference forces a side table and the slow reference-counting
path.
Path and trailing-path evaluation now sort the steps' result directly.
Unions and filter primaries, which genuinely can carry duplicates, keep the
de-duplicating pass. Measured xpath ~9% faster on the 20k-item generated
corpus via a same-state A/B, identical results. The existing node-set
de-duplication and union tests cover that a path still yields each node once.
`//X` compiles to `descendant-or-self::node()/child::X`, which materializes
every node of the subtree as an intermediate context before selecting the X
children of each. Fuse that pair into a single `descendant::X` step at compile
time: the two select the same node-set, so evaluation skips the intermediate.
The fusion is transparent except for a positional predicate: `//x[1]` is the
first x child of EACH parent, not the first x in document order, so it must
not become `descendant::x[1]`. Apply the rewrite only when every predicate is
provably non-positional: its value is a boolean, node-set, or string (never a
number, which XPath reads as position() = value) and it nowhere references
position() or last(). Anything not provably both is left unfused, so a result
can never change.
Measured xpath ~10x to ~7x slower than libxml2 on the 20k-item generated
corpus via a same-state A/B, identical results. XPathDescendantFusionTests
pins the positional forms keeping their per-parent meaning, the non-positional
forms selecting across the subtree, and composition with trailing and nested
steps. An adversarial differential review (fused versus a spec-literal unfused
reference over nine attack suites) found no query whose result changes. The
boolean-function allowlist carries a documented invariant: every entry must be
position- and size-independent, since the position scan is syntactic.
Each location step accumulated its context nodes' axis results into a
Set<Node> to drop nodes reachable from more than one context, hashing and
copying every node. The wrapped TreeNode is reference-counted, so each copy
is ARC traffic, and the profile showed the Set insert, hashing, and Node
copies dominating evaluation.
De-duplication is needed only when two distinct context nodes can reach the
same node. Skip it on a single context and on the axes whose results are
disjoint per context (child, attribute, namespace, self): those produce the
identical node sequence the Set would, since every insertion succeeds, with
no hashing. Keep the Set on the axes that genuinely overlap (descendant,
ancestor, parent, the siblings, following/preceding). Predicates stay applied
per context and the path result still passes through the final
order-and-unique pass, so results are identical.
Measured xpath ~13x to ~10x slower than libxml2 on the 20k-item generated
corpus via a same-state A/B, ~28% faster, same result. XPathNodeSetDedupTests
pins that a node shared by many contexts through ancestor/parent/descendant
is still counted once, the disjoint axes return every distinct result, and an
overlapping union still de-duplicates. The full XPath suite and the Apache
Xalan XSLT gold-output corpus stay green.
The serializer's escaping functions rebuilt every text, attribute, comment,
and processing-instruction value character by character, even when the value
contained nothing to escape, which is the common case. Each now scans the
value's UTF-8 bytes for the bytes it would escape (the markup characters, the
active attribute quote, the whitespace characters, a comment hyphen, a PI
question mark, and any non-ASCII byte in ASCII-only mode) and returns the
value unchanged when none is present, taking the original character path only
when something must be escaped.
Measured serialize roughly halved, ~5.5x to ~2.7x slower than libxml2 on the
20k-item generated corpus, with byte-identical output. EscapingFastPathTests
pins both the unchanged fast path and the escaped slow path for each function,
and the Apache Xalan XSLT gold-output corpus (a serialize-heavy differential)
stays green. No output changed.
The attribute-value scanner ran the entity decoder on every value, and the
decoder re-scanned the whole value for '&' before doing nothing when none
was present. The byte-run scanner already visits every value byte, so it
now notes any '&' and an ampersand-free value (the common case) returns its
normalized run directly. Measured against a same-machine-state baseline,
every run came in below every baseline run, ~4% faster parse on the
20k-item generated corpus.
Move the bulk byte-run scanners (contentRunBytes, attributeRunBytes) into
ReaderByteRuns.swift and centralize their cursor mutation through a single
advanceByteRun method, so Reader's position fields stay write-encapsulated
in Reader.swift while both files stay under the length cap. The byte and
Character paths are held to identical decoded values by the differential
oracle, extended with ampersand-free, entity, and character-reference
attribute cases. No spec behavior changed.
The element-content markup dispatch ran up to five literal string
comparisons (</, <!--, <![CDATA[, <?, <!DOCTYPE) before the start-tag
branch won, a per-element cost the parse profile indicted as
Reader.matchesBytes. Branch instead on the single byte after '<' on the
byte fast path: '/' end tag, '?' processing instruction, '!' a
declaration, otherwise a start tag.
The Character path keeps the equivalent matches()-based dispatch for a
streaming source or spliced entity text, and a '<!' that matches none of
comment/CDATA/DOCTYPE still falls through to the start-tag scanner with
the same expectedName error. Measured parse ~9.5x -> ~9.0x slower than
libxml2 on the 20k-item corpus; the byte and Character paths are held
identical by ParserPathDifferentialTests over its adversarial corpus,
and the Xalan, W3C, and RELAX NG suites stay green. No spec behavior
changed.
Keep the reader on its byte fast path through dispatch and name/attribute
scanning so plain-ASCII documents avoid the per-Character String machinery
(String.Iterator.next, string comparison, small-string materialization,
bridge-object ARC) that a profile indicted as the parse bottleneck.
The root cause was that a single buffered lookahead Character defeated every
existing byte path, each of which requires an empty buffer, and the content,
markup, and attribute dispatch each peeked a Character before scanning. Add a
non-buffering peekByte for dispatch, a byte-mode skipSpace, an all-ASCII
takeASCIIName, and an attributeRunBytes bulk scanner (with the 3.3.3
whitespace-to-space transform, deferring any carriage return to the Character
path so CRLF is not double-counted). Detect a text run's closing '<' via
peekByte instead of buffering it, which had cascaded the next element's whole
dispatch and name scan onto the slow path. Skip reference decoding for
ampersand-free text. The Character path remains the exact fallback for a
streaming source, spliced entity text, and any non-ASCII byte; the two paths
share scanContentMarkup so they cannot diverge.
Measured parse 20.1x -> 9.5x slower than libxml2 on the 20k-item generated
corpus, roughly a 53 percent wall-clock reduction. A differential oracle pins
the byte and Character paths to identical event streams and error marks over
an adversarial corpus, alongside boundary tests. Verified against the full
local suite, the Apache Xalan XSLT gold-output corpus, the W3C XML conformance
corpora, the RELAX NG spec suite, and the WASM build. No spec behavior changed.
* docs(design): derive the exact Particle Valid (Restriction) procedure
The false-negative tail (particles*/elemZ) routes through one approximate
oracle, ParticleRestriction.violation. Rather than patch cases, derive the
exact XSD 1.0 3.9.6 decision procedure once, clean-room from the spec and the
Xerces reference algorithm, with the soundness (no-false-positive) argument for
each clause: normalization, substitution-group expansion, effective total
range, the compositor dispatch table, and the rcases.
Identifies the closeable gaps (subst-group expansion for elemZ026/V020,
MapAndSum effective-total-range, NameAndTypeOK union-member derivation) and
names the genuine spec limitations (Ha161/Z001 contradiction, mgO013 contested,
union-member by-name derivation) as bounded, not bugs. Sets the implementation
order, each clause gated by the proof plus the XSTS valid-rejected=0 gate and an
adversarial over-rejection critic. No behaviour change.
* docs(design): record the empirical finding -- particle-restriction tail is contested
Probing the actual cases overturned the first hypothesis. elemZ026 and
particlesV020 are the same pattern (a derived ref to a substitution-group
MEMBER restricting a base ref to the HEAD), and PureXML already accepts them
because its content model expands a head ref to admit members (for instance
validation), which the restriction oracle inherits -- the Xerces-lenient
reading. They are CONTESTED (Xerces/PureXML accept, XSTS rejects under strict
NameAndTypeOK), so the keystone (adding expansion) would entrench the
acceptance, not close them; closing them risks a false positive. Every
remaining particle case is contested or FP-delicate, so the exact-oracle
keystone yields no clean conformance win. The remaining lever toward the full
bar is the proven resource bound (pushdown matcher), not this oracle. Proof
first earned its keep: it prevented shipping against contested targets.
Generalise the wildcard {namespace constraint} model to the four XSD 1.0 (with
Errata E1-10) expressible forms -- any; a set (with absent); a pair of not and
a namespace name; a pair of not and absent -- replacing the any/##other/set
model that could not represent the union/intersection results. Both not forms
exclude absent, so a union admitting "every namespace except a name, plus
absent" (E1-10 rule 5.1.3) has no expressible form.
A new SchemaWildcardExpressibility compile rule rejects a complexContent
extension whose attribute-wildcard union with its base is not expressible
(src-ct.5), closing XSTS wildZ013 (test328873i). The expressible unions are
computed exactly in XSDComplexContent, so a derived type's wildcard admits the
correct attributes at instance time, closing wildZ013a and wildZ013d.
invalid-schemas-accepted 12 -> 11, invalid-instances-accepted 17 -> 15, with
valid-schemas-rejected and valid-instances-rejected held at 0 (full XSTS
archive). The model is additive for element wildcards (which never produce the
new forms); admits/narrows/UPA-overlap updated and verified unchanged for the
##other cases.
The intersection of two not forms negating different namespace names is not
expressible (src-ct.4), a deferred rule: rather than drop the wildcard and
over-reject valid cross-namespace attributes on an accepted schema, the
intersection stays the less-restrictive not(lhs) over-approximation (FP-safe,
matching prior behaviour). Union/intersection algebra taken clean-room from
Xerces-J XSWildcardDecl. Guarded by SchemaWildcardUnionExpressibilityTests.
XSD charClassSub ::= (posCharGroup | negCharGroup) '-' charClassExpr is the set
difference of the (possibly negated) group and the subtrahend, so for
[^cde-[ag]] negation applies to the base first and the subtrahend is removed
from that result: the class admits everything except {a, c, d, e, g}.
CharClass.matches applied the subtraction to the positive membership and
negated afterwards, so [^cde-[ag]] excluded only {c, d, e} and wrongly matched
a/g, accepting instance values a correct engine rejects. Negation now applies
to the base before subtraction (new result !B && !S is a strict subset of the
old !B || S, matching the Xerces-J reference exactly), so only negated classes
that also carry a subtraction change behaviour.
Closes XSTS RegexTest_430 and RegexTest_422: invalid-instances-accepted 19 ->
17, valid-instances-rejected and valid-schemas-rejected held at 0 (full XSTS
archive). Guarded by RegexCategoryTests.test_negatedSubtraction.
XSD 1.0 charProp is IsCategory | IsBlock, an exhaustive set: every Unicode
general category plus a fixed Unicode-3.1 block list. The block table was
missing the nine supplementary-plane blocks (Old Italic, Gothic, Deseret, the
Byzantine/Musical/Mathematical symbol blocks, CJK Unified Ideographs Extension
B, CJK Compatibility Ideographs Supplement, Tags) and the three surrogate
blocks, so parseCategory could not tell a real-but-untabulated block from a
bogus name and stayed lenient (RegexError.unsupported), accepting the pattern
\p{IsaA0-a9} and the schema declaring it (XSTS reK88).
Complete the table to the full XSD set: supplementary blocks with their ranges,
and surrogate blocks as valid names whose ranges hold no scalar (so they match
nothing). The recognised set is now authoritative, so parseCategory throws the
rejecting RegexError.invalidProperty for any unrecognised name. The now-dead
RegexError.unsupported case and the isMalformedProperty heuristic are removed.
Closes XSTS reK88: invalid-schemas-accepted 13 -> 12, valid-schemas-rejected
held at 0, instance buckets unchanged (full XSTS archive). Side effect: a
pattern naming a supplementary-plane block (e.g. \p{IsMusicalSymbols}) now
matches its characters instead of failing to compile, removing a latent
instance false positive. Block names/ranges taken from the XSD-supported list
in Xerces-J Token.java (clean-room: factual data only). The list is
spec-frozen at Unicode 3.1, so later blocks are rejected by design.
XSD 1.0 Datatypes Appendix F defines SingleCharNoEsc as [^\[\]] and admits a
nested class only as a -[...] subtraction, so a `[` inside a character class
that is not introduced by `-` is a syntax error. RegexParser treated such a
`[` as a literal scalar, so `[a[:xyz:]` compiled and the schema declaring it
was wrongly accepted (XSTS RegexTest_993, RegexTest_1477).
nextClassMember now throws a dedicated RegexError.unescapedClassBracket, kept
distinct from the generic, deliberately-lenient badClass so the change cannot
regress a valid-but-unsupported pattern. XSDSimpleParser.rejectsPattern treats
it as a genuine syntax error, so the declaring schema is invalid.
Closes RegexTest_993 and RegexTest_1477: invalid-schemas-accepted 15 -> 13,
valid-schemas-rejected held at 0, instance buckets unchanged (full XSTS
archive). Escaped [a\[c] and subtraction [a-z-[aeiou]] stay valid; the
unterminated case [a[:xyz: now faults on the inner [.
The streaming-vs-tree differential previously exercised group references only
through the shallow g0/g1 mutual cycle, too weak to gate a future pushdown
content-model matcher whose call/return correctness hinges on deep group
nesting. Add a deep LINEAR group-reference chain (c0 -> ... -> c{chainDepth},
single ref per level so its inlined size is O(depth), not 2^depth) reachable
from any leaf, so the two engines are compared over deep call/return without the
exponential build a doubling chain would trigger. Run divergence-free over the
standing 2000-seed gate and a 60000-seed campaign.
Reconcile docs/release-hurdles.md Part 2 with the deleted positionCap and record
the one remaining structural residual, distinct from occurrence unrolling: the
per-instance NFA build still inlines group references, so a 2^K-inlining schema
makes automaton construction exponential on first instance validation. Document
the pushdown/RTN matcher as the general fix, gated by this hardened differential
(FP-critical: naive sub-model sharing without a stack is unsound).
The Unique Particle Attribution check no longer inlines group references or caps.
A new CompositionalDeterminism engine computes first/followlast over particle
identities and summarizes each xs:group once, so a multiply-referenced nested
group whose inlined Glushkov automaton would be 2^K positions is checked in
O(nodes * particles^2) time. ContentModelDeterminism now delegates to it and the
inlining position automaton plus positionCap are deleted, closing the last named
interactive-safety resource-bound blocker (production-readiness stopper #4) and
the silent under-rejection the cap caused (stopper #3).
The compositional engine was proven verdict-equivalent to the prior inlining
automaton by a differential over the entire XSTS corpus (zero divergences) before
replacing it. The followlast detail was essential: a first attempt that emitted a
repeating end-item's loop-back and the suffix-first as separate decision sets
under-detected 13 XSTS content models; the differential caught every one.
Verification: XSTS gate unchanged (invalid-schemas-accepted 15, valid-schemas-
rejected 0, instance buckets held); SchemaDeterminismGroupBlowupTests compiles a
2^20-position model without hanging and rejects an ambiguous one the old cap would
have skipped (WASI uses a smaller depth, native time limits gated off WASI); full
suite, style, namespacing, and WASM all pass. The per-particle model and QName-only
overlap are preserved, so the false-positive direction stays at zero.
Turns the 'blocked design frontier' for ContentModelDeterminism.positionCap into
a ready-to-implement, FP-safe plan derived from the current engine. Documents the
root cause (group-ref inlining gives 2^K positions), why naive position
dedup/sharing is unsound (cross-context followpos merge = false positive), the
sound compositional algorithm (per-group memoized nullable/firstParticles/
lastParticles/followlast over particle identities, cross-boundary conflicts
computed per reference-site by the parent, the star condition for repetition,
bounded O(nodes x particles^2)), and the differential-gated rollout (build
alongside the inlining oracle, prove verdict-equality on XSTS + a group-ref fuzz,
only then switch and delete the cap) with acceptance criteria.
cvc-elt.3.1: when an element's declaration is not nillable, the xsi:nil attribute
is forbidden entirely, whatever its value. PureXML only rejected xsi:nil="true" on
a non-nillable element, so xsi:nil="false" slipped through. ComplexValidator.nilErrors
now flags the presence of any xsi:nil on a non-nillable element via the new
nilAttributeValue helper, while a nillable element still accepts xsi:nil="false"
(explicitly not nilled, valid empty-or-content per its type).
XSTS invalid-instances-accepted 28 -> 26 (both xsi:nil-on-non-nillable cases), no
other bucket moved (valid-schemas-rejected and valid-instances-rejected held at 0
across all 14383 groups). 1452 unit tests pass.
XSD 1.0 3.3.2: an element with neither a `type` attribute nor an inline type, but
a `substitutionGroup`, takes the {type definition} of its head. PureXML defaulted
such an element to `anyType`, so an instance of it accepted any content.
`XSDParser.elementType` now resolves the head declaration through
`context.globalElements` (transitively, with a cycle guard) and uses its type.
XSTS invalid-instances-accepted 29 -> 28 (typeDef00204m: <root>Yes</root> is no
longer accepted where root substitutes a boolean head), no other bucket moved
(valid-schemas-rejected and valid-instances-rejected held at 0 across all 14383
groups). 1451 unit tests pass.
cvc-elt.4.3.2.1 requires an xsi:type to name a type DERIVED from the element's
declared type. anyType is the ur-supertype of every type, so it is never derived
from anySimpleType (its own subtype); an anyType override on an anySimpleType
element is therefore invalid. The general not-derived check stays silent on
ur-type declared types to avoid false positives, so this adds the narrow,
provably-invalid anyType-for-anySimpleType pairing only, leaving anyType valid on
an anyType-declared element (which resolves to a complex ur-type and is excluded).
XSTS invalid-instances-accepted 30 -> 29 (stZ056), no other bucket moved
(valid-schemas-rejected and valid-instances-rejected held at 0 across all 14383
groups). 1450 unit tests pass.
XSD 1.0 has no year zero (the year before 0001 is -0001), so 0000-01-01T00:00:00
and the other 0000 forms are not valid values. DateTimeParser.parseYearOnly now
rejects a zero magnitude for both signs, so negative years like -0001 still parse.
Closes the XSTS dateTime011 false negative: invalid-instances-accepted 31 -> 30,
no other bucket moved (valid-schemas-rejected and valid-instances-rejected held at
0). 1449 unit tests pass; the full XSTS suite settles at 0/43/0/30.
The Antigravity onboarding definition-of-done checklist still read
'valid-schemas-rejected still 1'; the enforced baseline is 0
(Tests/XSTSSuiteTests.swift), matching the rest of this file and GEMINI.md.
Caught by the PR review pass.
- README, roadmap: XSTS status quad and gates table now read the enforced
0/43/0/31 (Tests/XSTSSuiteTests.swift); test count 986/169 to 1448/258;
encoding list and the remaining-work section reflect shipped reality;
check-all expansion lists the validation and changelog gates
- GEMINI.md, onboarding-antigravity.md: valid-schemas-rejected 1 to 0 and the
pinned baselines updated to current
- SECURITY.md: complete the reporting address
- Parsing.InputEncoding: doc comment lists the full shipped encoding set
- Schema.XSDParserUnion: guard the localName interpolation with ?? "" to match
the same line and silence an optional-interpolation warning (internal dedup
key, behavior preserving)
- CHANGELOG: Unreleased entries for the above
