The two monolithic XSD content/identity rules only satisfied the OpenAPIKit
validation idiom at the wrong granularity: no per-constraint removability,
no isolable failing/near-miss tests. Split them into 32 named rules, each
finding tagged at its origin and cached per XSDContext so the walk still
runs once per document (the #101 decision against duplicating XSD's
recursive type resolution still holds). Wired the streaming path's ID/IDREF
and identity checks through the same framework instead of calling the
validators imperatively, and added the unwrapAndLookup combinator the rule
already named as required.
Also includes the mandated repo-wide swiftformat/swiftlint pass.
The rules-swift concurrency rule's acceptance check requires every `@unchecked
Sendable` and `nonisolated(unsafe)` to carry an adjacent comment explaining why
the race is acceptable; an unjustified one is a violation. An audit found six
escape hatches whose comment explained the type but not the race:
- The four XPath `FunctionTable` statics (`CoreFunctions`, `StringFunctions`,
`NodeFunctions`, `Evaluator.library`): each is an immutable, build-once lookup
table never mutated after initialization, so concurrent reads race on nothing;
`FunctionTable` is un-`Sendable` only because it stores closures.
- `Validation.Validator`: immutable after `init` (every stored validation list is
a `let`); the erased `AnyValidation` closures are simply not provably `Sendable`.
- The test `ResultBox`: written once by the worker thread before its semaphore
signals and read only after the wait, so the write happens-before the read.
Comments only; no behaviour change. `LoaderBox` and `ElementStorage` were already
justified. swiftformat/swiftlint --strict and the full suite stay green.
* perf(model): rebuild DTD-default application iteratively, not recursively
applyingDefaults rebuilt the tree with a per-node recursive walk, so applying
DTD attribute defaults to a deeply-nested document overflowed the stack (around
2000 deep on a Swift Task stack). It now rebuilds bottom-up through a new shared
iterative spine, Node.rebuildingBottomUp, which transforms each node once its
children are rebuilt via an explicit work stack.
The same primitive will carry the other tree-to-tree transforms (canonical
prefix rewrite, XSLT namespace fixup) in follow-ups. Defaults are applied
identically: 1788 tests pass. Part of #350.
* perf(canonical): serialize C14N iteratively, not recursively
Canonicalizer.emit recursed on element children, so canonicalizing a
deeply-nested document overflowed the stack (crashed around 5000 deep).
The element serializer now drives a deferred-close work stack (the same
shape as the main serializer): each step carries the in-scope and
already-rendered namespace context, children push reversed to emit in
document order, and a close step emits the end tag. The shallow document
case and all namespace-rendering logic are unchanged.
The canonical bytes are byte-identical: the C14N 1.0, 1.1, and exclusive
conformance corpora pass, full suite 1789. A 50000-deep document now
canonicalizes without crashing. The node-subset helpers moved to their
own file to keep within the file-length limit. Part of #350.
Building a TreeNode from a value Node, projecting it back via `node`, and
releasing it each recursed one native frame per tree level, so a deeply
nested document overflowed the call stack (signal 10 around 30-60k deep,
and far shallower on a constrained stack such as a Swift Task). None of
these is fixed by the serializer, which was already iterative.
- `TreeNode(_:)` and `node`/`projectedChildren` now drive explicit
post-order work stacks (entity-reference splicing, doctype/namespace
dropping, and the root special-cases are unchanged).
- `deinit` flattens the subtree into a heap worklist before releasing, so
each node frees with empty children and the teardown stays flat. Only a
node the teardown solely owns (isKnownUniquelyReferenced) is flattened,
so a child still referenced elsewhere keeps its subtree.
- The validator walk and the XPath descendant axis were made iterative for
the same reason.
A 60000-deep tree now releases in well under 0.1s; 1778 tests pass and no
accepted output changes. Part of #341; the value Node's own recursive
release and XPath result ordering/dedup remain depth-bounded and tracked
there.
Schema compile findings now carry ValidationError coding paths that resolve to source spans on a ranged tree, enabling IDE underlines for id, structure, include, and facet violations.
Ship the OpenAPIKit-style validation layer (builtin rules, warnings channel,
Validator tiers, coverage gates) and wire XSD compile checks through it. Advance
#161 with chameleon include validation, multi-container reference resolution,
and include targetNamespace composition rules so merged schema sets compile
correctly when a schemaLoader is provided.
DocumentType.validityFindings carries ValidityFinding values (reason
plus subject) instead of strings, DTDSchema.declarationErrors holds
ready ValidationErrors, and every declaration-level finding renders at
its declaration's coding path (x, r/@a) instead of at the document
root, matching the XSD consistency-errors precedent. Texts unchanged.
The eduni sections run with exact classified baselines. Fixes: 3.3.3
CDATA attribute-value normalization (literal whitespace to space before
reference decoding; char-ref whitespace survives; tokenized collapse on
SPACE only); production-68 lenient undeclared entities in content and
attribute values (validity finding, reference kept literal, nothing
fetched); errata E15 (references/comments/PIs/whitespace in EMPTY,
char-ref whitespace as data in element content, double-escape shape
included); errata E2 duplicate enumeration tokens; errata E14
declarations completed inside PE replacements (VC: Proper
Declaration/PE Nesting); NCName ID values; namespace declarations
subject to the every-attribute-declared rule. Reader validity machinery
split to EventReaderValidity for caps.
Closes#127
The IBM section (1094 cases) runs clean to two documented classes:
279 pre-Fifth-Edition character-class cases (P85-89, count asserted
exactly) and 2 PE-in-internal-subset feature cases. Real gaps closed:
a document needs a root element (production 1); entity references in
literals must be Names or character references; WFC: No Recursion
checked at declaration over the general/parameter reference graph;
undeclared PE references split into WFC (standalone, or no external
declarations possible) versus deferred validity finding, also for
attribute-default references; refused external PEs are neither; and
groups split across PE replacements report VC: Proper Group/PE
Nesting. Entity machinery split to DoctypeScannerEntities for caps.
Closes#125
The scanner records declaration provenance (internal-subset entities,
element models, attribute lists); the validator reports the three
standalone='yes' dependencies on the external subset: a default that
would be supplied, a non-CDATA normalization that would change a value,
and whitespace in externally-declared element content. The parser hides
externally-declared entities from standalone documents, so referencing
one reports as undeclared (WFC: Entity Declared, 2.9). The Sun section
is fully clean: not-wf 56/56, valid 28/28, invalid 74/74, no baseline.
Closes#134Closes#124
VC Root Element Type (root matches the DOCTYPE name), VC Unique
Element Type Declaration, VC No Duplicate Types in mixed content, VC
One ID per Element Type, VC Notation Attributes and Notation Declared,
VC Attribute Default Legal plus ID Attribute Default, lexical Name
checks for the ID family, and CDATA as character data in element
content. Strict mode adds VC Attribute Value Type (every attribute
declared; namespace declarations exempt). Declaration findings are
computed once at schema build and reported at the document root. Sun
invalid coverage moves from 43/74 to 62/74; the twelve remaining are
all the standalone='yes' family.
Tokenized (non-CDATA) attribute values normalize before validity
checks, so enumerations, NOTATION lists, and the ID/IDREF table see
collapsed values. External parsed entities lose their leading text
declaration (validated: version optional, encoding required, never
standalone) instead of carrying it into replacement text. Character
references in PE literals expand at declaration time, making the
Appendix D '%zz;' example parse. A PI target inside the DTD subset
must be separated from its data. Sun baselines tighten to not-wf 55/56
and valid 26/28; the two remaining valid failures share one cause
(replacement text with markup is included as character data, filed
separately).
Extract PathKey.steps(forChildNames:) as the single path construction behind
the validator walk, the XSD content validator, and the identity validator,
replacing three identical implementations, so located errors can never
disagree on a path. Add two differentials locking the duplicated engines
together: equivalent DTD and XSD content models (sequence, choice, *, +, ?,
nested choice) must agree on every child sequence across the positional
matcher and the Thompson NFA; and the pull (EventReader) and push
(PushScanner) tokenizers must yield the same event stream for the same
document, even with the push side fed one character at a time (adjacent text
events coalesced, since chunk boundaries legitimately split text). Both
pairs agree on everything today; the differentials guard them.
Closes#115
Begin Tier-2 conformance checking, expressed through the OpenAPIKit validation
idiom rather than an ad-hoc loop. ConformanceCase is a Validatable named
expectation; Conformance.matchesExpected is a Validation<ConformanceCase, Void>
that emits one located ValidationError per diverging case, so a suite reports
every failure at once with the case name as its coding path. Seeded with a
Canonical XML 1.0 corpus (attribute ordering, empty-element expansion, comment
removal, CDATA-to-escaped-text, PIs, namespace rendering) and a test proving a
divergence is reported as a located failure. Growing the corpora against the
official suites is the ongoing Tier-2 work.
Part of #105 (Tier 2)
Complete the DTD streaming validator with document-scoped ID/IDREF checks,
held in bounded sets (the identifier values, not the tree): a duplicate ID
is reported at its repeat occurrence, and forward IDREFs resolve at the end
of the stream via finish(). The streaming-vs-tree agreement test now covers
a duplicate ID as well. DTD streaming validation (content, attributes,
ID/IDREF) is now complete; RELAX NG and XSD streaming remain.
Part of #107, #105
Add PureXML.Validation.StreamingDTDValidator and PureXML.validate(streaming:
dtd:), validating a document against a DTD while it is pulled event by event
(the libxml2 xmlTextReader model), holding only the open-element stack rather
than the whole tree. Each element's content model and attributes are checked
as it closes by applying the exact per-element DTD rules to a shallow element
synthesized from the streamed child names, so streaming and tree validation
agree (a test asserts matching reasons). Errors carry an xmlGetNodePath-style
coding path. RELAX NG/XSD streaming and ID/IDREF streaming remain on #107.
Part of #107, #105
Schematron tests, lets, and message expressions can now use key() backed
by xsl:key declarations parsed from the schema, document() loading an
external file through an injected documentLoader, and current() resolving
to the rule context node. A node-keyed function-table factory threads
through the evaluation. Completes the Schematron audit.
An element whose children break its content model now names each stray
child outside the model's alphabet individually, and reports a pure
order/count violation once with the allowed elements as a hint, instead
of a single opaque "do not match its content model". Adds an alphabet
extractor to the content-model matcher. This brings DTD to the same
located standard as the XSD, RELAX NG, and Schematron validators, so all
four schema languages give editor-grade diagnostics.
A <pattern abstract="true"> is a template whose rule queries carry $name
references; a <pattern is-a="…"> with <param> children instantiates it,
substituting each parameter into the template's context, test, <let>
value, and <value-of select=> before the XPath is compiled. The
substitution is a single left-to-right pass, so a substituted value is
never rescanned and a short name never captures a longer one; $name
references that are not parameters stay as written. The template itself
contributes no rules.
<let> may now appear at schema and pattern level, not just in a rule.
Schema lets are evaluated once at the document root; pattern lets extend
them (also at the root); rule lets extend those at each context node. A
narrower-scope binding overrides a wider one of the same name.
Part of #80
An assert/report may reference <diagnostic> messages by id through its
diagnostics= attribute; each referenced diagnostic is rendered (with the
same value-of/name/let substitution) and appended to the finding for
extra per-failure detail. <diagnostics> is collected schema-wide.
Schema/pattern-level let, abstract patterns, and key()/document() remain.
Part of #80
A <rule abstract="true" id="x"> is a template that does not fire on its
own; a concrete rule pulls in its let bindings and assertions with
<extends rule="x"/>, prepended so the concrete rule extends rather than
replaces them.
Part of #80
<rule> may now declare <let name= value=> bindings, evaluated at each
context node in order (so a later binding can reference an earlier one
through $name). The bindings are available to the rule's assert/report
tests and to <value-of select="$name"> in messages, using the XPath
engine's existing variable support.
Schema- and pattern-level let, abstract rules/extends, diagnostics, and
key()/document() remain.
Part of #80
Two ISO Schematron gaps, the most validator-relevant first:
- Dynamic messages: an assert/report message is now a template, so
<value-of select=> renders the string value of its XPath at the context
node and <name> renders the node's name. A finding can report actual
values ("Age 120 must be under 100") instead of static text.
- Phases: <phase id><active pattern=> scopes which patterns run.
validate(_:phase:) honors a requested phase or the schema's
defaultPhase; nil or #ALL runs every pattern.
let variables, abstract rules/extends, diagnostics, and key()/document()
remain (the last depends on XSLT-style function injection).
Part of #80
Deepen DTD processing in the doctype scanner:
- Conditional sections are processed, not skipped: INCLUDE sections have
their declarations scanned, IGNORE sections are discarded, the keyword
may be a parameter-entity reference, and nested sections are handled.
- <!NOTATION> declarations are parsed and stored on DocumentType.
- Unparsed entities (<!ENTITY ... NDATA n>) are parsed and stored with
their notation, distinct from general entities.
- External parameter entities are loaded through the injected resolver so
their declarations expand; the default refusing resolver loads nothing,
keeping XXE closed.
- NOTATION attributes are a distinct type, validated against the declared
notations (value must be listed and the notation declared).
Tracking unparsed entities also lets ENTITY/ENTITIES attribute values be
validated against the declared NDATA entities, closing the gap left when
#73 could only check name syntax.
Closes#72
Complete the DTD attribute handling found in the audit:
- Default and #FIXED attribute values are applied to the parsed tree.
parseApplyingInternalDTDDefaults returns a tree where every element
carries the attributes its <!ATTLIST> defaults, so an editor sees the
same effective attributes a validating processor would.
- NMTOKEN, NMTOKENS, ENTITY, ENTITIES are validated: NMTOKEN(S) must be
XML name tokens, ENTITY/ENTITIES must be XML names. Previously all fell
through to unconstrained CDATA.
- DTD default values are normalized per XML 1.0: character and predefined
entity references are expanded, literal whitespace becomes a space, and
non-CDATA values are collapsed and trimmed.
ENTITY/ENTITIES check name syntax only; that the named entity is a
declared unparsed entity is not verified, since NDATA entity tracking is
not yet modeled.
Closes#73
Slice 5 of #92. The last validator moves onto the framework and the legacy
Issue type is removed, so all validation now flows through one error model.
- ValidationError gains a severity (error by default, warning for advisory
findings), chosen so Schematron's assert/report distinction survives in the
unified type.
- Schematron is a Validation<Node, Void> over the document root; a failed assert
is an error finding, a matched report a warning. Each finding is located at the
context node its rule fired on (built from the element chain, with a sibling
index only when a name repeats), not the document root.
- Schematron.validate returns [ValidationError] and runs through a Validator.
- Validation.Issue and its Severity-based result are deleted; Severity now backs
ValidationError.
Every validation layer (structural, DTD, XSD content and identity, Schematron)
now produces located ValidationError values through the shared framework.
Slice 3 of #92. The DTD validator's imperative walk becomes declared Validation
values over a DTDSchema document:
- DTDSchema is now just the schema (content models + attribute declarations) and
serves as the rules' Document context; its validate/walk methods are gone.
- contentModel and attributeDeclarations are Validation<Element, DTDSchema>, so
every content-model and attribute violation is now located by coding path
("... at path: r/a"). undeclaredElement is added only in strict mode.
- identifierIntegrity is a Validation<Node, DTDSchema> gated to the root, doing
the document-wide ID-uniqueness and IDREF-resolution checks in one pass.
- DTD.validator(strict:) assembles the rule set; validateAgainstInternalDTD runs
it and returns [ValidationError] instead of [Issue].
A new test pins the located result. Schema (XSD) and Schematron validators still
use Issue and migrate in the following slices.
Slice 1 of #92. Introduce the OpenAPIKit-idiom validation framework, replacing
the imperative if-tree structural validator:
- ValidationError (reason + coding path) renders "<reason> at path: root/row/cell";
ValidationErrorCollection gathers every failure and is thrown once.
- Validation<Subject, Document> is the atomic, composable rule value, with the
multi-error and single-error Bool inits (the latter auto-forms "Failed to
satisfy: <description>"). Predicate-gated, type-directed via an erased
AnyValidation that guards an optional from matching a non-optional rule.
- The framework is parameterized over both Subject and Document so the structural,
DTD, XSD, and Schematron layers share one machinery (Document = Void for rules
that need only their subject).
- Validator<Document> walks the node tree, building the coding path, with blank /
default factories and fluent validating / withoutValidating.
- The combinator algebra (&&, ||, comparison lifts, take, lift, unwrap, all) as
free functions returning closures.
The structural duplicate-attribute rule is reborn as Validation<Element, Void>.
Combinators and the erased wrapper are unit-tested as pure functions per the rule.
The legacy Validation.Issue stays only for the not-yet-migrated schema validators
and is retired in a later slice.
Add PureXML.Validation.Schematron: compile a Schematron schema (matching
schema/pattern/rule/assert/report by local name, so the ISO and legacy
namespaces both work) and validate a document. A rule's context XPath
selects the nodes it fires on; each assert/report test is evaluated relative
to the node via the XPath context-node API. A failed assert is an error, a
matched report a warning; within a pattern a node fires the first matching
rule. Adds XPath.Query.nodes(over:) returning matched tree nodes so one tree
can be queried repeatedly.
Closes#52Closes#53Closes#25
Complete the structural DTD validation layer. AttributeType now distinguishes
the ID family; the validator does a second pass after the structural walk
(since an IDREF may point forward to an ID later in the document):
- ID-typed attribute values must be unique across the document.
- IDREF values must each resolve to a declared ID.
- IDREFS values are split on whitespace and each token must resolve.
Reimplemented from the DTD ID/IDREF spec; no upstream code copied. 4 tests; 79
total.
Surface <!ATTLIST> declarations from the parser and validate attributes against
them. Reimplemented from the DTD attribute-list spec; no upstream code copied.
- DocumentType gains attributeLists (raw ATTLIST bodies per element); the
DoctypeScanner captures them and the EventReader plumbs them through.
- AttributeListParser parses a body into AttributeDeclaration values
(AttributeType: cdata or enumeration; AttributeDefault: required/implied/
fixed/value).
- DTDSchema now checks #REQUIRED presence, #FIXED value match (when present),
and enumerated-value membership, in addition to content models.
- Tokenized types (ID/IDREF/NMTOKEN) parse but are not yet value-enforced.
- 5 attribute tests; 75 total.
Surface the parsed <!DOCTYPE> internal subset's <!ELEMENT> declarations from the
parser and validate a tree against them. Reimplemented from the DTD content-model
spec; no upstream code copied.
- DoctypeScanner captures <!ELEMENT name model> declarations; the parser surfaces
them as Parsing.DocumentType (via parseWithDocumentType).
- Validation.ContentModel + ContentModelParser parse a model string (EMPTY, ANY,
(#PCDATA), mixed, and element content with sequence/choice and ?/*/+).
- ContentModelMatcher matches the child element-name sequence against a model as
a regular language (NFA-position matching).
- Validation.DTDSchema.validate reports issues for EMPTY/PCDATA/mixed violations,
element content with stray text, and child sequences that do not match.
- PureXML.validateAgainstInternalDTD(_:) parses and validates in one call.
- ATTLIST validation is out of scope. 9 tests; 70 total.
Root Swift package mirroring PureYAML's structure, rules, and verification
gates. Includes the XML node model (Node, Element, Attribute, QualifiedName),
a working emitter (Serializer) with escaping and pretty/compact options, and
structural validation. The parser surface is stable; parse() raises
notImplemented until the scanner lands, pinned by a test.
- dependency-free, namespaced under PureXML
- portability and style gates carried over (style, namespacing, forbidden
patterns, swiftformat, swiftlint, linux, wasm, windows CI)
- 11 tests passing
