Commit Graph

246 Commits

Author SHA1 Message Date
2c375ac74b feat(parsing): fatal encoding-declaration contradictions, E38 version rule
A declared 16/32-bit encoding over BOM-less 8-bit bytes is rejected; a
BOM followed by a declaration outside its family is rejected, reading
the declaration through the BOM's own byte pattern; and an external
entity whose text declaration names a higher XML version than the
document is refused (errata E38). The eduni not-wf baseline is empty.
Reader and decoder helpers split for the length caps.

Closes #137
2026-06-10 08:16:08 +02:00
4b03c45434 feat(parsing): Namespaces 1.0 constraint enforcement
The namespace layer rejects malformed qualified names, declarations of
the xmlns prefix, misbindings of the xml prefix and either reserved
namespace name (including as the default namespace), and prefix
undeclaration; attributes must be distinct by expanded name; PI targets
and entity/notation names are NCNames; and DTD-typed xmlns values bind
their normalized form. Clears all 18 namespace cases from the eduni
baseline; xmltest valid/sa/012 joins the documented deviations per the
suite's own NAMESPACE='no' flag.

Closes #136
2026-06-10 08:06:45 +02:00
6861cd5d62 feat(validation): eduni errata behaviors, 3.3.3 attribute normalization
The eduni sections run with exact classified baselines. Fixes: 3.3.3
CDATA attribute-value normalization (literal whitespace to space before
reference decoding; char-ref whitespace survives; tokenized collapse on
SPACE only); production-68 lenient undeclared entities in content and
attribute values (validity finding, reference kept literal, nothing
fetched); errata E15 (references/comments/PIs/whitespace in EMPTY,
char-ref whitespace as data in element content, double-escape shape
included); errata E2 duplicate enumeration tokens; errata E14
declarations completed inside PE replacements (VC: Proper
Declaration/PE Nesting); NCName ID values; namespace declarations
subject to the every-attribute-declared rule. Reader validity machinery
split to EventReaderValidity for caps.

Closes #127
2026-06-10 07:50:47 +02:00
c084d01d7c feat(parsing): IBM section clean, entity-declared WFC/VC split, declaration recursion
The IBM section (1094 cases) runs clean to two documented classes:
279 pre-Fifth-Edition character-class cases (P85-89, count asserted
exactly) and 2 PE-in-internal-subset feature cases. Real gaps closed:
a document needs a root element (production 1); entity references in
literals must be Names or character references; WFC: No Recursion
checked at declaration over the general/parameter reference graph;
undeclared PE references split into WFC (standalone, or no external
declarations possible) versus deferred validity finding, also for
attribute-default references; refused external PEs are neither; and
groups split across PE replacements report VC: Proper Group/PE
Nesting. Entity machinery split to DoctypeScannerEntities for caps.

Closes #125
2026-06-10 07:26:50 +02:00
b27b015f78 feat(validation): standalone validity constraints and entity WFC
The scanner records declaration provenance (internal-subset entities,
element models, attribute lists); the validator reports the three
standalone='yes' dependencies on the external subset: a default that
would be supplied, a non-CDATA normalization that would change a value,
and whitespace in externally-declared element content. The parser hides
externally-declared entities from standalone documents, so referencing
one reports as undeclared (WFC: Entity Declared, 2.9). The Sun section
is fully clean: not-wf 56/56, valid 28/28, invalid 74/74, no baseline.

Closes #134
Closes #124
2026-06-10 07:13:30 +02:00
89cb299201 feat(validation): DTD declaration-level validity constraints
VC Root Element Type (root matches the DOCTYPE name), VC Unique
Element Type Declaration, VC No Duplicate Types in mixed content, VC
One ID per Element Type, VC Notation Attributes and Notation Declared,
VC Attribute Default Legal plus ID Attribute Default, lexical Name
checks for the ID family, and CDATA as character data in element
content. Strict mode adds VC Attribute Value Type (every attribute
declared; namespace declarations exempt). Declaration findings are
computed once at schema build and reported at the document root. Sun
invalid coverage moves from 43/74 to 62/74; the twelve remaining are
all the standalone='yes' family.
2026-06-10 07:05:14 +02:00
497e2460a3 feat(parsing): splice entity replacement markup into content
A general entity whose replacement text contains markup (directly or
transitively) is reparsed as content at the reference (4.4.2 Included):
the replacement is WFC-validated, cycle-checked, and budgeted, then
pushed back into the reader stream, so its elements become elements in
the tree and the pull-event stream rather than escaped character data.
Character references in internal entity literals now expand at
declaration time for general entities too (4.4.5), making char-ref
markup splice and the Appendix D double escape render as a literal
ampersand; the reference-time WFC validates remaining references
instead of re-expanding. Text-only entities keep the coalescing
single-pass path. Sun valid is 28/28.

Closes #133
2026-06-10 06:55:40 +02:00
82e70679ef feat(validation): 3.3.3 attribute normalization, 4.3.1 text declarations, 4.4.5 PE char-refs
Tokenized (non-CDATA) attribute values normalize before validity
checks, so enumerations, NOTATION lists, and the ID/IDREF table see
collapsed values. External parsed entities lose their leading text
declaration (validated: version optional, encoding required, never
standalone) instead of carrying it into replacement text. Character
references in PE literals expand at declaration time, making the
Appendix D '%zz;' example parse. A PI target inside the DTD subset
must be separated from its data. Sun baselines tighten to not-wf 55/56
and valid 26/28; the two remaining valid failures share one cause
(replacement text with markup is included as character data, filed
separately).
2026-06-10 06:43:25 +02:00
5671fb3ce1 feat(parsing): strict DTD subset grammar, oasis section of xmlconf fully clean
The OASIS/NIST section of the W3C conformance suite runs via a new
XMLCONF_ROOT-gated runner driven by the section's own manifest (parsed
with PureXML): 247/247 not-wf rejected, all 100 well-formed cases parse,
zero deviations. Closing its 24 gaps makes the DTD subset grammar
strict: exact PEReference shape, strict declaration names and keywords,
NDATA and notation tails, no bare '%' in entity values, case-sensitive
balanced conditional sections, junk between declarations rejected,
external-subset violations now propagate (with text-declaration
support, production 77), and a PI target must be separated from its
data. The xmltest baseline tightens to 181/186: CDATA in the internal
subset (107) is now rejected, five documented deviations remain.

Closes #123
2026-06-10 06:29:13 +02:00
c8f91e329c refactor(xsd): schema-consistency checks through the validation framework
The derivation final check and Particle Valid (Restriction) move from
fail-fast throws at schema compile into composable Validation rules
(finalRespected, restrictionsAreSubsets) over per-type SchemaTypeFact
subjects, so a schema with several problems reports every finding at
once. SchemaError gains .inconsistent([String]) carrying all findings
(texts unchanged) and drops .finalViolation/.invalidRestriction.
checkRedefine, checkAllGroups, and notASchema stay throws: they examine
the raw schema source, the compilation analog of well-formedness.
2026-06-10 06:04:40 +02:00
32d42410b9 feat(parsing): entity replacement-text well-formedness, fifth suite pass
Implement the replacement-text constraint at the point of reference: when a
general entity is included, its replacement (character references expanded
as at declaration time) must reparse as balanced content in isolation, via a
new EntityReplacementGrammar mini-parser. This rejects a bare or incomplete
reference ('&', '&#9' unterminated), tags spanning the entity boundary
('</foo><foo>'), a raw '<' or incomplete reference inside an embedded
attribute value, '<' followed by a non-name character, and a reserved-target
PI in the replacement, exactly the suite's semantics, including that an
UNREFERENCED entity may carry any value and that only the first declaration
of a name binds. Attribute-list defaults must reference already-declared
internal (not external/unparsed/forward-declared) entities with a
non-recursive chain. A CDATA section inside replacement text now shields its
content from reference expansion, fixing the last valid-side case.

W3C xmltest final standing: valid/sa 119/119, not-wf/sa 180/186, with the
six remaining deviations all deliberate and documented in the baseline:
internal-subset conditional/CDATA sections (063, 107) and parameter-entity
references (160-162) are features this package supports, and 141 expects the
1998 suite's name-character classes where this package implements XML 1.0
Fifth Edition's. Zero false positives across all 846 tests at every step.

Closes #120
2026-06-10 05:47:42 +02:00
dd0ff5d086 feat(parsing): fourth W3C-suite strictness pass, declaration grammar
Make the DTD declaration grammar strict: <!ENTITY> requires its whitespace
(after the keyword, around %, after the name), a real name (not #DEFAULT),
nothing but '>' after the value, NDATA preceded by whitespace and forbidden
on parameter entities; entity-value literals are validated (XML Chars only,
every & a complete reference); SYSTEM/PUBLIC external identifiers parse
strictly (PUBLIC needs both literals for entities and the DOCTYPE, public-only
stays legal for notations) with PubidChar validation; <!ATTLIST> bodies are
validated against XML 1.0 productions 52-60 via a new DTDAttListGrammar
(spacing, attribute types, enumerations, quoted defaults); the reserved xml
PI target is rejected inside the subset. Also: strict UTF-8 decoding (no
silent replacement of malformed sequences), XML Char checks in attribute
values and CDATA, the declaration's version/encoding value grammar, and
required whitespace between pseudo-attributes. The strict machinery lives in
DoctypeScannerStrict/DTDAttListGrammar to stay under the length caps.

W3C xmltest: not-wf/sa 162/186 rejected (from 129; 80 at first contact),
valid/sa 118/119, zero false positives across all 846 tests at every step.
The remaining 24-case baseline is exactly: the entity replacement-text class
(19) and the deliberate internal-subset extensions this package supports as
features (conditional sections, PEs in the internal subset: 5).

Part of #120, #105
2026-06-10 05:34:11 +02:00
805993e656 feat(parsing): strict <!ELEMENT> content-model grammar
Third W3C-suite strictness pass: DTDContentModelGrammar validates each
element declaration's content specification against the XML 1.0 grammar
(productions 46-51) at DTD scan time, EMPTY/ANY, Mixed (with #PCDATA first,
names only, the trailing * required with alternatives, no nesting), and
children (one connector per group, quantifiers bound directly to their
particle, balanced parentheses, no empty groups, no SGML leftovers like
'- -', 'O O', '&' connectors, or inclusion/exclusion suffixes). The element
declaration also requires whitespace after <!ELEMENT and after the name. The
lenient ContentModelParser still drives validation behavior; this is a
well-formedness gate. not-wf/sa: 129/186 rejected (from 109); valid/sa
unchanged; zero false positives across all 846 suite tests; baseline rebased
to the exact 57 remaining (declaration-grammar details and the entity
replacement-text class).

Part of #120, #105
2026-06-10 05:20:33 +02:00
848c7002c7 feat(parsing): second W3C-suite strictness pass
Character references now require lowercase 'x' for hex and a referenced code
point inside the XML Char production (rejecting &#0;, &#31;, &#xFFFF;,
surrogates, and &#X58;). The reserved 'xml' processing-instruction target
(any case) is legal only as the declaration at the document's very first
bytes: whitespace or a comment before <?xml ...?>, a declaration inside
content or after the root, and case variants like <?XML?> are all rejected.
Comments and PI data are checked for the XML Char production like content.
not-wf/sa: 109/186 rejected (from 93); valid/sa unchanged at 118/119; all
846 suite tests green, baseline rebased to the exact 77 remaining (the
internal-subset grammar and entity replacement-text classes on #120).

Part of #120, #105
2026-06-10 05:16:05 +02:00
4ed8e9518a feat(parsing): first W3C-suite strictness pass; opt-in xmltest runner
Add five well-formedness checks the W3C xmltest suite surfaced: '--' inside
comments (also rejecting the ---> ending), raw '<' in attribute values, the
literal ]]> sequence in character data, attributes not separated by
whitespace, and characters outside the XML Char production in content, each
a located ParseError. The attribute scanners move to EventReaderAttributes
to stay under the type-body cap.

Add the opt-in W3C xmltest runner (XMLTS_ROOT-gated; the suite is never
vendored, James Clark's license permits redistribution only as the unmodified
archive; Foundation is used in this one test file for file access only).
Baseline: valid/sa 118/119, not-wf/sa 93/186 rejected (up from 80), the
remaining 93 carried as an exact known-deviations baseline so progress and
regressions are both visible. The two remaining leniency classes (internal-
subset DTD grammar, entity replacement-text well-formedness) are #120.

Part of #105 (Tier 2)
2026-06-10 04:59:39 +02:00
f792fd6e8b feat(xsd): Particle Valid (Restriction)
A complex type derived by restriction is checked at schema compile to accept
a structural subset of its base, closing the declared XSD 1.0 spec gap.
ParticleRestriction covers element name matching, element-vs-wildcard
admission, wildcard narrowing, occurrence-range subsumption, and the group
recursions: sequence/all order-preserving with emptiable skips (Recurse),
choice mapping (RecurseLax), sequence-into-choice (MapAndSum, approximated by
per-particle checks, a documented simplification), sequence-into-all
(RecurseUnordered), and RecurseAsIfGroup for a lone element against a group.
An unfaithful restriction throws SchemaError.invalidRestriction(type:base:
reason:). All 834 pre-existing tests pass unchanged (no false positives on
the existing restriction-using schemas); 7 new tests cover the accept and
reject paths plus a conformance-harness pair.

Closes #117
2026-06-10 04:13:54 +02:00
125274786e refactor(validation): one shared childSteps; matcher and tokenizer differentials
Extract PathKey.steps(forChildNames:) as the single path construction behind
the validator walk, the XSD content validator, and the identity validator,
replacing three identical implementations, so located errors can never
disagree on a path. Add two differentials locking the duplicated engines
together: equivalent DTD and XSD content models (sequence, choice, *, +, ?,
nested choice) must agree on every child sequence across the positional
matcher and the Thompson NFA; and the pull (EventReader) and push
(PushScanner) tokenizers must yield the same event stream for the same
document, even with the push side fed one character at a time (adjacent text
events coalesced, since chunk boundaries legitimately split text). Both
pairs agree on everything today; the differentials guard them.

Closes #115
2026-06-10 04:06:26 +02:00
4130d905c8 refactor(parsing): drop slice copies; document the expansion bounds
The whole-buffer ISO-2022-JP, GB18030, and EUC-TW decoders index the input
ArraySlice directly instead of materializing an Array first (one avoidable
O(n) allocation per decode each); declaredEncoding's subsequence scan uses
elementsEqual instead of allocating an Array per candidate offset. Document
the two analysis-flagged bounds: entity expansion is doubly bounded (the
character budget caps output; the visiting set caps recursion depth at the
number of declared entities), and the RELAX NG derivative engine carries no
memoization by design (the simplifying constructors keep real grammars
small; adversarial nested-choice grammars can grow the residual, so treat
untrusted schemas accordingly). Behavior unchanged, all suites green.

Closes #118
2026-06-10 04:01:16 +02:00
363a88070a docs(model): TreeNode ownership and concurrency contracts
Document the weak-parent ownership trap at the public entry points: parseTree
and readTree now state that a tree (or ownerDocument) must be held while using
any node inside it, that TreeNode has no internal synchronization (confine to
one thread or actor), and that the immutable Sendable Node projection is the
way across concurrency boundaries; the TreeNode class doc carries the same
contract. QualifiedName's malformed-colon fallback (':local', 'prefix:', '::'
stay whole and unprefixed), already implemented, is now documented on init
and locked by a test.

Closes #114
2026-06-10 03:46:44 +02:00
b8dbf8e97a perf(xpath): compute document-order keys once per sort
Node-set ordering previously recomputed both nodes' root paths on every
precedes comparison (treePath is O(depth x siblings), so sorts were
O(n log n x depth)). Add decorate-sort-undecorate helpers,
sortedByDocumentOrder() and firstInDocumentOrder(), that compute each node's
order key once, and switch the evaluator's order-and-dedup, the EXSLT set
functions, string-value coercion, and NodeFunctions' target lookup to them.
set:leading/trailing also decorate once, reusing the keys for the pivot
filter and the final sort. precedes stays for pairwise use, sharing the
key comparator (Node.ordered). Behavior unchanged, all suites green.

Closes #113
2026-06-10 03:39:19 +02:00
0edc1f77f9 fix(xsd): report identity-constraint XPath compile errors
A selector or field XPath that fails to compile is now reported once up
front as a located error (identity constraint 'k': invalid selector/field
XPath '...'), instead of try? silently turning a schema author's typo into
a disabled constraint. The per-run query cache from #112 is the choke point:
compileErrors() probes every constraint's selector and fields in
deterministic order, the failures cache as nil, and the walk afterwards
treats those constraints as matching nothing, so the rest of the document
still validates. Tests cover a malformed field and a malformed selector.

Closes #111
2026-06-10 03:33:20 +02:00
7d3ef26558 perf(xpath): compile patterns once per run, not per node
XSLT: template selection resolves each match pattern once per transform via
a per-run MatchCache (the pattern's node set is computed once over the source
tree; selection becomes an identity lookup), replacing per-(node, template)
recompilation and full-tree re-walks. XSD identity validation: a per-run query
cache compiles each selector/field XPath once instead of at every visited
element. Schematron was verified to already store compiled queries. Behavior
unchanged (all suites green); a multi-template shape test over 300 nodes
guards the optimized path. MatchCache lives in its own file to keep the
transformer under the length cap.

Closes #112
2026-06-10 03:24:38 +02:00
38b3f75243 fix(xsd): error on unknown xsi:type; cycle-detect type references
An xsi:type naming an undeclared type is now a located error in both the
tree validator (validateChild) and the streaming driver, instead of a silent
fallback to the declared type. typeReference chains resolve through one
shared resolver (resolveReference, with a visited set), replacing the inline
recursion, the 64-hop cap, and the completions guard: a circular chain now
reports 'circular type reference' instead of silently truncating, and the
streaming shallow check no longer recurses without bound on a cyclic table
(a latent crash). Tests cover unknown xsi:type (tree + streaming agree),
the circular chain (tree + shallow), the resolver's three outcomes, and a
conformance-harness case pair.

Closes #110
2026-06-10 03:10:00 +02:00
4c93555d1f refactor(xsd): express the streaming content check as a Validation value
Rule-canon audit (validation-rules.md / the OpenAPIKit idiom): the streaming
XSD validator was the one surface applying a bare method (validateShallow)
rather than a Validation<> value. Wrap it: ComplexValidator.shallowValidity is
a Validation<ResolvedElement, ComplexValidator> whose subject carries the
element with its already-resolved effective type (the contextual typing a
node-tree walk cannot express), and the streaming driver applies it via
apply(to:at:in:). Now every validation surface, DTD, XSD, RELAX NG, Schematron,
HTML, structural, conformance, and both streaming validators, routes through a
Validation value. Adds a direct unit test of the rule.
2026-06-10 02:14:02 +02:00
067aa91861 fix(html): fragment parser reconstructs active formatting elements
The fragment TreeBuilder gains the HTML5 active-formatting-list and reconstruct
step that the document builder already had: when a formatting element (b, i,
em, ...) is closed out of order, it is remembered and re-opened for the
following content. So HTML.parse now agrees with HTML.parseDocument and the
spec, e.g. <b><i></b>X</i> -> <b><i></i></b><i>X</i> and
<p>1<b>2<i>3</b>4</i>5</p> -> <p>1<b>2<i>3</i></b><i>4</i>5</p>. The conformance
corpus that surfaced this now checks the fragment path directly.

Closes #109
2026-06-10 01:59:16 +02:00
83507939de feat(validation): conformance-corpus harness via the validation framework
Begin Tier-2 conformance checking, expressed through the OpenAPIKit validation
idiom rather than an ad-hoc loop. ConformanceCase is a Validatable named
expectation; Conformance.matchesExpected is a Validation<ConformanceCase, Void>
that emits one located ValidationError per diverging case, so a suite reports
every failure at once with the case name as its coding path. Seeded with a
Canonical XML 1.0 corpus (attribute ordering, empty-element expansion, comment
removal, CDATA-to-escaped-text, PIs, namespace rendering) and a test proving a
divergence is reported as a located failure. Growing the corpora against the
official suites is the ongoing Tier-2 work.

Part of #105 (Tier 2)
2026-06-09 19:08:17 +02:00
963c7c74eb fix(xsd): locate identity-constraint errors at the offending field
Extend an identity error's coding path past the target element to the single
field the constraint names: an attribute (list/item[2]/@id) or a child element
(.../code), resolved from the field XPath. Multi-field or complex-path
constraints stay on the target. Also add a cross-scope keyref test confirming
a keyref resolves against a key declared on an ancestor element. Updates the
identity-path tests to the more precise locations.

Closes #104
2026-06-09 18:58:27 +02:00
69678d31a1 feat(xpath): EXSLT common, math, and sets extension functions
Add the EXSLT common/math/sets modules, dispatched by resolving a function's
prefix to its namespace (FunctionTable.call falls back to namespace lookup),
so any prefix bound to an EXSLT namespace works. math:min/max/highest/lowest/
abs/sqrt (with the NaN rule), set:distinct/difference/intersection/
has-same-node/leading/trailing (by node identity and document order), and
exsl:object-type. math:power and the transcendentals are omitted (they need a
C math library the Foundation-free target excludes); exsl:node-set awaits
result-tree-fragment support.

Part of #105 (Tier 3)
2026-06-09 18:31:50 +02:00
93f31d7758 feat(c14n): Canonical XML 1.1 (xml:base merging, no xml:id inheritance)
Add a canonical11 preset / mergeInheritedBase option. On a document subset,
the omitted-ancestor xml:base chain is merged into the apex by RFC 3986
reference resolution (vs 1.0 nearest-wins), the apex's own relative xml:base
is resolved against it, and xml:id is no longer inherited; xml:lang/xml:space
still inherit nearest. A self-contained RFC 3986 resolver (parse, path merge,
dot-segment removal) backs the merge, with tests covering the 1.1-vs-1.0
difference, dot segments, xml:id, and lang/space.

Part of #105 (Tier 3)
2026-06-09 17:59:35 +02:00
56e889f790 feat(parsing): EUC-TW (CNS 11643) decoding, closing #108
Add EUC-TW for the whole-buffer and streaming decoders: ASCII, a two-byte
plane-1 pair, and a 0x8E four-byte plane selector plus a pair. CNS planes 1
and 2, which carry essentially all real-world EUC-TW text, are vendored from
the ICU euc-tw-2014 mapping (private-use fallbacks dropped); planes 3-15 are a
documented follow-up. Tested for plane 1 (two-byte and four-byte forms), plane
2, and streaming. The streaming method moved to its own file to stay under the
type-body cap.

Closes #108
2026-06-09 17:44:22 +02:00
191d2245de feat(parsing): streaming honors the declared encoding; streaming ISO-2022-JP
The streaming decoder previously selected an encoding only from a byte-order
mark or the UTF sniff, so its single-byte and CJK decode paths were latent. It
now reads the encoding from the XML declaration (the libxml2 behavior) when
there is no BOM, activating those paths over an incremental byte source. Add
the streaming form of ISO-2022-JP, carrying the stateful mode across reads
(extracted to its own file to stay under the type-body cap). A test asserts
streaming agrees with whole-buffer decode for a single-byte (ISO-8859-1), a
CJK (Shift_JIS), and an ISO-2022-JP document.

Part of #108, #105
2026-06-09 17:33:52 +02:00
a7f5407552 feat(validation): streaming XSD validation, completing #107
Add Schema.Document.validate(streaming:) and ComplexValidator.validateShallow:
resolve each element's type from its parent's content model (and xsi:type /
typeReference) as it opens, then check its attributes and content-model
structure shallowly as it closes, against a shallow element synthesized from
the streamed child names. A child's deeper validity is checked when it closes,
so only the open-element stack is held. A differential test asserts streaming
agrees with the tree validator across valid and invalid documents (missing
attributes, bad simple values, missing/extra/misordered children). Wildcard
processContents and substitution-group resolution in the streaming form stay
on the tree validator. The streaming methods moved to ComplexValidatorStreaming
to keep ComplexValidator under the file/type-body caps.

Closes #107
2026-06-09 16:52:17 +02:00
1e815d6309 feat(parsing): ISO-2022-JP decoding
Add the WHATWG iso-2022-jp decoder, the first stateful encoding: escape
sequences switch between ASCII, JIS X 0201 Roman (yen, overline), JIS X 0201
katakana, and the JIS X 0208 plane, resolved through the shared jis0208 table
(no new data). Tested across all four modes, with the JIS X 0208 kanji
cross-checked against the EUC-JP decoder (same table, same pointer). EUC-TW and
the streaming form of ISO-2022-JP remain.

Part of #108, #105
2026-06-09 10:55:34 +02:00
051ef02fdb feat(validation): streaming RELAX NG validation
Add RelaxNG.validate(streaming:), driving the Brzozowski derivative directly
from the parse-event stream (the libxml2 xmlTextReader model). The derivative
is already incremental: the after-threaded pattern chain is the validation
stack, so only the residual pattern and a light per-element frame (for the
childrenDeriv ignorable-whitespace rule) are retained, never the tree. A
differential test asserts streaming agrees with the trusted tree engine across
valid, invalid, whitespace, and mixed-content documents. XSD streaming remains.

Part of #107, #105
2026-06-09 10:35:42 +02:00
f9f9985aac feat(validation): streaming DTD ID/IDREF integrity
Complete the DTD streaming validator with document-scoped ID/IDREF checks,
held in bounded sets (the identifier values, not the tree): a duplicate ID
is reported at its repeat occurrence, and forward IDREFs resolve at the end
of the stream via finish(). The streaming-vs-tree agreement test now covers
a duplicate ID as well. DTD streaming validation (content, attributes,
ID/IDREF) is now complete; RELAX NG and XSD streaming remain.

Part of #107, #105
2026-06-09 10:21:20 +02:00
b5fa216a7c feat(validation): streaming DTD validation (reader-driven)
Add PureXML.Validation.StreamingDTDValidator and PureXML.validate(streaming:
dtd:), validating a document against a DTD while it is pulled event by event
(the libxml2 xmlTextReader model), holding only the open-element stack rather
than the whole tree. Each element's content model and attributes are checked
as it closes by applying the exact per-element DTD rules to a shallow element
synthesized from the streamed child names, so streaming and tree validation
agree (a test asserts matching reasons). Errors carry an xmlGetNodePath-style
coding path. RELAX NG/XSD streaming and ID/IDREF streaming remain on #107.

Part of #107, #105
2026-06-09 10:06:55 +02:00
b98c2ff41f feat(emitting): CJK output encodings, closing output-encoding parity
Extend ByteEncoder to the multi-byte CJK encodings (Shift-JIS, EUC-JP,
EUC-KR, GBK, GB18030, Big5). Each inverse is built by running the existing,
tested whole-buffer decoder over every valid byte sequence and recording
the shortest sequence per scalar, so the encoder is by construction the
exact inverse of the decoder, with no new data or separate encode tables.
GB18030's astral range uses the four-byte pointer formula, so every scalar
encodes. Round-trip tests cover Japanese, Korean, Chinese, and a GB18030
astral character.

Closes #106
2026-06-09 09:41:28 +02:00
e62995a7a8 feat(emitting): output encoding for UTF and single-byte families
Add PureXML.Parsing.ByteEncoder, the inverse of ByteDecoder, and a
Serializer.serialize(_:encoding:) / PureXML.serialize(_:encoding:) that
emits bytes in a chosen encoding (the libxml2 save-with-encoding model).
Covers the Unicode transformation formats and the single-byte legacy
families, built by inverting the existing vendored to-Unicode tables, so
no new data is introduced. The declaration carries the canonical encoding
name, a UTF-16/32 stream is preceded by a byte-order mark, and any
unrepresentable scalar becomes a decimal numeric character reference, so
output round-trips. CJK output encodings remain (tracked on #106).

Part of #106, #105
2026-06-09 09:32:39 +02:00
36a40eec08 fix(xsd): locate identity-constraint errors at the offending node
Refine the identity-constraint path: rather than locating at the element
that declares the key, walk from the constraint-bearing element down to the
selected target and report the actual offender, so a duplicate key renders
"at path: list/item[2]" and a dangling keyref "at path: orders/line".
Also locate the "no element declaration for the root" XSD error at the root
element, matching the namespace check beside it that already did so. Lock
both with coding-path assertions in the identity tests.
2026-06-09 08:57:33 +02:00
9730903803 fix(xsd): locate identity-constraint errors at their element
The IdentityValidator emitted every key/keyref/unique error with an empty
coding path (at: []), so each rendered "at root of document" regardless of
which element violated the constraint. Thread a [PathKey] through the walk,
the way the content-model validator already does, so the failures carry the
constraining element's path. This restores the validation framework's
path-carrying contract for the one rule that was bypassing it.
2026-06-09 08:50:39 +02:00
74acad4628 feat(parsing): Big5 (Traditional Chinese / HKSCS)
Add the WHATWG big5 decoder over the vendored index-big5 (18,590 entries),
for both the whole-buffer and streaming decoders. A lead in 0x81-0xFE plus
a trail in 0x40-0x7E or 0xA1-0xFE resolves through the index, covering
Traditional Chinese and the HKSCS extensions including the astral CJK
ideographs. Four pointers decode to a base letter plus a combining mark.
Selected by the big5 / big5-hkscs declaration names.

Closes #103
2026-06-09 08:30:57 +02:00
2b2c1fb692 feat(parsing): full GB18030 four-byte sequences
Extend the GBK two-byte subset to full GB18030: a lead byte followed by
a digit opens a four-byte form whose pointer is mapped through the WHATWG
index-gb18030-ranges linear ranges, reaching the supplementary planes.
The 207 ranges are vendored verbatim. Wired into the whole-buffer and
streaming decoders and the encoding-name table (gb18030/gb-18030).

Closes #102
2026-06-09 08:24:00 +02:00
b688952cef feat(parsing): GBK/GB2312 decoding completes the CJK encodings (#99) 2026-06-09 08:08:52 +02:00
7f2991a90b Merge #99 (stage 3): EUC-KR 2026-06-09 08:03:02 +02:00
d8b2db4943 feat(parsing): EUC-KR (CP949) decoding with vendored index (#99) 2026-06-09 08:02:14 +02:00
5b5ee7b3fe feat(parsing): EUC-JP decoding with vendored JIS X 0212 index (#99) 2026-06-09 07:57:52 +02:00
4ff9306a0e feat(parsing): Shift-JIS decoding with vendored JIS X 0208 index (#99) 2026-06-09 07:54:05 +02:00
a07537007e feat(html): vendor the complete WHATWG named entity set (#84) 2026-06-09 07:45:55 +02:00
7eb33c0d08 feat(html): full HTML 4.01 named entity set (#84) 2026-06-09 07:39:32 +02:00
3f45dacf62 feat(parsing): Windows-1255/1256/1258, KOI8-U complete the single-byte set (#97) 2026-06-09 07:29:25 +02:00
5646e81b94 feat(parsing): Windows-1250/1251/1253/1257, KOI8-R, ISO-8859-11 (#97) 2026-06-09 07:25:18 +02:00
3f293ec1e7 feat(parsing): ISO-8859-6/8/10/14/16 vendored from Unicode mappings (#97) 2026-06-09 07:19:50 +02:00
43dc7b00c4 feat(parsing): ISO-8859-3/4/7/13 vendored from Unicode mappings (#97) 2026-06-09 07:15:35 +02:00
5f35621ae6 feat(parsing): ISO-8859-2 (Latin-2) vendored from Unicode mapping (#97) 2026-06-09 07:08:19 +02:00
a005def333 feat(parsing): Windows-1254 (Turkish) encoding (#97) 2026-06-09 07:02:13 +02:00
94890172f2 feat(parsing): single-byte encoding framework + ISO-8859-5/9/15 (#97) 2026-06-09 06:57:21 +02:00
a5b032e9dc feat(html): in-select-in-table closes select on table tags (#83) 2026-06-09 06:37:35 +02:00
3ebbaff07c feat(html): select content model (optgroup/nested-select/input close) (#83) 2026-06-09 06:32:24 +02:00
7add3b3fa5 refactor(html): build head on the live TreeNode model, remove DocFrame 2026-06-09 06:29:00 +02:00
b245640ee2 fix(html): keep template content nested instead of splitting to body (#83) 2026-06-09 06:21:10 +02:00
f637b8e154 feat(html): frameset documents and void frame element (#83) 2026-06-09 06:17:55 +02:00
2945de0855 feat(html): SVG HTML integration points switch back to HTML namespace (#83) 2026-06-09 06:06:54 +02:00
47e0e3e644 feat(html): table foster-parenting for stray content (#83) 2026-06-09 06:04:01 +02:00
038774cfc2 feat(html): SVG attribute-name case adjustment in foreign content (#83) 2026-06-09 05:52:26 +02:00
9092ab3dbd feat(html): adoption agency algorithm and active formatting elements (#83) 2026-06-09 05:46:12 +02:00
fccb407797 refactor(html): build body as a live mutable TreeNode open-elements stack (#83) 2026-06-09 05:35:26 +02:00
40c5973d5a feat(html): SVG element-name case adjustment in foreign content (#83) 2026-06-09 05:29:14 +02:00
d64bcaf2fa feat(html): SVG/MathML foreign-content namespace switching (#83) 2026-06-09 05:22:37 +02:00
b7f7e15fbc feat(html): table tree construction with implied tbody/tr (#83) 2026-06-09 05:18:13 +02:00
e86c87af3c feat(html): numeric-ref validation, NUL, RCDATA decoding, semicolon-less refs (#84) 2026-06-09 05:11:18 +02:00
3a55eeeaa6 feat(html): conformance validator as composable Validation rules 2026-06-09 04:39:03 +02:00
506c502602 refactor(validation): decompose DTD attribute checks into per-constraint rules (#101) 2026-06-09 04:28:03 +02:00
ecb9b8c881 feat(xinclude): range inclusion via XPointer range model (#87) 2026-06-09 04:19:38 +02:00
8871fee977 feat(xpointer): range(), range-to(), and string-range() schemes (#88) 2026-06-09 04:16:23 +02:00
b3bbacb4e8 feat(canonical): C14N 2.0 sequential prefix rewrite, QName-aware values, node-set selection (#85) 2026-06-09 04:06:02 +02:00
e77b39a449 feat(canonical): node-subset C14N with ancestor namespace and xml:* inheritance (#85) 2026-06-09 03:55:54 +02:00
87fd7ba889 feat(emitting): textEscaping round-trip option to preserve carriage returns (#91) 2026-06-09 03:48:20 +02:00
46131bff1b feat(model): first-class doctype/entity-reference/namespace nodes, ownerDocument, adoption (#90) 2026-06-09 03:43:24 +02:00
2db9915fb2 feat(schematron): support key(), document(), and current() in tests
Schematron tests, lets, and message expressions can now use key() backed
by xsl:key declarations parsed from the schema, document() loading an
external file through an injected documentLoader, and current() resolving
to the rule context node. A node-keyed function-table factory threads
through the evaluation. Completes the Schematron audit.
2026-06-09 03:30:43 +02:00
dbdf98e231 feat(relaxng): skip RNC >> follow-annotations, completing #79
A >> follow-annotation after a pattern (a foreign element name with
optional [ ... ] content) is skipped, chaining and combining with leading
annotations; neither carries schema semantics. Namespace declarations
inside annotations were already dropped by the annotation skipper, so the
RNC compact syntax is now fully covered.
2026-06-09 03:20:47 +02:00
7c24ada92b feat(xslt): document() fragments and base-URI resolution, completing #82
A document() reference with a #fragment selects a subset of the loaded
document via XPointer; a node-set first argument unions the documents
named by each node's string value; and a relative URI is resolved against
a caller-supplied baseURI (the loader is wrapped so xsl:include/xsl:import
resolve the same way). Closes the XSLT function/output audit.
2026-06-09 03:12:49 +02:00
4ea84b3d58 feat(xslt): make result-tree fragments usable as node-sets
An xsl:variable with a body now binds a queryable document fragment (its
instantiated result converted to a TreeNode document) instead of a plain
string, so count($rtf/*), $rtf/child, and for-each over $rtf/... work.
It still has its concatenated text as a string value, so existing string
uses are unchanged. Part of #82.
2026-06-09 03:04:56 +02:00
3fec78da51 feat(xslt): implement xsl:output cdata-section-elements
The text content of the named result elements is emitted in CDATA
sections rather than escaped: the result tree's text children of those
elements are converted to CDATA nodes before serialization, leaving
nested and other elements unaffected. Part of #82.
2026-06-09 02:58:34 +02:00
2a40300479 feat(xslt): emit xsl:output doctype-public/doctype-system
When doctype-system (optionally with doctype-public) is set on xsl:output,
a <!DOCTYPE> for the result's root element is emitted before the
serialized output: PUBLIC when both identifiers are given, SYSTEM
otherwise. Works for the XML and HTML output methods. Part of #82.
2026-06-09 02:53:50 +02:00
47b88f19b2 feat(xslt): key() over a node-set second argument
key(name, node-set), including key('k', .), now unions the indexed nodes
for every value node's string value and de-duplicates, instead of using
only the first node's string value. A string value still works directly.
Part of #82.
2026-06-09 02:50:24 +02:00
41a48d3130 feat(xslt): serialize method=html output by the HTML rules
An xsl:output method=html result is serialized through PureXML.HTML
(void elements emitted without a self-closing slash, raw-text elements
like script/style left unescaped) instead of as XML. Part of #82.
2026-06-09 02:48:00 +02:00
e30672d72b feat(xslt): implement xsl:sort case-order
case-order=upper-first/lower-first makes the string comparison
case-insensitive, with case used only to break ties among otherwise-equal
strings; without it the default codepoint order is unchanged. Part of #82.
2026-06-09 02:37:43 +02:00
322fd013e9 feat(xslt): add generate-id, system-property, *-available functions
generate-id returns a stable per-node identifier (the context node when
no argument is given, the empty string for an empty node-set);
system-property reports the XSLT version and vendor; element-available and
function-available answer against the implemented instruction and function
sets; unparsed-entity-uri is a stub. Part of #82.
2026-06-09 02:20:36 +02:00
610b7d0ca0 feat(xslt): implement xsl:namespace-alias, completing #81
A literal result element and its attributes in a stylesheet namespace
bound by stylesheet-prefix are rewritten on output to the namespace and
prefix bound by result-prefix (#default supported), so a stylesheet can
emit literal xsl: elements. Prefixes resolve through the in-scope xmlns
declarations; aliases fold across include/import. Splits AVT parsing into
XSLTAVT.swift and trims the transformer to stay within length budgets.
2026-06-09 02:10:16 +02:00
2d2572e265 feat(xslt): implement xsl:decimal-format for format-number
format-number now honors named and default decimal formats: the decimal
separator, grouping separator, percent, minus sign, zero-digit (for
non-Latin digit sets), NaN, and infinity symbols come from the chosen
xsl:decimal-format, in both the picture and the output. FormatNumber is
parameterized over the symbol set and the formats thread through the
function table; folded across include/import.
2026-06-09 01:59:37 +02:00
61ab8ce8f9 feat(xslt): implement xsl:fallback for forwards-compatible processing
An unrecognized XSLT element instantiates its xsl:fallback children in its
place, or is dropped when it has none; a fallback under a supported
instruction is ignored. The fallback content is handled in the instantiate
loop alongside variables, so the evaluate dispatch stays within budget.
2026-06-09 01:52:21 +02:00
4f11bafa28 feat(xslt): implement xsl:apply-imports
Re-applies templates to the current node in the current mode, considering
only templates of lower import precedence than the one being instantiated,
falling back to the built-in rule when none match. Threads the current
mode and import precedence through the evaluation context, and moves the
transformer runtime types into XSLTRuntime.swift (namespaced, via private
aliases) to stay within the file-length and namespacing gates.
2026-06-09 01:49:33 +02:00
21ed15b1ec refactor(xslt): keep ResultItem/XSLTContext file-private for namespacing
Only the static deduplicated helper moves to XSLTElements.swift (it needs
no private types); attributeSetAttributes stays in the transformer so the
evaluation context and result-item types remain file-private, satisfying
the one-file-scope-type namespacing gate.
2026-06-09 01:41:48 +02:00
22cd88f8ba feat(xslt): implement xsl:attribute-set and use-attribute-sets
Named attribute sets apply to a literal result element or xsl:element via
use-attribute-sets; sets may include other sets recursively with a cycle
guard, set attributes are lowest precedence so the element's own
attributes override them, and same-named attributes collapse to the last.
Folded across xsl:include/xsl:import. Splits the element-building helpers
into XSLTElements.swift to stay within the file-length budget.
2026-06-09 01:39:45 +02:00
367b5b9425 feat(xslt): implement xsl:message with terminate
xsl:message instantiates its body as diagnostic text and produces no
result-tree output; terminate=yes records the text and aborts the
transform with an XSLTError.terminated, halting further instantiation.
Splits the transform entry points into XSLTTransform.swift and moves the
message case to keep within the complexity and file-length budgets.
2026-06-09 01:29:37 +02:00
10d9525e3e feat(xslt): implement xsl:strip-space and xsl:preserve-space
Whitespace-only text nodes are removed from the source tree for elements
named by strip-space before transformation, with preserve-space and a
more specific name test winning over *, and xml:space=preserve on a
source element or ancestor overriding the stripping. The name sets fold
across xsl:include/xsl:import. Refactors the stylesheet compile loop
through a Parts accumulator to absorb the new top-level elements within
the complexity budget.
2026-06-09 01:22:59 +02:00
d80718c530 feat(relaxng): distinguish undeclared attribute from invalid value
A RELAX NG attribute failure now reports an undeclared attribute
("attribute @b is not allowed on <n>") separately from a declared
attribute carrying a bad value ("attribute @count on <n>: 'abc' is not a
valid integer"), quoting the value and naming its datatype by reusing the
text detail. The declared value pattern is found by matching the
attribute name through the residual.
2026-06-09 01:11:54 +02:00
3be6832c23 feat(relaxng): quote the offending value and name the datatype in text errors
A RELAX NG text/datatype mismatch now reports the rich datatype reason
(quoting the value and naming the type, e.g. 'abc' is not a valid
integer) and a value mismatch quotes both the actual text and the
required literal, instead of a generic "text content is not valid". The
detail is peeled from the content pattern at the failure point through
choice, group, list, and ref.
2026-06-09 01:09:29 +02:00
fb31033ce8 feat(dtd): located content-model errors with recovery hints
An element whose children break its content model now names each stray
child outside the model's alphabet individually, and reports a pure
order/count violation once with the allowed elements as a hint, instead
of a single opaque "do not match its content model". Adds an alphabet
extractor to the content-model matcher. This brings DTD to the same
located standard as the XSD, RELAX NG, and Schematron validators, so all
four schema languages give editor-grade diagnostics.
2026-06-09 00:59:25 +02:00
ea5387db90 feat(xsd): located, recovering content-model errors
A content-model violation is now placed at the offending child with a
recovery hint naming what was expected, instead of one opaque "content
does not match the content model" per element. The sequence/choice
automaton is walked child by child via its follow-set to flag the first
child the model rejects, or the incomplete content at the end; an xs:all
group recovers past each stray child and reports every missing required
member. Well-placed children are still validated for their own content,
so an editor sees every problem at once. Brings XSD to the same located,
recovering standard as the RELAX NG, DTD, and Schematron validators.
2026-06-09 00:55:17 +02:00