Add five well-formedness checks the W3C xmltest suite surfaced: '--' inside
comments (also rejecting the ---> ending), raw '<' in attribute values, the
literal ]]> sequence in character data, attributes not separated by
whitespace, and characters outside the XML Char production in content, each
a located ParseError. The attribute scanners move to EventReaderAttributes
to stay under the type-body cap.
Add the opt-in W3C xmltest runner (XMLTS_ROOT-gated; the suite is never
vendored, James Clark's license permits redistribution only as the unmodified
archive; Foundation is used in this one test file for file access only).
Baseline: valid/sa 118/119, not-wf/sa 93/186 rejected (up from 80), the
remaining 93 carried as an exact known-deviations baseline so progress and
regressions are both visible. The two remaining leniency classes (internal-
subset DTD grammar, entity replacement-text well-formedness) are #120.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 7 OASIS catalog cases through the
harness: exact system/public/uri entries, rewriteSystem and rewriteURI prefix
matching, and the unresolved outcomes for unmatched identifiers. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 17 XSD regex cases through the
pattern-facet path: implicit whole-string anchoring, the character-class
escapes (backslash-d/w/s), Unicode category and negated-category escapes,
quantifier ranges (exact, bounded, open), alternation, grouping, negated
character classes, and class subtraction. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 10 cases through the harness:
XInclude substitution, parse=text inclusion with escaping, xi:fallback on a
missing href, and relative href resolution against the base; XPointer
shorthand ids, element() navigation from an id and from the root, and
xpointer() including the classic per-step predicate semantic (//para[1]
selects every parent-first para; (//para)[1] the document-first), which the
corpus now locks explicitly.
Part of #105 (Tier 2)
A complex type derived by restriction is checked at schema compile to accept
a structural subset of its base, closing the declared XSD 1.0 spec gap.
ParticleRestriction covers element name matching, element-vs-wildcard
admission, wildcard narrowing, occurrence-range subsumption, and the group
recursions: sequence/all order-preserving with emptiable skips (Recurse),
choice mapping (RecurseLax), sequence-into-choice (MapAndSum, approximated by
per-particle checks, a documented simplification), sequence-into-all
(RecurseUnordered), and RecurseAsIfGroup for a lone element against a group.
An unfaithful restriction throws SchemaError.invalidRestriction(type:base:
reason:). All 834 pre-existing tests pass unchanged (no false positives on
the existing restriction-using schemas); 7 new tests cover the accept and
reject paths plus a conformance-harness pair.
Closes#117
Extract PathKey.steps(forChildNames:) as the single path construction behind
the validator walk, the XSD content validator, and the identity validator,
replacing three identical implementations, so located errors can never
disagree on a path. Add two differentials locking the duplicated engines
together: equivalent DTD and XSD content models (sequence, choice, *, +, ?,
nested choice) must agree on every child sequence across the positional
matcher and the Thompson NFA; and the pull (EventReader) and push
(PushScanner) tokenizers must yield the same event stream for the same
document, even with the push side fed one character at a time (adjacent text
events coalesced, since chunk boundaries legitimately split text). Both
pairs agree on everything today; the differentials guard them.
Closes#115
The whole-buffer ISO-2022-JP, GB18030, and EUC-TW decoders index the input
ArraySlice directly instead of materializing an Array first (one avoidable
O(n) allocation per decode each); declaredEncoding's subsequence scan uses
elementsEqual instead of allocating an Array per candidate offset. Document
the two analysis-flagged bounds: entity expansion is doubly bounded (the
character budget caps output; the visiting set caps recursion depth at the
number of declared entities), and the RELAX NG derivative engine carries no
memoization by design (the simplifying constructors keep real grammars
small; adversarial nested-choice grammars can grow the residual, so treat
untrusted schemas accordingly). Behavior unchanged, all suites green.
Closes#118
Deepen the two thin coverage zones (#119) through the validation-driven
harness. Serializer options matrix (10 cases): self-close on/off, CDATA kept
vs as-text, asciiOnly hex character references, round-trip carriage-return
escaping, declaration with standalone, exact pretty-print shape, attribute
escapes, and the Latin-1 output-encoding declaration via the byte path. HTML
depth (8 cases): the adoption agency's furthest-block and deep-misnesting
paths, foster-parented table text (with the implied tbody per the HTML5
algorithm), template content nesting through the document parser, and
raw-text script, implied dt/dd and table-cell closes, and comments through
the fragment parser. All conform; the one initial mismatch was a wrong
expectation in the test (the implied tbody), not a code divergence.
Closes#119
Lock the strict and recovering parse paths together: recovery synthesizes
events through machinery the strict path never exercises, so a 21-document
corpus of well-formed input (the C14N conformance inputs plus nesting,
mixed content, attribute quoting/escapes, namespaces and redeclaration,
entities, character references, prolog, comments, CDATA, xml:space) asserts
that PureXML.read produces the identical Node tree as PureXML.parse with no
diagnostics, and that readTree's editor tree projects to the same node. Both
paths agree on every document; the differential now guards them.
Closes#116
Document the weak-parent ownership trap at the public entry points: parseTree
and readTree now state that a tree (or ownerDocument) must be held while using
any node inside it, that TreeNode has no internal synchronization (confine to
one thread or actor), and that the immutable Sendable Node projection is the
way across concurrency boundaries; the TreeNode class doc carries the same
contract. QualifiedName's malformed-colon fallback (':local', 'prefix:', '::'
stay whole and unprefixed), already implemented, is now documented on init
and locked by a test.
Closes#114
Node-set ordering previously recomputed both nodes' root paths on every
precedes comparison (treePath is O(depth x siblings), so sorts were
O(n log n x depth)). Add decorate-sort-undecorate helpers,
sortedByDocumentOrder() and firstInDocumentOrder(), that compute each node's
order key once, and switch the evaluator's order-and-dedup, the EXSLT set
functions, string-value coercion, and NodeFunctions' target lookup to them.
set:leading/trailing also decorate once, reusing the keys for the pivot
filter and the final sort. precedes stays for pairwise use, sharing the
key comparator (Node.ordered). Behavior unchanged, all suites green.
Closes#113
A selector or field XPath that fails to compile is now reported once up
front as a located error (identity constraint 'k': invalid selector/field
XPath '...'), instead of try? silently turning a schema author's typo into
a disabled constraint. The per-run query cache from #112 is the choke point:
compileErrors() probes every constraint's selector and fields in
deterministic order, the failures cache as nil, and the walk afterwards
treats those constraints as matching nothing, so the rest of the document
still validates. Tests cover a malformed field and a malformed selector.
Closes#111
XSLT: template selection resolves each match pattern once per transform via
a per-run MatchCache (the pattern's node set is computed once over the source
tree; selection becomes an identity lookup), replacing per-(node, template)
recompilation and full-tree re-walks. XSD identity validation: a per-run query
cache compiles each selector/field XPath once instead of at every visited
element. Schematron was verified to already store compiled queries. Behavior
unchanged (all suites green); a multi-template shape test over 300 nodes
guards the optimized path. MatchCache lives in its own file to keep the
transformer under the length cap.
Closes#112
An xsi:type naming an undeclared type is now a located error in both the
tree validator (validateChild) and the streaming driver, instead of a silent
fallback to the declared type. typeReference chains resolve through one
shared resolver (resolveReference, with a visited set), replacing the inline
recursion, the 64-hop cap, and the completions guard: a circular chain now
reports 'circular type reference' instead of silently truncating, and the
streaming shallow check no longer recurses without bound on a cyclic table
(a latent crash). Tests cover unknown xsi:type (tree + streaming agree),
the circular chain (tree + shallow), the resolver's three outcomes, and a
conformance-harness case pair.
Closes#110
Grow the Tier-2 conformance corpus with 14 DTD content-model cases (sequence
order/completeness, choice exclusivity, the *, +, ? occurrence indicators,
#PCDATA, mixed content, EMPTY, and ANY) validated against the internal subset
and checked through the validation-driven harness. Own suite.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 13 XPath navigation cases (descendant
and all-element counts, positional and last() predicates, ancestor and
ancestor-or-self axes, following/preceding-sibling, the attribute axis, an
attribute predicate, union, the parent abbreviation, and position()) over a
fixed tree, checked through the validation-driven harness. Own suite.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 4 Canonical XML mode cases: exclusive
drops an unused namespace and renders one at its point of use; Canonical XML
2.0 drops whitespace-only text nodes and trims surrounding whitespace. Its own
suite (ConformanceCanonicalModesTests).
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 7 Schematron cases (assert satisfied/
violated, an inert non-matching context, numeric comparison, and report fired/
not-fired) checked through the validation-driven harness. Moved to its own
suite (ConformanceSchematronTests) so the main corpus stays under the
type-body cap.
Part of #105 (Tier 2)
Rule-canon audit (validation-rules.md / the OpenAPIKit idiom): the streaming
XSD validator was the one surface applying a bare method (validateShallow)
rather than a Validation<> value. Wrap it: ComplexValidator.shallowValidity is
a Validation<ResolvedElement, ComplexValidator> whose subject carries the
element with its already-resolved effective type (the contextual typing a
node-tree walk cannot express), and the streaming driver applies it via
apply(to:at:in:). Now every validation surface, DTD, XSD, RELAX NG, Schematron,
HTML, structural, conformance, and both streaming validators, routes through a
Validation value. Adds a direct unit test of the rule.
The fragment TreeBuilder gains the HTML5 active-formatting-list and reconstruct
step that the document builder already had: when a formatting element (b, i,
em, ...) is closed out of order, it is remembered and re-opened for the
following content. So HTML.parse now agrees with HTML.parseDocument and the
spec, e.g. <b><i></b>X</i> -> <b><i></i></b><i>X</i> and
<p>1<b>2<i>3</b>4</i>5</p> -> <p>1<b>2<i>3</i></b><i>4</i>5</p>. The conformance
corpus that surfaced this now checks the fragment path directly.
Closes#109
Grow the Tier-2 conformance corpus with 8 HTML5 cases (void elements, implied
li/p closing, tag-case normalization, the adoption agency with and without
trailing text, entity decoding, stray-end-tag) checked through the
validation-driven harness against the HTML5 parsing algorithm.
The corpus surfaced its first real divergence: HTML.parse (fragment) does not
reconstruct active formatting elements the way HTML.parseDocument (document,
spec-correct) does, so <b><i></b>X</i> yields <b><i></i></b>X instead of
<b><i></i></b><i>X</i>. The corpus uses the document path (spec-correct) and
the divergence is filed as #109.
Grow the Tier-2 conformance corpus with 10 XSLT cases (value-of, for-each,
if, choose/otherwise, sort, variable, format-number, concat, apply-templates,
and a named template with a parameter) transformed through the public API and
checked through the validation-driven harness against the spec output.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 12 RELAX NG pattern cases (text,
optional, oneOrMore, choice, ordered group, required attribute, interleave,
empty) validated through the public compact-syntax API and checked through
the validation-driven harness against the pattern semantics.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 13 XSD datatype and facet cases
(integer lexical space, minInclusive/maxInclusive range, length, pattern,
enumeration, boolean) validated end-to-end through the public schema API and
checked through the validation-driven harness against the spec verdict.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 16 XPath core-function cases (concat,
substring, string-length, normalize-space, translate, substring-before/after,
contains, starts-with, round/floor/ceiling, count, and the Infinity/NaN
division cases) checked through the same validation-driven harness against
their spec-mandated string results.
Part of #105 (Tier 2)
Begin Tier-2 conformance checking, expressed through the OpenAPIKit validation
idiom rather than an ad-hoc loop. ConformanceCase is a Validatable named
expectation; Conformance.matchesExpected is a Validation<ConformanceCase, Void>
that emits one located ValidationError per diverging case, so a suite reports
every failure at once with the case name as its coding path. Seeded with a
Canonical XML 1.0 corpus (attribute ordering, empty-element expansion, comment
removal, CDATA-to-escaped-text, PIs, namespace rendering) and a test proving a
divergence is reported as a located failure. Growing the corpora against the
official suites is the ongoing Tier-2 work.
Part of #105 (Tier 2)
Extend an identity error's coding path past the target element to the single
field the constraint names: an attribute (list/item[2]/@id) or a child element
(.../code), resolved from the field XPath. Multi-field or complex-path
constraints stay on the target. Also add a cross-scope keyref test confirming
a keyref resolves against a key declared on an ancestor element. Updates the
identity-path tests to the more precise locations.
Closes#104
Add depth tests confirming the existing DTD machinery handles parameter
entities within element-model and attribute-list declarations, deep (5-level)
PE composition, and an external subset that itself loads a further external
parameter entity through the resolver. The Tier-3 DTD PE / external-subset
depth item was already implemented; these lock it to full depth. (The XML
1.0 PE-reference spacing rule, leading/trailing space on DTD-content
inclusion, remains a known minor non-conformance.)
Part of #105 (Tier 3)
Add the EXSLT common/math/sets modules, dispatched by resolving a function's
prefix to its namespace (FunctionTable.call falls back to namespace lookup),
so any prefix bound to an EXSLT namespace works. math:min/max/highest/lowest/
abs/sqrt (with the NaN rule), set:distinct/difference/intersection/
has-same-node/leading/trailing (by node identity and document order), and
exsl:object-type. math:power and the transcendentals are omitted (they need a
C math library the Foundation-free target excludes); exsl:node-set awaits
result-tree-fragment support.
Part of #105 (Tier 3)
Add a canonical11 preset / mergeInheritedBase option. On a document subset,
the omitted-ancestor xml:base chain is merged into the apex by RFC 3986
reference resolution (vs 1.0 nearest-wins), the apex's own relative xml:base
is resolved against it, and xml:id is no longer inherited; xml:lang/xml:space
still inherit nearest. A self-contained RFC 3986 resolver (parse, path merge,
dot-segment removal) backs the merge, with tests covering the 1.1-vs-1.0
difference, dot segments, xml:id, and lang/space.
Part of #105 (Tier 3)
Add EUC-TW for the whole-buffer and streaming decoders: ASCII, a two-byte
plane-1 pair, and a 0x8E four-byte plane selector plus a pair. CNS planes 1
and 2, which carry essentially all real-world EUC-TW text, are vendored from
the ICU euc-tw-2014 mapping (private-use fallbacks dropped); planes 3-15 are a
documented follow-up. Tested for plane 1 (two-byte and four-byte forms), plane
2, and streaming. The streaming method moved to its own file to stay under the
type-body cap.
Closes#108
The streaming decoder previously selected an encoding only from a byte-order
mark or the UTF sniff, so its single-byte and CJK decode paths were latent. It
now reads the encoding from the XML declaration (the libxml2 behavior) when
there is no BOM, activating those paths over an incremental byte source. Add
the streaming form of ISO-2022-JP, carrying the stateful mode across reads
(extracted to its own file to stay under the type-body cap). A test asserts
streaming agrees with whole-buffer decode for a single-byte (ISO-8859-1), a
CJK (Shift_JIS), and an ISO-2022-JP document.
Part of #108, #105
Add Schema.Document.validate(streaming:) and ComplexValidator.validateShallow:
resolve each element's type from its parent's content model (and xsi:type /
typeReference) as it opens, then check its attributes and content-model
structure shallowly as it closes, against a shallow element synthesized from
the streamed child names. A child's deeper validity is checked when it closes,
so only the open-element stack is held. A differential test asserts streaming
agrees with the tree validator across valid and invalid documents (missing
attributes, bad simple values, missing/extra/misordered children). Wildcard
processContents and substitution-group resolution in the streaming form stay
on the tree validator. The streaming methods moved to ComplexValidatorStreaming
to keep ComplexValidator under the file/type-body caps.
Closes#107
Add the WHATWG iso-2022-jp decoder, the first stateful encoding: escape
sequences switch between ASCII, JIS X 0201 Roman (yen, overline), JIS X 0201
katakana, and the JIS X 0208 plane, resolved through the shared jis0208 table
(no new data). Tested across all four modes, with the JIS X 0208 kanji
cross-checked against the EUC-JP decoder (same table, same pointer). EUC-TW and
the streaming form of ISO-2022-JP remain.
Part of #108, #105
Add RelaxNG.validate(streaming:), driving the Brzozowski derivative directly
from the parse-event stream (the libxml2 xmlTextReader model). The derivative
is already incremental: the after-threaded pattern chain is the validation
stack, so only the residual pattern and a light per-element frame (for the
childrenDeriv ignorable-whitespace rule) are retained, never the tree. A
differential test asserts streaming agrees with the trusted tree engine across
valid, invalid, whitespace, and mixed-content documents. XSD streaming remains.
Part of #107, #105
Complete the DTD streaming validator with document-scoped ID/IDREF checks,
held in bounded sets (the identifier values, not the tree): a duplicate ID
is reported at its repeat occurrence, and forward IDREFs resolve at the end
of the stream via finish(). The streaming-vs-tree agreement test now covers
a duplicate ID as well. DTD streaming validation (content, attributes,
ID/IDREF) is now complete; RELAX NG and XSD streaming remain.
Part of #107, #105
Add PureXML.Validation.StreamingDTDValidator and PureXML.validate(streaming:
dtd:), validating a document against a DTD while it is pulled event by event
(the libxml2 xmlTextReader model), holding only the open-element stack rather
than the whole tree. Each element's content model and attributes are checked
as it closes by applying the exact per-element DTD rules to a shallow element
synthesized from the streamed child names, so streaming and tree validation
agree (a test asserts matching reasons). Errors carry an xmlGetNodePath-style
coding path. RELAX NG/XSD streaming and ID/IDREF streaming remain on #107.
Part of #107, #105
Extend ByteEncoder to the multi-byte CJK encodings (Shift-JIS, EUC-JP,
EUC-KR, GBK, GB18030, Big5). Each inverse is built by running the existing,
tested whole-buffer decoder over every valid byte sequence and recording
the shortest sequence per scalar, so the encoder is by construction the
exact inverse of the decoder, with no new data or separate encode tables.
GB18030's astral range uses the four-byte pointer formula, so every scalar
encodes. Round-trip tests cover Japanese, Korean, Chinese, and a GB18030
astral character.
Closes#106
Add PureXML.Parsing.ByteEncoder, the inverse of ByteDecoder, and a
Serializer.serialize(_:encoding:) / PureXML.serialize(_:encoding:) that
emits bytes in a chosen encoding (the libxml2 save-with-encoding model).
Covers the Unicode transformation formats and the single-byte legacy
families, built by inverting the existing vendored to-Unicode tables, so
no new data is introduced. The declaration carries the canonical encoding
name, a UTF-16/32 stream is preceded by a byte-order mark, and any
unrepresentable scalar becomes a decimal numeric character reference, so
output round-trips. CJK output encodings remain (tracked on #106).
Part of #106, #105
The error-array forms of && and || were unit-tested; the Bool predicate
forms were not. Add a pure-function test so every combinator overload in
the algebra is exercised, completing the framework's combinator coverage.
The identity-constraint location walk handles arbitrary selector depth, but
the existing assertions only exercised a single relative step. Add a nested
selector (group/item) whose duplicate sits in the second group, so the
offender is a grandchild and an intermediate level needs a sibling index:
list/group[2]/item. Locks the multi-level path construction.
Refine the identity-constraint path: rather than locating at the element
that declares the key, walk from the constraint-bearing element down to the
selected target and report the actual offender, so a duplicate key renders
"at path: list/item[2]" and a dangling keyref "at path: orders/line".
Also locate the "no element declaration for the root" XSD error at the root
element, matching the namespace check beside it that already did so. Lock
both with coding-path assertions in the identity tests.
The IdentityValidator emitted every key/keyref/unique error with an empty
coding path (at: []), so each rendered "at root of document" regardless of
which element violated the constraint. Thread a [PathKey] through the walk,
the way the content-model validator already does, so the failures carry the
constraining element's path. This restores the validation framework's
path-carrying contract for the one rule that was bypassing it.
Add the WHATWG big5 decoder over the vendored index-big5 (18,590 entries),
for both the whole-buffer and streaming decoders. A lead in 0x81-0xFE plus
a trail in 0x40-0x7E or 0xA1-0xFE resolves through the index, covering
Traditional Chinese and the HKSCS extensions including the astral CJK
ideographs. Four pointers decode to a base letter plus a combining mark.
Selected by the big5 / big5-hkscs declaration names.
Closes#103
Extend the GBK two-byte subset to full GB18030: a lead byte followed by
a digit opens a four-byte form whose pointer is mapped through the WHATWG
index-gb18030-ranges linear ranges, reaching the supplementary planes.
The 207 ranges are vendored verbatim. Wired into the whole-buffer and
streaming decoders and the encoding-name table (gb18030/gb-18030).
Closes#102
