EventReader.next(), both initializers, and the Event alias are public
(an internal typealias in a split file had been shadowing the public
Parsing.Event); a new PureXMLPublicAPITests target compiles without
testable import so facade gaps fail CI. The large-corpus benchmarks
exposed document-order sorting as quadratic on flat fan-outs; the fix's
first attempt was itself worse (an eager sibling table built per
predicate evaluation), caught by sampling the live run, and replaced
with a lazy table: single-node sets never touch order keys and a
parent's table builds on its second lookup. XPath at 42 MB drops
27.1 s to 1.57 s (190x to 11.7x); at 210 MB PureXML completes in 9.4 s
the query libxml2 refuses at its 10M node-set cap (#143 tracks our
configurable budget). Benchmark harness: size series to 210 MB, CORPUS
mode for real files (NASA, SwissProt measured), SAX stage,
refusal-aware table, NULL-safe C driver. All conformance suites,
Linux, and WASM verified. Closes#142.
String input copies its UTF-8 once into owned storage (an unsafe
pointer encapsulated in a class lifetime, never escaping the parser)
and scalars decode straight off the pointer, eliminating the
per-character closure chain; ASCII literals match and consume at the
byte level; and plain-ASCII character data scans as a single byte run
with one String built per run. Parse drops 32 percent cumulative
(0.49 s to 0.34 s on the 4.2 MB corpus, libxml2 ratio 30x to 20x).
The W3C suite caught one boundary (a run starting with '>' could
swallow the ]]> check after the character path consumed ]]); the run
scanner leaves a leading '>' to the character path, pinned as a unit
regression. All conformance suites, Linux, and WASM verified.
An include or import cycle recursed without bound and crashed the
host with a stack overflow, reachable from stylesheet data. The
compile chain tracks the active load chain by resolved href and drops
re-entry, while diamond composition keeps loading both copies with
their own import precedences per 2.6.2. id() builds a per-document
index on first use instead of re-walking the document per call,
matching key(). Cycle and diamond regression tests added.
PureXML.XSLT.transform accepts parameters overriding xsl:param globals
by name; xsl:variable globals are never overridable, with parameter
names recorded across imports and includes. The conformance runner
honors the xalan harness contract, reading <case>.param files (CRLF
is one Character in Swift, so the runner splits on newline graphemes,
which the param files exposed). Closes#141. Xalan baseline
128 -> 127.
The XSLT layer's id() follows the XPath definition exactly: only
attributes the source's DTD declares as type ID identify elements
(internal or external subset, via the transform's resolver), and a
document without ID declarations yields the empty set. generate-id()
produces non-empty, distinct identifiers for attribute and namespace
nodes through their identity hash. Lookup implementations move to
XSLTLookups.swift for the file caps. Xalan baseline 131 -> 129.
A top-level variable evaluates with every previously evaluated global
in scope, so referencing an earlier top-level param works, directly or
via call-template; same-name globals resolve by import precedence with
the last fold winning, so a later sibling import beats an earlier one;
the stylesheet fold uniformly favors later contributions, matching the
post-order precedence model; and the XPath compiler accepts NCName:*
name tests, matching any name in the prefix's namespace at the -0.25
default priority, so @ped:* outranks @*. Xalan baseline 144 -> 134
(variable 10 -> 5, conflictres 8 -> 5).
Import precedences are assigned in post-order over the import tree, so
sibling imports are distinct and every unit sits above its own
imports; apply-imports searches only the current template's
stylesheet's own import subtree per 5.6, reverting to the built-in
rule when that unit imports nothing; an included stylesheet's
declarations join the including unit at its precedence while its
imports join the import list; and include/import hrefs resolve against
the including stylesheet's URI. Included documents' roots stay
retained until their declarations are parsed (tree parents are weak,
so the in-scope xmlns chain could vanish mid-compile). Xalan baseline
159 -> 144 (reluri 11 -> 3, impincl 9 -> 2).
Same-name xsl:attribute-set declarations merge as ordered definitions
per 7.1.4, each expanding its used sets before its own attributes with
later definitions winning; xsl:copy carries the source element's
in-scope namespace nodes per 7.5; the namespace fixup lets a carried
attribute prefix shadow an inherited binding unless the element's own
name relies on it; the html method minimizes boolean attributes whose
value repeats their name; and external DTDs and entities in the
stylesheet or source resolve through the transform's document loader.
Four unit tests pin the behaviors. Xalan baseline 174 -> 159
(attribset 13 -> 4, copy 12 -> 7).
format-number of a value at or beyond 2^53 ran an unguarded
Double-to-Int conversion, a fatal trap reachable from user data; a
fraction picture beyond 15 places overflowed the scale conversion the
same way; and xsl:number value= beyond 2^53 trapped identically.
Digit extraction clamps fraction places to 15 (the 2^53 exactness
bound), pads back to the picture's minimum, renders out-of-range
integers through the exponent expander, and xsl:number falls back to
plain number rendering outside [1, 2^53). Sort-key rows always carry
one key per sort. Regression tests cover all three trap paths.
Raw-output marker resolution and 16.4 stripping run only when the
stylesheet mentions disable-output-escaping, so source text containing
U+E000/U+E001 survives every other transform untouched, and the text
method removes markers without unescaping content that was never
escaped. The pattern-matching function table is built once per
transform instead of per (template, node) membership test, and a dead
variable left in the numbering engine is gone. Eight focused unit
tests cover d-o-e (with the fragment rule and marker safety), CDATA
terminator splitting, simplified syntax, the html default method with
META injection, per-document key(), and attribute-start axes, so the
burn-down behaviors are CI-visible without the opt-in corpus.
The picture parses into affixes, number part, and the negative
subpattern: literal prefixes and suffixes pass through, the negative
subpattern's affixes apply (with the default minus when it has no
distinguishing ones), per-mille and percent multiply and stay literal,
the grouping size derives from the pattern's last grouping separator,
colliding decimal/grouping separators read as grouping, and U+00A4
renders as the en-US currency symbol. Digit extraction now uses one
scaled rounding and integer arithmetic, eliminating per-digit float
drift. Xalan numberformat 30 -> 1; baseline 475 -> 446.
xsl:number implements all of XSLT 1.0 section 7.7: level
single/multiple/any with count and from matched as patterns through the
match cache, the value expression with the below-one bypass, and the
format-token engine (punctuation, separator and token reuse, zero
padding, alphabetic/roman/Greek sequences, digit grouping). An
unmatched single/multiple count renders empty; level=any always yields
a number. document() now caches per transform so repeated loads of the
same URI return the identical document and generate-id() stays unique
against live nodes (the idkey49 flake). Xalan baseline 557 -> 475
(numbering 84 -> 7).
xsl:attribute/xsl:element honor the namespace attribute and resolve
prefixed names against the instruction's in-scope stylesheet
declarations (captured at parse); xsl:copy applies use-attribute-sets;
and a namespace-fixup pass before serialization declares every
namespace a created name carries, reusing same-URI bindings or
generating ns0.. prefixes, and undeclares an inherited default that
would capture an unqualified name. Xalan baseline 604 to 557.
The Apache xalan-test conf corpus (1690 gold-bearing cases, 36
categories) runs opt-in via XALAN_TS_ROOT with normalized comparison:
canonical XML when both sides parse, collapsed whitespace otherwise.
First measurement passes 1086; the 604 known failures live in
Tests/Fixtures/xalan-baseline.txt, asserted exactly in both directions.
Top classes: namespace handling, xsl:number, axes corners, output
options, attribute sets, id()/key().
A RELAX NG schema document is validated before pattern interpretation
in three layers: the section-3 grammar check (vocabulary, attributes,
arity, text, QNames, datatypeLibrary URIs, hrefs, 4.16 excepts, xmlns
prohibitions), compiler checks (4.17 combine rules, unresolved refs,
parentRef scope, startless grammars, include/externalRef cycles and
overrides, grammar-checked loaded documents, unknown datatypes,
built-in-library params), and the 4.19/7.1-7.4 restrictions over the
compiled algebra after 4.20/4.21 normalization (prohibited paths,
content types, duplicate attributes via representative-name overlap,
infinite-class attributes outside oneOrMore, interleave constraints),
with recursion detection deliberately before normalization. Incorrect
schemas: 213 compiling down to 7, all the pre-Fifth-Edition name-class
class (count asserted). Two leniency tests updated to spec behavior.
Closes#131
A value of type QName compiles to its (namespace, local) pair via the
schema's xmlns/ns scope; the derivative engine threads the instance
element's in-scope bindings to value comparison in both the tree walk
and the streaming state (frames carry their bindings), so instance
prefixes and the default namespace resolve against the instance
document and unbound prefixes match nothing. Streaming also gains the
tree walk's empty-element rule. The spec suite's instance level is
fully clean: valid 289/289, invalid 291/291; only the asserted
schema-correctness class remains.
DocumentType.validityFindings carries ValidityFinding values (reason
plus subject) instead of strings, DTDSchema.declarationErrors holds
ready ValidationErrors, and every declaration-level finding renders at
its declaration's coding path (x, r/@a) instead of at the document
root, matching the XSD consistency-errors precedent. Texts unchanged.
James Clark's spectest.xml runs opt-in via RNG_TS_ROOT: nested
testSuite scopes thread resource/dir definitions into the compiler
loader, and each case asserts schema compile/reject plus per-instance
accept/reject. First measurement carried as exact baselines: all
correct schemas compile, valid 233/289, invalid 277/291; schema
correctness is not yet validated (all 213 incorrect schemas compile,
count asserted). Burn-down classes recorded: grammar-level schema
validation with the 4.16-4.18 restrictions, foreign-namespace
stripping, xml:base-aware resource resolution, and the section 6
compatibility datatypes.
The six worked examples of the C14N 1.0 Recommendation's section 3 run
as input/expected-output tests. They exposed and fixed: top-level
siblings now separated by single line feeds in the canonical form;
superfluous empty default-namespace declarations dropped in inclusive
mode (xmlns="" renders only when undeclaring a rendered non-empty
default); and scalar-level escape loops, so a CR clustered with a
following LF still escapes to 
. Escaping split to its own file
for the length caps.
Closes#132
The reader buffers one Unicode scalar per element, splitting
multi-scalar graphemes from any source, so combining marks cannot merge
with delimiters and the Fifth Edition's scalar-defined name productions
classify correctly; the replacement-text, content-model, and ATTLIST
sub-scanners and the entity-literal validator follow, while grapheme
clusters reassemble wherever scanned text accumulates. Clears the last
eduni baseline, grows the asserted IBM pre-5e class to 300, and with
the xmltest manifest's own EDITION exclusions for 140/141 empties the
xmltest baseline: every xmlconf case applicable to a Fifth Edition,
namespace-aware processor passes.
Closes#135
Limits(strictInternalSubset: true) rejects conditional sections and
parameter-entity references inside markup declarations in the internal
subset, the two feature classes XML 1.0 reserves for the external
subset. DeclSep references and quoted attribute defaults (AttValue)
stay legal; defaults unchanged. The xmltest runner opts in, leaving
case 141 (pre-Fifth-Edition name classes) as its only deviation.
Closes#128
A relative system identifier declared inside an external entity
resolves against that entity's own URI (RFC 3986, the C14N resolver):
ExternalID gains base and resolvedSystemID, the scanner threads the
current entity's URI through external-subset and PE scans, and
resolvers receive the merged identifier. Clears eduni E18; the eduni
baseline now holds only the grapheme-lexing class (#135).
Closes#138
A declared 16/32-bit encoding over BOM-less 8-bit bytes is rejected; a
BOM followed by a declaration outside its family is rejected, reading
the declaration through the BOM's own byte pattern; and an external
entity whose text declaration names a higher XML version than the
document is refused (errata E38). The eduni not-wf baseline is empty.
Reader and decoder helpers split for the length caps.
Closes#137
The namespace layer rejects malformed qualified names, declarations of
the xmlns prefix, misbindings of the xml prefix and either reserved
namespace name (including as the default namespace), and prefix
undeclaration; attributes must be distinct by expanded name; PI targets
and entity/notation names are NCNames; and DTD-typed xmlns values bind
their normalized form. Clears all 18 namespace cases from the eduni
baseline; xmltest valid/sa/012 joins the documented deviations per the
suite's own NAMESPACE='no' flag.
Closes#136
The eduni sections run with exact classified baselines. Fixes: 3.3.3
CDATA attribute-value normalization (literal whitespace to space before
reference decoding; char-ref whitespace survives; tokenized collapse on
SPACE only); production-68 lenient undeclared entities in content and
attribute values (validity finding, reference kept literal, nothing
fetched); errata E15 (references/comments/PIs/whitespace in EMPTY,
char-ref whitespace as data in element content, double-escape shape
included); errata E2 duplicate enumeration tokens; errata E14
declarations completed inside PE replacements (VC: Proper
Declaration/PE Nesting); NCName ID values; namespace declarations
subject to the every-attribute-declared rule. Reader validity machinery
split to EventReaderValidity for caps.
Closes#127
All 13 cases of the xmlconf japanese section decode through the
package byte decoder (UTF-8, UTF-16 both orders, Shift_JIS, EUC-JP,
ISO-2022-JP), parse, and DTD-validate cleanly with zero deviations.
The optional encodings the suite marks TYPE='error' are all supported,
so every case is held to the must-parse bar.
Closes#126
The IBM section (1094 cases) runs clean to two documented classes:
279 pre-Fifth-Edition character-class cases (P85-89, count asserted
exactly) and 2 PE-in-internal-subset feature cases. Real gaps closed:
a document needs a root element (production 1); entity references in
literals must be Names or character references; WFC: No Recursion
checked at declaration over the general/parameter reference graph;
undeclared PE references split into WFC (standalone, or no external
declarations possible) versus deferred validity finding, also for
attribute-default references; refused external PEs are neither; and
groups split across PE replacements report VC: Proper Group/PE
Nesting. Entity machinery split to DoctypeScannerEntities for caps.
Closes#125
The scanner records declaration provenance (internal-subset entities,
element models, attribute lists); the validator reports the three
standalone='yes' dependencies on the external subset: a default that
would be supplied, a non-CDATA normalization that would change a value,
and whitespace in externally-declared element content. The parser hides
externally-declared entities from standalone documents, so referencing
one reports as undeclared (WFC: Entity Declared, 2.9). The Sun section
is fully clean: not-wf 56/56, valid 28/28, invalid 74/74, no baseline.
Closes#134Closes#124
VC Root Element Type (root matches the DOCTYPE name), VC Unique
Element Type Declaration, VC No Duplicate Types in mixed content, VC
One ID per Element Type, VC Notation Attributes and Notation Declared,
VC Attribute Default Legal plus ID Attribute Default, lexical Name
checks for the ID family, and CDATA as character data in element
content. Strict mode adds VC Attribute Value Type (every attribute
declared; namespace declarations exempt). Declaration findings are
computed once at schema build and reported at the document root. Sun
invalid coverage moves from 43/74 to 62/74; the twelve remaining are
all the standalone='yes' family.
A general entity whose replacement text contains markup (directly or
transitively) is reparsed as content at the reference (4.4.2 Included):
the replacement is WFC-validated, cycle-checked, and budgeted, then
pushed back into the reader stream, so its elements become elements in
the tree and the pull-event stream rather than escaped character data.
Character references in internal entity literals now expand at
declaration time for general entities too (4.4.5), making char-ref
markup splice and the Appendix D double escape render as a literal
ampersand; the reference-time WFC validates remaining references
instead of re-expanding. Text-only entities keep the coalescing
single-pass path. Sun valid is 28/28.
Closes#133
Tokenized (non-CDATA) attribute values normalize before validity
checks, so enumerations, NOTATION lists, and the ID/IDREF table see
collapsed values. External parsed entities lose their leading text
declaration (validated: version optional, encoding required, never
standalone) instead of carrying it into replacement text. Character
references in PE literals expand at declaration time, making the
Appendix D '%zz;' example parse. A PI target inside the DTD subset
must be separated from its data. Sun baselines tighten to not-wf 55/56
and valid 26/28; the two remaining valid failures share one cause
(replacement text with markup is included as character data, filed
separately).
The first runner to exercise the validity layer: not-wf must be
rejected (54/56), valid must parse and DTD-validate clean (19/28),
invalid must parse but report at least one validity error (43/74).
Deviation baselines are exact and annotated with their causes, the
burn-down list for #124: tokenized attribute-value normalization,
external-entity text declarations, declaration-time char-ref expansion
in PE literals, standalone VCs, ID/IDREF corners, REQUIRED enforcement,
per-type attribute VCs, and the root/DOCTYPE name match.
The OASIS/NIST section of the W3C conformance suite runs via a new
XMLCONF_ROOT-gated runner driven by the section's own manifest (parsed
with PureXML): 247/247 not-wf rejected, all 100 well-formed cases parse,
zero deviations. Closing its 24 gaps makes the DTD subset grammar
strict: exact PEReference shape, strict declaration names and keywords,
NDATA and notation tails, no bare '%' in entity values, case-sensitive
balanced conditional sections, junk between declarations rejected,
external-subset violations now propagate (with text-declaration
support, production 77), and a PI target must be separated from its
data. The xmltest baseline tightens to 181/186: CDATA in the internal
subset (107) is now rejected, five documented deviations remain.
Closes#123
The derivation final check and Particle Valid (Restriction) move from
fail-fast throws at schema compile into composable Validation rules
(finalRespected, restrictionsAreSubsets) over per-type SchemaTypeFact
subjects, so a schema with several problems reports every finding at
once. SchemaError gains .inconsistent([String]) carrying all findings
(texts unchanged) and drops .finalViolation/.invalidRestriction.
checkRedefine, checkAllGroups, and notASchema stay throws: they examine
the raw schema source, the compilation analog of well-formedness.
Add five well-formedness checks the W3C xmltest suite surfaced: '--' inside
comments (also rejecting the ---> ending), raw '<' in attribute values, the
literal ]]> sequence in character data, attributes not separated by
whitespace, and characters outside the XML Char production in content, each
a located ParseError. The attribute scanners move to EventReaderAttributes
to stay under the type-body cap.
Add the opt-in W3C xmltest runner (XMLTS_ROOT-gated; the suite is never
vendored, James Clark's license permits redistribution only as the unmodified
archive; Foundation is used in this one test file for file access only).
Baseline: valid/sa 118/119, not-wf/sa 93/186 rejected (up from 80), the
remaining 93 carried as an exact known-deviations baseline so progress and
regressions are both visible. The two remaining leniency classes (internal-
subset DTD grammar, entity replacement-text well-formedness) are #120.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 7 OASIS catalog cases through the
harness: exact system/public/uri entries, rewriteSystem and rewriteURI prefix
matching, and the unresolved outcomes for unmatched identifiers. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 17 XSD regex cases through the
pattern-facet path: implicit whole-string anchoring, the character-class
escapes (backslash-d/w/s), Unicode category and negated-category escapes,
quantifier ranges (exact, bounded, open), alternation, grouping, negated
character classes, and class subtraction. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 10 cases through the harness:
XInclude substitution, parse=text inclusion with escaping, xi:fallback on a
missing href, and relative href resolution against the base; XPointer
shorthand ids, element() navigation from an id and from the root, and
xpointer() including the classic per-step predicate semantic (//para[1]
selects every parent-first para; (//para)[1] the document-first), which the
corpus now locks explicitly.
Part of #105 (Tier 2)
A complex type derived by restriction is checked at schema compile to accept
a structural subset of its base, closing the declared XSD 1.0 spec gap.
ParticleRestriction covers element name matching, element-vs-wildcard
admission, wildcard narrowing, occurrence-range subsumption, and the group
recursions: sequence/all order-preserving with emptiable skips (Recurse),
choice mapping (RecurseLax), sequence-into-choice (MapAndSum, approximated by
per-particle checks, a documented simplification), sequence-into-all
(RecurseUnordered), and RecurseAsIfGroup for a lone element against a group.
An unfaithful restriction throws SchemaError.invalidRestriction(type:base:
reason:). All 834 pre-existing tests pass unchanged (no false positives on
the existing restriction-using schemas); 7 new tests cover the accept and
reject paths plus a conformance-harness pair.
Closes#117
Extract PathKey.steps(forChildNames:) as the single path construction behind
the validator walk, the XSD content validator, and the identity validator,
replacing three identical implementations, so located errors can never
disagree on a path. Add two differentials locking the duplicated engines
together: equivalent DTD and XSD content models (sequence, choice, *, +, ?,
nested choice) must agree on every child sequence across the positional
matcher and the Thompson NFA; and the pull (EventReader) and push
(PushScanner) tokenizers must yield the same event stream for the same
document, even with the push side fed one character at a time (adjacent text
events coalesced, since chunk boundaries legitimately split text). Both
pairs agree on everything today; the differentials guard them.
Closes#115
Deepen the two thin coverage zones (#119) through the validation-driven
harness. Serializer options matrix (10 cases): self-close on/off, CDATA kept
vs as-text, asciiOnly hex character references, round-trip carriage-return
escaping, declaration with standalone, exact pretty-print shape, attribute
escapes, and the Latin-1 output-encoding declaration via the byte path. HTML
depth (8 cases): the adoption agency's furthest-block and deep-misnesting
paths, foster-parented table text (with the implied tbody per the HTML5
algorithm), template content nesting through the document parser, and
raw-text script, implied dt/dd and table-cell closes, and comments through
the fragment parser. All conform; the one initial mismatch was a wrong
expectation in the test (the implied tbody), not a code divergence.
Closes#119
Lock the strict and recovering parse paths together: recovery synthesizes
events through machinery the strict path never exercises, so a 21-document
corpus of well-formed input (the C14N conformance inputs plus nesting,
mixed content, attribute quoting/escapes, namespaces and redeclaration,
entities, character references, prolog, comments, CDATA, xml:space) asserts
that PureXML.read produces the identical Node tree as PureXML.parse with no
diagnostics, and that readTree's editor tree projects to the same node. Both
paths agree on every document; the differential now guards them.
Closes#116
Document the weak-parent ownership trap at the public entry points: parseTree
and readTree now state that a tree (or ownerDocument) must be held while using
any node inside it, that TreeNode has no internal synchronization (confine to
one thread or actor), and that the immutable Sendable Node projection is the
way across concurrency boundaries; the TreeNode class doc carries the same
contract. QualifiedName's malformed-colon fallback (':local', 'prefix:', '::'
stay whole and unprefixed), already implemented, is now documented on init
and locked by a test.
Closes#114
A selector or field XPath that fails to compile is now reported once up
front as a located error (identity constraint 'k': invalid selector/field
XPath '...'), instead of try? silently turning a schema author's typo into
a disabled constraint. The per-run query cache from #112 is the choke point:
compileErrors() probes every constraint's selector and fields in
deterministic order, the failures cache as nil, and the walk afterwards
treats those constraints as matching nothing, so the rest of the document
still validates. Tests cover a malformed field and a malformed selector.
Closes#111
XSLT: template selection resolves each match pattern once per transform via
a per-run MatchCache (the pattern's node set is computed once over the source
tree; selection becomes an identity lookup), replacing per-(node, template)
recompilation and full-tree re-walks. XSD identity validation: a per-run query
cache compiles each selector/field XPath once instead of at every visited
element. Schematron was verified to already store compiled queries. Behavior
unchanged (all suites green); a multi-template shape test over 300 nodes
guards the optimized path. MatchCache lives in its own file to keep the
transformer under the length cap.
Closes#112
An xsi:type naming an undeclared type is now a located error in both the
tree validator (validateChild) and the streaming driver, instead of a silent
fallback to the declared type. typeReference chains resolve through one
shared resolver (resolveReference, with a visited set), replacing the inline
recursion, the 64-hop cap, and the completions guard: a circular chain now
reports 'circular type reference' instead of silently truncating, and the
streaming shallow check no longer recurses without bound on a cyclic table
(a latent crash). Tests cover unknown xsi:type (tree + streaming agree),
the circular chain (tree + shallow), the resolver's three outcomes, and a
conformance-harness case pair.
Closes#110
