format-number of a value at or beyond 2^53 ran an unguarded
Double-to-Int conversion, a fatal trap reachable from user data; a
fraction picture beyond 15 places overflowed the scale conversion the
same way; and xsl:number value= beyond 2^53 trapped identically.
Digit extraction clamps fraction places to 15 (the 2^53 exactness
bound), pads back to the picture's minimum, renders out-of-range
integers through the exponent expander, and xsl:number falls back to
plain number rendering outside [1, 2^53). Sort-key rows always carry
one key per sort. Regression tests cover all three trap paths.
Raw-output marker resolution and 16.4 stripping run only when the
stylesheet mentions disable-output-escaping, so source text containing
U+E000/U+E001 survives every other transform untouched, and the text
method removes markers without unescaping content that was never
escaped. The pattern-matching function table is built once per
transform instead of per (template, node) membership test, and a dead
variable left in the numbering engine is gone. Eight focused unit
tests cover d-o-e (with the fragment rule and marker safety), CDATA
terminator splitting, simplified syntax, the html default method with
META injection, per-document key(), and attribute-start axes, so the
burn-down behaviors are CI-visible without the opt-in corpus.
The picture parses into affixes, number part, and the negative
subpattern: literal prefixes and suffixes pass through, the negative
subpattern's affixes apply (with the default minus when it has no
distinguishing ones), per-mille and percent multiply and stay literal,
the grouping size derives from the pattern's last grouping separator,
colliding decimal/grouping separators read as grouping, and U+00A4
renders as the en-US currency symbol. Digit extraction now uses one
scaled rounding and integer arithmetic, eliminating per-digit float
drift. Xalan numberformat 30 -> 1; baseline 475 -> 446.
xsl:number implements all of XSLT 1.0 section 7.7: level
single/multiple/any with count and from matched as patterns through the
match cache, the value expression with the below-one bypass, and the
format-token engine (punctuation, separator and token reuse, zero
padding, alphabetic/roman/Greek sequences, digit grouping). An
unmatched single/multiple count renders empty; level=any always yields
a number. document() now caches per transform so repeated loads of the
same URI return the identical document and generate-id() stays unique
against live nodes (the idkey49 flake). Xalan baseline 557 -> 475
(numbering 84 -> 7).
xsl:attribute/xsl:element honor the namespace attribute and resolve
prefixed names against the instruction's in-scope stylesheet
declarations (captured at parse); xsl:copy applies use-attribute-sets;
and a namespace-fixup pass before serialization declares every
namespace a created name carries, reusing same-URI bindings or
generating ns0.. prefixes, and undeclares an inherited default that
would capture an unqualified name. Xalan baseline 604 to 557.
The Apache xalan-test conf corpus (1690 gold-bearing cases, 36
categories) runs opt-in via XALAN_TS_ROOT with normalized comparison:
canonical XML when both sides parse, collapsed whitespace otherwise.
First measurement passes 1086; the 604 known failures live in
Tests/Fixtures/xalan-baseline.txt, asserted exactly in both directions.
Top classes: namespace handling, xsl:number, axes corners, output
options, attribute sets, id()/key().
A RELAX NG schema document is validated before pattern interpretation
in three layers: the section-3 grammar check (vocabulary, attributes,
arity, text, QNames, datatypeLibrary URIs, hrefs, 4.16 excepts, xmlns
prohibitions), compiler checks (4.17 combine rules, unresolved refs,
parentRef scope, startless grammars, include/externalRef cycles and
overrides, grammar-checked loaded documents, unknown datatypes,
built-in-library params), and the 4.19/7.1-7.4 restrictions over the
compiled algebra after 4.20/4.21 normalization (prohibited paths,
content types, duplicate attributes via representative-name overlap,
infinite-class attributes outside oneOrMore, interleave constraints),
with recursion detection deliberately before normalization. Incorrect
schemas: 213 compiling down to 7, all the pre-Fifth-Edition name-class
class (count asserted). Two leniency tests updated to spec behavior.
Closes#131
A value of type QName compiles to its (namespace, local) pair via the
schema's xmlns/ns scope; the derivative engine threads the instance
element's in-scope bindings to value comparison in both the tree walk
and the streaming state (frames carry their bindings), so instance
prefixes and the default namespace resolve against the instance
document and unbound prefixes match nothing. Streaming also gains the
tree walk's empty-element rule. The spec suite's instance level is
fully clean: valid 289/289, invalid 291/291; only the asserted
schema-correctness class remains.
DocumentType.validityFindings carries ValidityFinding values (reason
plus subject) instead of strings, DTDSchema.declarationErrors holds
ready ValidationErrors, and every declaration-level finding renders at
its declaration's coding path (x, r/@a) instead of at the document
root, matching the XSD consistency-errors precedent. Texts unchanged.
James Clark's spectest.xml runs opt-in via RNG_TS_ROOT: nested
testSuite scopes thread resource/dir definitions into the compiler
loader, and each case asserts schema compile/reject plus per-instance
accept/reject. First measurement carried as exact baselines: all
correct schemas compile, valid 233/289, invalid 277/291; schema
correctness is not yet validated (all 213 incorrect schemas compile,
count asserted). Burn-down classes recorded: grammar-level schema
validation with the 4.16-4.18 restrictions, foreign-namespace
stripping, xml:base-aware resource resolution, and the section 6
compatibility datatypes.
The six worked examples of the C14N 1.0 Recommendation's section 3 run
as input/expected-output tests. They exposed and fixed: top-level
siblings now separated by single line feeds in the canonical form;
superfluous empty default-namespace declarations dropped in inclusive
mode (xmlns="" renders only when undeclaring a rendered non-empty
default); and scalar-level escape loops, so a CR clustered with a
following LF still escapes to 
. Escaping split to its own file
for the length caps.
Closes#132
The reader buffers one Unicode scalar per element, splitting
multi-scalar graphemes from any source, so combining marks cannot merge
with delimiters and the Fifth Edition's scalar-defined name productions
classify correctly; the replacement-text, content-model, and ATTLIST
sub-scanners and the entity-literal validator follow, while grapheme
clusters reassemble wherever scanned text accumulates. Clears the last
eduni baseline, grows the asserted IBM pre-5e class to 300, and with
the xmltest manifest's own EDITION exclusions for 140/141 empties the
xmltest baseline: every xmlconf case applicable to a Fifth Edition,
namespace-aware processor passes.
Closes#135
Limits(strictInternalSubset: true) rejects conditional sections and
parameter-entity references inside markup declarations in the internal
subset, the two feature classes XML 1.0 reserves for the external
subset. DeclSep references and quoted attribute defaults (AttValue)
stay legal; defaults unchanged. The xmltest runner opts in, leaving
case 141 (pre-Fifth-Edition name classes) as its only deviation.
Closes#128
A relative system identifier declared inside an external entity
resolves against that entity's own URI (RFC 3986, the C14N resolver):
ExternalID gains base and resolvedSystemID, the scanner threads the
current entity's URI through external-subset and PE scans, and
resolvers receive the merged identifier. Clears eduni E18; the eduni
baseline now holds only the grapheme-lexing class (#135).
Closes#138
A declared 16/32-bit encoding over BOM-less 8-bit bytes is rejected; a
BOM followed by a declaration outside its family is rejected, reading
the declaration through the BOM's own byte pattern; and an external
entity whose text declaration names a higher XML version than the
document is refused (errata E38). The eduni not-wf baseline is empty.
Reader and decoder helpers split for the length caps.
Closes#137
The namespace layer rejects malformed qualified names, declarations of
the xmlns prefix, misbindings of the xml prefix and either reserved
namespace name (including as the default namespace), and prefix
undeclaration; attributes must be distinct by expanded name; PI targets
and entity/notation names are NCNames; and DTD-typed xmlns values bind
their normalized form. Clears all 18 namespace cases from the eduni
baseline; xmltest valid/sa/012 joins the documented deviations per the
suite's own NAMESPACE='no' flag.
Closes#136
The eduni sections run with exact classified baselines. Fixes: 3.3.3
CDATA attribute-value normalization (literal whitespace to space before
reference decoding; char-ref whitespace survives; tokenized collapse on
SPACE only); production-68 lenient undeclared entities in content and
attribute values (validity finding, reference kept literal, nothing
fetched); errata E15 (references/comments/PIs/whitespace in EMPTY,
char-ref whitespace as data in element content, double-escape shape
included); errata E2 duplicate enumeration tokens; errata E14
declarations completed inside PE replacements (VC: Proper
Declaration/PE Nesting); NCName ID values; namespace declarations
subject to the every-attribute-declared rule. Reader validity machinery
split to EventReaderValidity for caps.
Closes#127
All 13 cases of the xmlconf japanese section decode through the
package byte decoder (UTF-8, UTF-16 both orders, Shift_JIS, EUC-JP,
ISO-2022-JP), parse, and DTD-validate cleanly with zero deviations.
The optional encodings the suite marks TYPE='error' are all supported,
so every case is held to the must-parse bar.
Closes#126
The IBM section (1094 cases) runs clean to two documented classes:
279 pre-Fifth-Edition character-class cases (P85-89, count asserted
exactly) and 2 PE-in-internal-subset feature cases. Real gaps closed:
a document needs a root element (production 1); entity references in
literals must be Names or character references; WFC: No Recursion
checked at declaration over the general/parameter reference graph;
undeclared PE references split into WFC (standalone, or no external
declarations possible) versus deferred validity finding, also for
attribute-default references; refused external PEs are neither; and
groups split across PE replacements report VC: Proper Group/PE
Nesting. Entity machinery split to DoctypeScannerEntities for caps.
Closes#125
The scanner records declaration provenance (internal-subset entities,
element models, attribute lists); the validator reports the three
standalone='yes' dependencies on the external subset: a default that
would be supplied, a non-CDATA normalization that would change a value,
and whitespace in externally-declared element content. The parser hides
externally-declared entities from standalone documents, so referencing
one reports as undeclared (WFC: Entity Declared, 2.9). The Sun section
is fully clean: not-wf 56/56, valid 28/28, invalid 74/74, no baseline.
Closes#134Closes#124
VC Root Element Type (root matches the DOCTYPE name), VC Unique
Element Type Declaration, VC No Duplicate Types in mixed content, VC
One ID per Element Type, VC Notation Attributes and Notation Declared,
VC Attribute Default Legal plus ID Attribute Default, lexical Name
checks for the ID family, and CDATA as character data in element
content. Strict mode adds VC Attribute Value Type (every attribute
declared; namespace declarations exempt). Declaration findings are
computed once at schema build and reported at the document root. Sun
invalid coverage moves from 43/74 to 62/74; the twelve remaining are
all the standalone='yes' family.
A general entity whose replacement text contains markup (directly or
transitively) is reparsed as content at the reference (4.4.2 Included):
the replacement is WFC-validated, cycle-checked, and budgeted, then
pushed back into the reader stream, so its elements become elements in
the tree and the pull-event stream rather than escaped character data.
Character references in internal entity literals now expand at
declaration time for general entities too (4.4.5), making char-ref
markup splice and the Appendix D double escape render as a literal
ampersand; the reference-time WFC validates remaining references
instead of re-expanding. Text-only entities keep the coalescing
single-pass path. Sun valid is 28/28.
Closes#133
Tokenized (non-CDATA) attribute values normalize before validity
checks, so enumerations, NOTATION lists, and the ID/IDREF table see
collapsed values. External parsed entities lose their leading text
declaration (validated: version optional, encoding required, never
standalone) instead of carrying it into replacement text. Character
references in PE literals expand at declaration time, making the
Appendix D '%zz;' example parse. A PI target inside the DTD subset
must be separated from its data. Sun baselines tighten to not-wf 55/56
and valid 26/28; the two remaining valid failures share one cause
(replacement text with markup is included as character data, filed
separately).
The first runner to exercise the validity layer: not-wf must be
rejected (54/56), valid must parse and DTD-validate clean (19/28),
invalid must parse but report at least one validity error (43/74).
Deviation baselines are exact and annotated with their causes, the
burn-down list for #124: tokenized attribute-value normalization,
external-entity text declarations, declaration-time char-ref expansion
in PE literals, standalone VCs, ID/IDREF corners, REQUIRED enforcement,
per-type attribute VCs, and the root/DOCTYPE name match.
The OASIS/NIST section of the W3C conformance suite runs via a new
XMLCONF_ROOT-gated runner driven by the section's own manifest (parsed
with PureXML): 247/247 not-wf rejected, all 100 well-formed cases parse,
zero deviations. Closing its 24 gaps makes the DTD subset grammar
strict: exact PEReference shape, strict declaration names and keywords,
NDATA and notation tails, no bare '%' in entity values, case-sensitive
balanced conditional sections, junk between declarations rejected,
external-subset violations now propagate (with text-declaration
support, production 77), and a PI target must be separated from its
data. The xmltest baseline tightens to 181/186: CDATA in the internal
subset (107) is now rejected, five documented deviations remain.
Closes#123
The derivation final check and Particle Valid (Restriction) move from
fail-fast throws at schema compile into composable Validation rules
(finalRespected, restrictionsAreSubsets) over per-type SchemaTypeFact
subjects, so a schema with several problems reports every finding at
once. SchemaError gains .inconsistent([String]) carrying all findings
(texts unchanged) and drops .finalViolation/.invalidRestriction.
checkRedefine, checkAllGroups, and notASchema stay throws: they examine
the raw schema source, the compilation analog of well-formedness.
Add five well-formedness checks the W3C xmltest suite surfaced: '--' inside
comments (also rejecting the ---> ending), raw '<' in attribute values, the
literal ]]> sequence in character data, attributes not separated by
whitespace, and characters outside the XML Char production in content, each
a located ParseError. The attribute scanners move to EventReaderAttributes
to stay under the type-body cap.
Add the opt-in W3C xmltest runner (XMLTS_ROOT-gated; the suite is never
vendored, James Clark's license permits redistribution only as the unmodified
archive; Foundation is used in this one test file for file access only).
Baseline: valid/sa 118/119, not-wf/sa 93/186 rejected (up from 80), the
remaining 93 carried as an exact known-deviations baseline so progress and
regressions are both visible. The two remaining leniency classes (internal-
subset DTD grammar, entity replacement-text well-formedness) are #120.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 7 OASIS catalog cases through the
harness: exact system/public/uri entries, rewriteSystem and rewriteURI prefix
matching, and the unresolved outcomes for unmatched identifiers. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 17 XSD regex cases through the
pattern-facet path: implicit whole-string anchoring, the character-class
escapes (backslash-d/w/s), Unicode category and negated-category escapes,
quantifier ranges (exact, bounded, open), alternation, grouping, negated
character classes, and class subtraction. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 10 cases through the harness:
XInclude substitution, parse=text inclusion with escaping, xi:fallback on a
missing href, and relative href resolution against the base; XPointer
shorthand ids, element() navigation from an id and from the root, and
xpointer() including the classic per-step predicate semantic (//para[1]
selects every parent-first para; (//para)[1] the document-first), which the
corpus now locks explicitly.
Part of #105 (Tier 2)
A complex type derived by restriction is checked at schema compile to accept
a structural subset of its base, closing the declared XSD 1.0 spec gap.
ParticleRestriction covers element name matching, element-vs-wildcard
admission, wildcard narrowing, occurrence-range subsumption, and the group
recursions: sequence/all order-preserving with emptiable skips (Recurse),
choice mapping (RecurseLax), sequence-into-choice (MapAndSum, approximated by
per-particle checks, a documented simplification), sequence-into-all
(RecurseUnordered), and RecurseAsIfGroup for a lone element against a group.
An unfaithful restriction throws SchemaError.invalidRestriction(type:base:
reason:). All 834 pre-existing tests pass unchanged (no false positives on
the existing restriction-using schemas); 7 new tests cover the accept and
reject paths plus a conformance-harness pair.
Closes#117
Extract PathKey.steps(forChildNames:) as the single path construction behind
the validator walk, the XSD content validator, and the identity validator,
replacing three identical implementations, so located errors can never
disagree on a path. Add two differentials locking the duplicated engines
together: equivalent DTD and XSD content models (sequence, choice, *, +, ?,
nested choice) must agree on every child sequence across the positional
matcher and the Thompson NFA; and the pull (EventReader) and push
(PushScanner) tokenizers must yield the same event stream for the same
document, even with the push side fed one character at a time (adjacent text
events coalesced, since chunk boundaries legitimately split text). Both
pairs agree on everything today; the differentials guard them.
Closes#115
Deepen the two thin coverage zones (#119) through the validation-driven
harness. Serializer options matrix (10 cases): self-close on/off, CDATA kept
vs as-text, asciiOnly hex character references, round-trip carriage-return
escaping, declaration with standalone, exact pretty-print shape, attribute
escapes, and the Latin-1 output-encoding declaration via the byte path. HTML
depth (8 cases): the adoption agency's furthest-block and deep-misnesting
paths, foster-parented table text (with the implied tbody per the HTML5
algorithm), template content nesting through the document parser, and
raw-text script, implied dt/dd and table-cell closes, and comments through
the fragment parser. All conform; the one initial mismatch was a wrong
expectation in the test (the implied tbody), not a code divergence.
Closes#119
Lock the strict and recovering parse paths together: recovery synthesizes
events through machinery the strict path never exercises, so a 21-document
corpus of well-formed input (the C14N conformance inputs plus nesting,
mixed content, attribute quoting/escapes, namespaces and redeclaration,
entities, character references, prolog, comments, CDATA, xml:space) asserts
that PureXML.read produces the identical Node tree as PureXML.parse with no
diagnostics, and that readTree's editor tree projects to the same node. Both
paths agree on every document; the differential now guards them.
Closes#116
Document the weak-parent ownership trap at the public entry points: parseTree
and readTree now state that a tree (or ownerDocument) must be held while using
any node inside it, that TreeNode has no internal synchronization (confine to
one thread or actor), and that the immutable Sendable Node projection is the
way across concurrency boundaries; the TreeNode class doc carries the same
contract. QualifiedName's malformed-colon fallback (':local', 'prefix:', '::'
stay whole and unprefixed), already implemented, is now documented on init
and locked by a test.
Closes#114
A selector or field XPath that fails to compile is now reported once up
front as a located error (identity constraint 'k': invalid selector/field
XPath '...'), instead of try? silently turning a schema author's typo into
a disabled constraint. The per-run query cache from #112 is the choke point:
compileErrors() probes every constraint's selector and fields in
deterministic order, the failures cache as nil, and the walk afterwards
treats those constraints as matching nothing, so the rest of the document
still validates. Tests cover a malformed field and a malformed selector.
Closes#111
XSLT: template selection resolves each match pattern once per transform via
a per-run MatchCache (the pattern's node set is computed once over the source
tree; selection becomes an identity lookup), replacing per-(node, template)
recompilation and full-tree re-walks. XSD identity validation: a per-run query
cache compiles each selector/field XPath once instead of at every visited
element. Schematron was verified to already store compiled queries. Behavior
unchanged (all suites green); a multi-template shape test over 300 nodes
guards the optimized path. MatchCache lives in its own file to keep the
transformer under the length cap.
Closes#112
An xsi:type naming an undeclared type is now a located error in both the
tree validator (validateChild) and the streaming driver, instead of a silent
fallback to the declared type. typeReference chains resolve through one
shared resolver (resolveReference, with a visited set), replacing the inline
recursion, the 64-hop cap, and the completions guard: a circular chain now
reports 'circular type reference' instead of silently truncating, and the
streaming shallow check no longer recurses without bound on a cyclic table
(a latent crash). Tests cover unknown xsi:type (tree + streaming agree),
the circular chain (tree + shallow), the resolver's three outcomes, and a
conformance-harness case pair.
Closes#110
Grow the Tier-2 conformance corpus with 14 DTD content-model cases (sequence
order/completeness, choice exclusivity, the *, +, ? occurrence indicators,
#PCDATA, mixed content, EMPTY, and ANY) validated against the internal subset
and checked through the validation-driven harness. Own suite.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 13 XPath navigation cases (descendant
and all-element counts, positional and last() predicates, ancestor and
ancestor-or-self axes, following/preceding-sibling, the attribute axis, an
attribute predicate, union, the parent abbreviation, and position()) over a
fixed tree, checked through the validation-driven harness. Own suite.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 4 Canonical XML mode cases: exclusive
drops an unused namespace and renders one at its point of use; Canonical XML
2.0 drops whitespace-only text nodes and trims surrounding whitespace. Its own
suite (ConformanceCanonicalModesTests).
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 7 Schematron cases (assert satisfied/
violated, an inert non-matching context, numeric comparison, and report fired/
not-fired) checked through the validation-driven harness. Moved to its own
suite (ConformanceSchematronTests) so the main corpus stays under the
type-body cap.
Part of #105 (Tier 2)
Rule-canon audit (validation-rules.md / the OpenAPIKit idiom): the streaming
XSD validator was the one surface applying a bare method (validateShallow)
rather than a Validation<> value. Wrap it: ComplexValidator.shallowValidity is
a Validation<ResolvedElement, ComplexValidator> whose subject carries the
element with its already-resolved effective type (the contextual typing a
node-tree walk cannot express), and the streaming driver applies it via
apply(to:at:in:). Now every validation surface, DTD, XSD, RELAX NG, Schematron,
HTML, structural, conformance, and both streaming validators, routes through a
Validation value. Adds a direct unit test of the rule.
The fragment TreeBuilder gains the HTML5 active-formatting-list and reconstruct
step that the document builder already had: when a formatting element (b, i,
em, ...) is closed out of order, it is remembered and re-opened for the
following content. So HTML.parse now agrees with HTML.parseDocument and the
spec, e.g. <b><i></b>X</i> -> <b><i></i></b><i>X</i> and
<p>1<b>2<i>3</b>4</i>5</p> -> <p>1<b>2<i>3</i></b><i>4</i>5</p>. The conformance
corpus that surfaced this now checks the fragment path directly.
Closes#109
Grow the Tier-2 conformance corpus with 8 HTML5 cases (void elements, implied
li/p closing, tag-case normalization, the adoption agency with and without
trailing text, entity decoding, stray-end-tag) checked through the
validation-driven harness against the HTML5 parsing algorithm.
The corpus surfaced its first real divergence: HTML.parse (fragment) does not
reconstruct active formatting elements the way HTML.parseDocument (document,
spec-correct) does, so <b><i></b>X</i> yields <b><i></i></b>X instead of
<b><i></i></b><i>X</i>. The corpus uses the document path (spec-correct) and
the divergence is filed as #109.
Grow the Tier-2 conformance corpus with 10 XSLT cases (value-of, for-each,
if, choose/otherwise, sort, variable, format-number, concat, apply-templates,
and a named template with a parameter) transformed through the public API and
checked through the validation-driven harness against the spec output.
Part of #105 (Tier 2)
