format-number of a value at or beyond 2^53 ran an unguarded
Double-to-Int conversion, a fatal trap reachable from user data; a
fraction picture beyond 15 places overflowed the scale conversion the
same way; and xsl:number value= beyond 2^53 trapped identically.
Digit extraction clamps fraction places to 15 (the 2^53 exactness
bound), pads back to the picture's minimum, renders out-of-range
integers through the exponent expander, and xsl:number falls back to
plain number rendering outside [1, 2^53). Sort-key rows always carry
one key per sort. Regression tests cover all three trap paths.
Raw-output marker resolution and 16.4 stripping run only when the
stylesheet mentions disable-output-escaping, so source text containing
U+E000/U+E001 survives every other transform untouched, and the text
method removes markers without unescaping content that was never
escaped. The pattern-matching function table is built once per
transform instead of per (template, node) membership test, and a dead
variable left in the numbering engine is gone. Eight focused unit
tests cover d-o-e (with the fragment rule and marker safety), CDATA
terminator splitting, simplified syntax, the html default method with
META injection, per-document key(), and attribute-start axes, so the
burn-down behaviors are CI-visible without the opt-in corpus.
The following and preceding axes work from attribute and namespace
starts, anchored at the owner element per XPath 1.0 document order:
an attribute follows its element, precedes the element's children,
and has no descendants of its own. Xalan baseline 182 -> 174
(axes 16 -> 8).
key() resolves its index against the current node's own document,
built lazily per document root, so keys work inside document() trees;
a node-set use expression indexes the matched node under every
member's string value; and key() results return in document order
regardless of lookup value order. Xalan baseline 187 -> 182.
Sort keys evaluate once per node with the node's own position, size,
variables, and bindings, so position()-based keys work, and NaN keys
order before numbers under data-type=number. Match patterns evaluate
over the matched node's own document, so templates match nodes loaded
through document(); the match cache keys include the document
identity. Xalan baseline 204 -> 187 (sort 18 -> 11, mdocs 14 -> 6).
The XML declaration precedes the document type declaration; a
public-only doctype is emitted by the html method and ignored by the
xml method, which requires doctype-system; a literal result element
carrying xsl:version compiles as a single match="/" template (the 2.3
simplified syntax); and cdata-section text containing ]]> splits
across section boundaries. Parser entry and Parts move to
XSLTParserEntry.swift for the file caps. Xalan baseline 218 -> 204
(output 34 -> 21).
Templates carry their in-scope xmlns bindings into match patterns
(match cache keyed on pattern plus bindings) and instantiated
expressions, resolving prefixed name tests by namespace URI; with
bindings supplied an unprefixed test selects only the null namespace,
the XPath 1.0 rule. Instruction-scope capture includes the default
declaration: xsl:element honors in-scope xmlns="uri" for unprefixed
names, xmlns="" keeps them namespace-free, and an unusable created
name drops the wrapper, keeping content. Xalan baseline 263 -> 218
(namespace 41 -> 13).
key() and document() read the string value of attribute and namespace
node arguments instead of dropping them, so the Muenchian grouping
idiom works; match patterns evaluate with the XSLT function table,
making key('k','v') and id('x') patterns match; and the XSLT entry
points parse sources, stylesheets, and document() loads with doctypes
allowed, as the conformance corpora require. Xalan baseline
292 -> 263 (idkey 39 -> 16).
apply-templates and for-each keep attribute and namespace nodes in
their selections; the context carries the non-tree current node with
its owner element backing the tree-only machinery, XPath evaluation
and current() start from it, match patterns select attribute nodes
through attribute identities in the match cache, the built-in rules
copy an attribute's or namespace's value as text, xsl:copy copies the
focused attribute or declaration itself, and an attribute result
arriving after element content is ignored (the XSLT recovery).
Sorting, for-each, and choose move to XSLTSorting.swift for the file
caps. Xalan baseline 352 -> 292 (axes 35 -> 16, copy 29 -> 16).
disable-output-escaping on xsl:value-of and xsl:text (sentinel-marked
raw text, unescaped after serialization; string extraction from a
fragment drops the markers per 16.4), the html default method for a
null-namespace html root (16.1), the spec-default XML declaration for
the xml method, META Content-Type injection into head (16.2),
case-insensitive raw-text and void element classes, and HTML 4.01
Latin-1 named entities on html output. Whitespace-only stripping now
means XML whitespace, so NBSP-only text survives. Conf comparison
treats tag-adjacent whitespace as layout outside the canonical path.
Xalan baseline 371 -> 352 (output 49 -> 30).
A literal result element copies its in-scope namespace declarations to
the result per 7.1.1, minus the XSLT namespace and the namespaces of
prefixes listed in exclude-result-prefixes and
extension-element-prefixes in scope (#default included); an aliased
stylesheet namespace declares its result prefix and URI instead. The
fixup pass drops declarations that repeat an inherited binding so the
copied nodes do not redeclare per descendant, and created names reuse
stylesheet prefixes. xsl:output indent=yes switches to the Xalan
indentation form (own lines, no leading spaces). Xalan baseline
446 -> 371 (namespace 89 -> 42).
The picture parses into affixes, number part, and the negative
subpattern: literal prefixes and suffixes pass through, the negative
subpattern's affixes apply (with the default minus when it has no
distinguishing ones), per-mille and percent multiply and stay literal,
the grouping size derives from the pattern's last grouping separator,
colliding decimal/grouping separators read as grouping, and U+00A4
renders as the en-US currency symbol. Digit extraction now uses one
scaled rounding and integer arithmetic, eliminating per-digit float
drift. Xalan numberformat 30 -> 1; baseline 475 -> 446.
xsl:number implements all of XSLT 1.0 section 7.7: level
single/multiple/any with count and from matched as patterns through the
match cache, the value expression with the below-one bypass, and the
format-token engine (punctuation, separator and token reuse, zero
padding, alphabetic/roman/Greek sequences, digit grouping). An
unmatched single/multiple count renders empty; level=any always yields
a number. document() now caches per transform so repeated loads of the
same URI return the identical document and generate-id() stays unique
against live nodes (the idkey49 flake). Xalan baseline 557 -> 475
(numbering 84 -> 7).
xsl:attribute/xsl:element honor the namespace attribute and resolve
prefixed names against the instruction's in-scope stylesheet
declarations (captured at parse); xsl:copy applies use-attribute-sets;
and a namespace-fixup pass before serialization declares every
namespace a created name carries, reusing same-URI bindings or
generating ns0.. prefixes, and undeclares an inherited default that
would capture an unqualified name. Xalan baseline 604 to 557.
The Apache xalan-test conf corpus (1690 gold-bearing cases, 36
categories) runs opt-in via XALAN_TS_ROOT with normalized comparison:
canonical XML when both sides parse, collapsed whitespace otherwise.
First measurement passes 1086; the 604 known failures live in
Tests/Fixtures/xalan-baseline.txt, asserted exactly in both directions.
Top classes: namespace handling, xsl:number, axes corners, output
options, attribute sets, id()/key().
A RELAX NG schema document is validated before pattern interpretation
in three layers: the section-3 grammar check (vocabulary, attributes,
arity, text, QNames, datatypeLibrary URIs, hrefs, 4.16 excepts, xmlns
prohibitions), compiler checks (4.17 combine rules, unresolved refs,
parentRef scope, startless grammars, include/externalRef cycles and
overrides, grammar-checked loaded documents, unknown datatypes,
built-in-library params), and the 4.19/7.1-7.4 restrictions over the
compiled algebra after 4.20/4.21 normalization (prohibited paths,
content types, duplicate attributes via representative-name overlap,
infinite-class attributes outside oneOrMore, interleave constraints),
with recursion detection deliberately before normalization. Incorrect
schemas: 213 compiling down to 7, all the pre-Fifth-Edition name-class
class (count asserted). Two leniency tests updated to spec behavior.
Closes#131
A value of type QName compiles to its (namespace, local) pair via the
schema's xmlns/ns scope; the derivative engine threads the instance
element's in-scope bindings to value comparison in both the tree walk
and the streaming state (frames carry their bindings), so instance
prefixes and the default namespace resolve against the instance
document and unbound prefixes match nothing. Streaming also gains the
tree walk's empty-element rule. The spec suite's instance level is
fully clean: valid 289/289, invalid 291/291; only the asserted
schema-correctness class remains.
DocumentType.validityFindings carries ValidityFinding values (reason
plus subject) instead of strings, DTDSchema.declarationErrors holds
ready ValidationErrors, and every declaration-level finding renders at
its declaration's coding path (x, r/@a) instead of at the document
root, matching the XSD consistency-errors precedent. Texts unchanged.
Each grammar element opens a define scope (scope-mangled names in the
flat table) with parentRef reaching the enclosing scope; includes are
transitive and include-level defines replace the included grammar's
same-name defines (4.7 override); div wrappers flatten while keeping
ns/datatypeLibrary inheritance (4.11); value text stays untrimmed so
whitespace-preserving datatypes compare exactly (6.2.9); dataExcept
recurses through choices. Spec suite: valid 286/289, invalid 287/291;
the six remaining instance cases are all the QName datatype.
The compiler implements simplification 4.9-4.10 (ns inheritance across
ancestors and document boundaries; QName resolution against in-scope
xmlns declarations; the attribute name= shorthand stays namespace-less),
include brings in the included grammar's start with overrides (4.7),
multiple starts and defines combine with a single omitted combine
merging by the named method (4.17), data/except lands as a dataExcept
pattern in the derivative engine (4.12), an empty element matches data
whose lexical space admits the empty string, and whitespace-only
attribute values match empty-matching patterns. Spec suite: valid
274/289, invalid 279/291.
Foreign-namespace elements are stripped before pattern interpretation
(namespace-less elements count only inside fully unqualified schemas),
name/type/combine values are whitespace-trimmed, and include and
externalRef hrefs resolve against in-scope xml:base attributes and the
loaded document's URI, threaded through nested loads with visited
tracking on resolved hrefs. Spec-suite baselines tighten to valid
242/289 and invalid 278/291.
James Clark's spectest.xml runs opt-in via RNG_TS_ROOT: nested
testSuite scopes thread resource/dir definitions into the compiler
loader, and each case asserts schema compile/reject plus per-instance
accept/reject. First measurement carried as exact baselines: all
correct schemas compile, valid 233/289, invalid 277/291; schema
correctness is not yet validated (all 213 incorrect schemas compile,
count asserted). Burn-down classes recorded: grammar-level schema
validation with the 4.16-4.18 restrictions, foreign-namespace
stripping, xml:base-aware resource resolution, and the section 6
compatibility datatypes.
The six worked examples of the C14N 1.0 Recommendation's section 3 run
as input/expected-output tests. They exposed and fixed: top-level
siblings now separated by single line feeds in the canonical form;
superfluous empty default-namespace declarations dropped in inclusive
mode (xmlns="" renders only when undeclaring a rendered non-empty
default); and scalar-level escape loops, so a CR clustered with a
following LF still escapes to 
. Escaping split to its own file
for the length caps.
Closes#132
The reader buffers one Unicode scalar per element, splitting
multi-scalar graphemes from any source, so combining marks cannot merge
with delimiters and the Fifth Edition's scalar-defined name productions
classify correctly; the replacement-text, content-model, and ATTLIST
sub-scanners and the entity-literal validator follow, while grapheme
clusters reassemble wherever scanned text accumulates. Clears the last
eduni baseline, grows the asserted IBM pre-5e class to 300, and with
the xmltest manifest's own EDITION exclusions for 140/141 empties the
xmltest baseline: every xmlconf case applicable to a Fifth Edition,
namespace-aware processor passes.
Closes#135
Limits(strictInternalSubset: true) rejects conditional sections and
parameter-entity references inside markup declarations in the internal
subset, the two feature classes XML 1.0 reserves for the external
subset. DeclSep references and quoted attribute defaults (AttValue)
stay legal; defaults unchanged. The xmltest runner opts in, leaving
case 141 (pre-Fifth-Edition name classes) as its only deviation.
Closes#128
A relative system identifier declared inside an external entity
resolves against that entity's own URI (RFC 3986, the C14N resolver):
ExternalID gains base and resolvedSystemID, the scanner threads the
current entity's URI through external-subset and PE scans, and
resolvers receive the merged identifier. Clears eduni E18; the eduni
baseline now holds only the grapheme-lexing class (#135).
Closes#138
A declared 16/32-bit encoding over BOM-less 8-bit bytes is rejected; a
BOM followed by a declaration outside its family is rejected, reading
the declaration through the BOM's own byte pattern; and an external
entity whose text declaration names a higher XML version than the
document is refused (errata E38). The eduni not-wf baseline is empty.
Reader and decoder helpers split for the length caps.
Closes#137
The namespace layer rejects malformed qualified names, declarations of
the xmlns prefix, misbindings of the xml prefix and either reserved
namespace name (including as the default namespace), and prefix
undeclaration; attributes must be distinct by expanded name; PI targets
and entity/notation names are NCNames; and DTD-typed xmlns values bind
their normalized form. Clears all 18 namespace cases from the eduni
baseline; xmltest valid/sa/012 joins the documented deviations per the
suite's own NAMESPACE='no' flag.
Closes#136
The eduni sections run with exact classified baselines. Fixes: 3.3.3
CDATA attribute-value normalization (literal whitespace to space before
reference decoding; char-ref whitespace survives; tokenized collapse on
SPACE only); production-68 lenient undeclared entities in content and
attribute values (validity finding, reference kept literal, nothing
fetched); errata E15 (references/comments/PIs/whitespace in EMPTY,
char-ref whitespace as data in element content, double-escape shape
included); errata E2 duplicate enumeration tokens; errata E14
declarations completed inside PE replacements (VC: Proper
Declaration/PE Nesting); NCName ID values; namespace declarations
subject to the every-attribute-declared rule. Reader validity machinery
split to EventReaderValidity for caps.
Closes#127
All 13 cases of the xmlconf japanese section decode through the
package byte decoder (UTF-8, UTF-16 both orders, Shift_JIS, EUC-JP,
ISO-2022-JP), parse, and DTD-validate cleanly with zero deviations.
The optional encodings the suite marks TYPE='error' are all supported,
so every case is held to the must-parse bar.
Closes#126
The IBM section (1094 cases) runs clean to two documented classes:
279 pre-Fifth-Edition character-class cases (P85-89, count asserted
exactly) and 2 PE-in-internal-subset feature cases. Real gaps closed:
a document needs a root element (production 1); entity references in
literals must be Names or character references; WFC: No Recursion
checked at declaration over the general/parameter reference graph;
undeclared PE references split into WFC (standalone, or no external
declarations possible) versus deferred validity finding, also for
attribute-default references; refused external PEs are neither; and
groups split across PE replacements report VC: Proper Group/PE
Nesting. Entity machinery split to DoctypeScannerEntities for caps.
Closes#125
The scanner records declaration provenance (internal-subset entities,
element models, attribute lists); the validator reports the three
standalone='yes' dependencies on the external subset: a default that
would be supplied, a non-CDATA normalization that would change a value,
and whitespace in externally-declared element content. The parser hides
externally-declared entities from standalone documents, so referencing
one reports as undeclared (WFC: Entity Declared, 2.9). The Sun section
is fully clean: not-wf 56/56, valid 28/28, invalid 74/74, no baseline.
Closes#134Closes#124
VC Root Element Type (root matches the DOCTYPE name), VC Unique
Element Type Declaration, VC No Duplicate Types in mixed content, VC
One ID per Element Type, VC Notation Attributes and Notation Declared,
VC Attribute Default Legal plus ID Attribute Default, lexical Name
checks for the ID family, and CDATA as character data in element
content. Strict mode adds VC Attribute Value Type (every attribute
declared; namespace declarations exempt). Declaration findings are
computed once at schema build and reported at the document root. Sun
invalid coverage moves from 43/74 to 62/74; the twelve remaining are
all the standalone='yes' family.
A general entity whose replacement text contains markup (directly or
transitively) is reparsed as content at the reference (4.4.2 Included):
the replacement is WFC-validated, cycle-checked, and budgeted, then
pushed back into the reader stream, so its elements become elements in
the tree and the pull-event stream rather than escaped character data.
Character references in internal entity literals now expand at
declaration time for general entities too (4.4.5), making char-ref
markup splice and the Appendix D double escape render as a literal
ampersand; the reference-time WFC validates remaining references
instead of re-expanding. Text-only entities keep the coalescing
single-pass path. Sun valid is 28/28.
Closes#133
Tokenized (non-CDATA) attribute values normalize before validity
checks, so enumerations, NOTATION lists, and the ID/IDREF table see
collapsed values. External parsed entities lose their leading text
declaration (validated: version optional, encoding required, never
standalone) instead of carrying it into replacement text. Character
references in PE literals expand at declaration time, making the
Appendix D '%zz;' example parse. A PI target inside the DTD subset
must be separated from its data. Sun baselines tighten to not-wf 55/56
and valid 26/28; the two remaining valid failures share one cause
(replacement text with markup is included as character data, filed
separately).
The first runner to exercise the validity layer: not-wf must be
rejected (54/56), valid must parse and DTD-validate clean (19/28),
invalid must parse but report at least one validity error (43/74).
Deviation baselines are exact and annotated with their causes, the
burn-down list for #124: tokenized attribute-value normalization,
external-entity text declarations, declaration-time char-ref expansion
in PE literals, standalone VCs, ID/IDREF corners, REQUIRED enforcement,
per-type attribute VCs, and the root/DOCTYPE name match.
The OASIS/NIST section of the W3C conformance suite runs via a new
XMLCONF_ROOT-gated runner driven by the section's own manifest (parsed
with PureXML): 247/247 not-wf rejected, all 100 well-formed cases parse,
zero deviations. Closing its 24 gaps makes the DTD subset grammar
strict: exact PEReference shape, strict declaration names and keywords,
NDATA and notation tails, no bare '%' in entity values, case-sensitive
balanced conditional sections, junk between declarations rejected,
external-subset violations now propagate (with text-declaration
support, production 77), and a PI target must be separated from its
data. The xmltest baseline tightens to 181/186: CDATA in the internal
subset (107) is now rejected, five documented deviations remain.
Closes#123
The derivation final check and Particle Valid (Restriction) move from
fail-fast throws at schema compile into composable Validation rules
(finalRespected, restrictionsAreSubsets) over per-type SchemaTypeFact
subjects, so a schema with several problems reports every finding at
once. SchemaError gains .inconsistent([String]) carrying all findings
(texts unchanged) and drops .finalViolation/.invalidRestriction.
checkRedefine, checkAllGroups, and notASchema stay throws: they examine
the raw schema source, the compilation analog of well-formedness.
Implement the replacement-text constraint at the point of reference: when a
general entity is included, its replacement (character references expanded
as at declaration time) must reparse as balanced content in isolation, via a
new EntityReplacementGrammar mini-parser. This rejects a bare or incomplete
reference ('&', '	' unterminated), tags spanning the entity boundary
('</foo><foo>'), a raw '<' or incomplete reference inside an embedded
attribute value, '<' followed by a non-name character, and a reserved-target
PI in the replacement, exactly the suite's semantics, including that an
UNREFERENCED entity may carry any value and that only the first declaration
of a name binds. Attribute-list defaults must reference already-declared
internal (not external/unparsed/forward-declared) entities with a
non-recursive chain. A CDATA section inside replacement text now shields its
content from reference expansion, fixing the last valid-side case.
W3C xmltest final standing: valid/sa 119/119, not-wf/sa 180/186, with the
six remaining deviations all deliberate and documented in the baseline:
internal-subset conditional/CDATA sections (063, 107) and parameter-entity
references (160-162) are features this package supports, and 141 expects the
1998 suite's name-character classes where this package implements XML 1.0
Fifth Edition's. Zero false positives across all 846 tests at every step.
Closes#120
Make the DTD declaration grammar strict: <!ENTITY> requires its whitespace
(after the keyword, around %, after the name), a real name (not #DEFAULT),
nothing but '>' after the value, NDATA preceded by whitespace and forbidden
on parameter entities; entity-value literals are validated (XML Chars only,
every & a complete reference); SYSTEM/PUBLIC external identifiers parse
strictly (PUBLIC needs both literals for entities and the DOCTYPE, public-only
stays legal for notations) with PubidChar validation; <!ATTLIST> bodies are
validated against XML 1.0 productions 52-60 via a new DTDAttListGrammar
(spacing, attribute types, enumerations, quoted defaults); the reserved xml
PI target is rejected inside the subset. Also: strict UTF-8 decoding (no
silent replacement of malformed sequences), XML Char checks in attribute
values and CDATA, the declaration's version/encoding value grammar, and
required whitespace between pseudo-attributes. The strict machinery lives in
DoctypeScannerStrict/DTDAttListGrammar to stay under the length caps.
W3C xmltest: not-wf/sa 162/186 rejected (from 129; 80 at first contact),
valid/sa 118/119, zero false positives across all 846 tests at every step.
The remaining 24-case baseline is exactly: the entity replacement-text class
(19) and the deliberate internal-subset extensions this package supports as
features (conditional sections, PEs in the internal subset: 5).
Part of #120, #105
Third W3C-suite strictness pass: DTDContentModelGrammar validates each
element declaration's content specification against the XML 1.0 grammar
(productions 46-51) at DTD scan time, EMPTY/ANY, Mixed (with #PCDATA first,
names only, the trailing * required with alternatives, no nesting), and
children (one connector per group, quantifiers bound directly to their
particle, balanced parentheses, no empty groups, no SGML leftovers like
'- -', 'O O', '&' connectors, or inclusion/exclusion suffixes). The element
declaration also requires whitespace after <!ELEMENT and after the name. The
lenient ContentModelParser still drives validation behavior; this is a
well-formedness gate. not-wf/sa: 129/186 rejected (from 109); valid/sa
unchanged; zero false positives across all 846 suite tests; baseline rebased
to the exact 57 remaining (declaration-grammar details and the entity
replacement-text class).
Part of #120, #105
Character references now require lowercase 'x' for hex and a referenced code
point inside the XML Char production (rejecting �, , ,
surrogates, and X). The reserved 'xml' processing-instruction target
(any case) is legal only as the declaration at the document's very first
bytes: whitespace or a comment before <?xml ...?>, a declaration inside
content or after the root, and case variants like <?XML?> are all rejected.
Comments and PI data are checked for the XML Char production like content.
not-wf/sa: 109/186 rejected (from 93); valid/sa unchanged at 118/119; all
846 suite tests green, baseline rebased to the exact 77 remaining (the
internal-subset grammar and entity replacement-text classes on #120).
Part of #120, #105
Add five well-formedness checks the W3C xmltest suite surfaced: '--' inside
comments (also rejecting the ---> ending), raw '<' in attribute values, the
literal ]]> sequence in character data, attributes not separated by
whitespace, and characters outside the XML Char production in content, each
a located ParseError. The attribute scanners move to EventReaderAttributes
to stay under the type-body cap.
Add the opt-in W3C xmltest runner (XMLTS_ROOT-gated; the suite is never
vendored, James Clark's license permits redistribution only as the unmodified
archive; Foundation is used in this one test file for file access only).
Baseline: valid/sa 118/119, not-wf/sa 93/186 rejected (up from 80), the
remaining 93 carried as an exact known-deviations baseline so progress and
regressions are both visible. The two remaining leniency classes (internal-
subset DTD grammar, entity replacement-text well-formedness) are #120.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 7 OASIS catalog cases through the
harness: exact system/public/uri entries, rewriteSystem and rewriteURI prefix
matching, and the unresolved outcomes for unmatched identifiers. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 17 XSD regex cases through the
pattern-facet path: implicit whole-string anchoring, the character-class
escapes (backslash-d/w/s), Unicode category and negated-category escapes,
quantifier ranges (exact, bounded, open), alternation, grouping, negated
character classes, and class subtraction. All conform.
Part of #105 (Tier 2)
Grow the Tier-2 conformance corpus with 10 cases through the harness:
XInclude substitution, parse=text inclusion with escaping, xi:fallback on a
missing href, and relative href resolution against the base; XPointer
shorthand ids, element() navigation from an id and from the root, and
xpointer() including the classic per-step predicate semantic (//para[1]
selects every parent-first para; (//para)[1] the document-first), which the
corpus now locks explicitly.
Part of #105 (Tier 2)
