Fifth serialize-path pass (#139, docs/benchmarks.md pass 31). textNeedsEscaping,
attributeNeedsEscaping, and the comment/processingInstruction existence checks
run unconditionally on every text node, attribute value, comment, and PI in a
document, scanning through UTF8View's default Collection iteration. Profiling
a new deeply-nested, attribute-dense corpus (the standard flat catalog corpus
has no density to indict this) put the two needsEscaping functions as the
largest PureXML-specific frames in the sample. A standalone micro-benchmark
confirmed withContiguousStorageIfAvailable's raw buffer scan measures ~2-2.75x
faster than the Collection-based scan, with a Collection fallback kept for a
non-contiguous (foreign/bridged) string.
Two abstraction attempts were tried and rejected before landing this shape: a
shared generic helper taking a predicate closure recovered most but not all of
the win, and calling a separate small function from inside the buffer loop (to
dodge a cyclomatic-complexity violation) erased nearly the entire win even
though it isn't a closure value. The violation is instead resolved by splitting
each function into a contiguous/collection pair, moving the call boundary
outside the loop rather than inside it.
Output is byte-identical by construction; full local suite (1925 tests) and
full lint unchanged. Measured with interleaved same-state A/B under real
concurrent machine load: clean with zero overlap across 5 pairs at the nested
corpus (~9%), noisier at the standard corpus but the means still separate
clearly (~4%).
Two profile-guided serialize passes (#139, docs/benchmarks.md passes 29-30).
The attribute writer now precomputes the quote character as a String once
per element instead of interpolating a Character into every attribute,
avoiding the grapheme-breadcrumb machinery that repeats twice per attribute
for a value that never changes within an element. Escaping.comment and
processingInstruction's escaping fallback (taken whenever a comment has a
hyphen or PI data has a '?', a case pass 8 could skip past but not speed up)
is rewritten in the same byte-scan bulk-copy shape as passes 23-24, so a
value with one rare trigger no longer pays a full character-by-character
rebuild.
Both are same-observable-behavior changes. The quote change is byte-
identical by construction. The comment/PI rewrite is verified against a new
differential suite that reimplements the original character-loop as an
independent oracle and checks adjacency/trailing/combining-mark cases plus
two 20000-case seeded fuzzes, since a byte match inside a combining-mark
cluster is not, on its own, the bare trigger character. Full local suite
(1925 tests) unchanged.
Measured with interleaved same-state A/B: pass 29 ~1.5% at a 100k-item
corpus; pass 30 ~2x on a dedicated comment/PI micro-benchmark (the standard
corpus has no comment/PI content, so real documents with copyright banners
or hyphenated comment prose see this, not the headline ratio). A same-
session multi-append restructuring attempt measured a clear ~6% regression
and was reverted; a suspected hasInlineContent double-scan was profiled at
~0.2% of samples and not worth a pass. Both outcomes are recorded in
docs/benchmarks.md rather than dropped.
The byte-path close-quote consume candidate measured NEUTRAL in an
interleaved A/B despite the sampler attributing ~4% of parse to the
advance() chain it replaced: decoding one ASCII scalar into a
small-string-backed Character costs about the same as consume's
literal-matching path. Recorded under Known structure of the gap so no
future round redoes it, with the meta-lesson that sample attribution
overstates single-character round trips and only the interleaved A/B
verdict counts.
Also records the #8 decision: the TreeNode weak-parent side-table cost
(~9% of parse) is accepted for now. unowned is rejected outright since a
trap after release contradicts the never-crash posture and the documented
nil-after-release behavior; the arena/index-based tree remains the
designed future path, sequenced after the scanning-architecture work,
which carries ~47% of parse to this finding's ~9%.
Fourth profile-guided pass (#139, docs/benchmarks.md pass 28). scanAttributes
allocated a fresh Set<String>'s native storage on every element with
attributes, just to check a handful of names for duplicates; the set is now a
stored EventReader scratch property cleared with removeAll(keepingCapacity:)
per element, so each element's check reuses the previous element's storage.
Safe because scanAttributes cannot re-enter itself (nothing inside an
attribute scan starts another start-tag scan), so the shared storage never
sees two interleaved scans; duplicate detection, recovering-mode behavior,
and thrown errors are unchanged.
Verified in an isolated worktree at the previous commit: full local suite
(1858 tests) and full W3C XSTS suite (14383 groups) pass unchanged. Measured
with an interleaved same-state A/B, baseline/patched/baseline/patched each
best-of-5, because the machine carried concurrent load this session: both
patched runs (0.1073s, 0.1067s) beat both baseline runs (0.1091s, 0.1117s),
~2-4% on parse.
Recorded alongside, per the benchmarks log's discipline: a sibling candidate
(reserving each tree frame's children capacity on first append) measured
neutral under the same protocol and was reverted rather than landed, and the
TreeNode weak-parent side-table cost (~9% of parse, swift_weakAssign per
child plus a runtime side table per parent) is documented under Known
structure of the gap as needing a design decision, since parent's
nil-after-release semantics are documented public API. Touched lines carry
repo-standard swiftformat rewraps applied by a concurrent formatter run.
Third profile-guided pass (#139, docs/benchmarks.md pass 27), targeting
scanContentMarkup(), which inlines the start-tag/attribute/namespace scanners
and carries ~46% of parse time.
scanAttributeValue's opening and closing quote checks ran on the Character
path (peek + advance, buffering the quote as a Character only to compare and
discard it) on every attribute value in the document; both now check the raw
byte first, reaching the byte fast path with no Character buffering, with the
Character path kept as the off-fast-path fallback for streaming sources. The
quote bytes are ASCII and non-CR/LF, so the raw byte and the normalized
stream agree, the same reasoning peekByte's other callers already rely on.
NamespaceContext.checkExpandedNames allocated a Set's native storage and
built a concatenated URI+local key string per element even when at most one
attribute carries a namespace URI, where no duplicate is possible (the
default namespace never applies to attributes, and xmlns declarations carry
no URI). Counting URI-bearing attributes first, with no allocation, and
returning early below two skips all of it on the overwhelming majority of
elements; for xml:* attributes specifically the key exceeded the small-string
limit, so this also removes a real per-element heap allocation.
EntityDecoder.decode/decodeLenient guarded with raw.contains("&"), a
grapheme-by-grapheme walk to find one ASCII byte, once per ampersand-bearing
text node or attribute value; raw.utf8.contains(0x26) finds the same byte
without decoding graphemes, the same shape as checkNameShape's colon check.
The quote-handling extraction into two helpers also resolves the cyclomatic
complexity the inline version tripped. Six new regression tests pin the
quote edge cases (empty value, single-quoted, a quote kind nested inside the
other, unterminated at end of input) plus byte/Character-path agreement for
each in the parser path differential.
Verified: full local suite (1858 tests) and full W3C XSTS suite (14383
groups) pass unchanged. Parse ~6.00x -> ~4.75x versus libxml2 at the
20000-item corpus, best of 5.
Two profile-guided passes (#139), found by sample-profiling a 42MB generated
corpus after re-measuring the parse/libxml2 ratio fresh:
scanAttributeValue's closing-quote check re-derived an already-known answer
through Reader.consume's general String-iteration path, which allocates a
fresh String(quote) and iterates it via Character machinery twice, on every
single attribute value in the document. The content loop's own break
condition already guarantees the buffered peek is either the quote or end of
input, so checking it directly and advancing once is the same behavior for
less work.
NamespaceContext.enterElement, called once per element: checkNameShape's
name.localName.contains(":") resolved to Sequence.contains(_:), walking
extended grapheme clusters to find one ASCII byte; utf8.contains(0x3A) finds
the same colon via the UTF-8 byte view. Separately, attributes.map{
resolveAttribute } allocated a fresh backing array on every element even
though resolveAttribute returns its argument unchanged whenever the attribute
has no prefix or the xmlns prefix (the common case); checking first with no
allocation and reusing the input array when nothing would change gives the
identical result without it.
Both are same-observable-behavior changes, verified against the full local
suite (1852 tests) and the full W3C XSTS conformance suite (14383 groups),
both unchanged. Measured parse ~6.59x -> ~6.00x versus libxml2 at the
20000-item/4.2MB corpus, best of 5; see docs/benchmarks.md passes 25-26.
xs:unique/xs:key/xs:keyref were quadratic over a wide list of targets.
The duplicate check compared each value tuple against every tuple already
seen (and a keyref against every referenced key), and the error-location
helper rescanned the parent's whole child list to position each target.
Key tuples in a Set by their raw string when field equality provably
reduces to a raw string comparison (whitespace-preserving lexical types,
no colon, which is most ids); value-space types (numeric, boolean, date),
whitespace-collapsing types, and QName-bearing values keep the exact
pairwise comparison, so which documents pass is unchanged. Position each
target through a per-parent, per-name index built once.
A 16000-item xs:unique document drops from about 4.0 s to about 0.12 s
(~34x), and the cost is now linear. The full local suite and the W3C
XSTS datatype/identity conformance corpus are unchanged.
firstInDocumentOrder() computed a document-order key even for a zero- or
one-node set. The key walks a node up to the root, taking its index among each
parent's children, which is a linear scan of a wide parent. string(@x) and
value-of take the string-value of a (usually single) node-set through this
path, so an XSLT transform over a wide flat fan-out rescanned the wide parent
once per extraction, turning the transform quadratic.
For a set of zero or one node the document order is irrelevant: the node is
trivially first. Return it directly without computing the key, the same
count <= 1 early-out sortedByDocumentOrder() already has.
A transform reading a couple of attributes and child values per element over a
20k-element document drops from ~2.3s to ~0.65s (~3.5x). The output is
unchanged: the full suite, the Apache Xalan XSLT gold-output corpus, the W3C
XPath/XSLT/EXSLT tests, and the WASM build stay green, and FirstInDocumentOrderTests
pins the single-node fast path and the multi-node document-first selection.
The bulk-copy escape loop still iterated Characters and compared each to '&',
'<', '>' (a grapheme comparison per character), the dominant remaining serialize
cost for content that needs escaping. Every escaping trigger is an ASCII marker,
or in ASCII-only mode a non-ASCII lead byte, so find the next one with
value.utf8.firstIndex(where:) and decode no grapheme for the run between markers.
The marker byte always lands on a grapheme boundary: an ASCII byte is its own
scalar and forces a cluster break (so '&', '<', '>', the quote, and whitespace
are never mid-cluster), and after a grapheme-aligned runStart the first byte at
or above 0x80 is a lead byte. So value[marker] and value[runStart ..< marker]
stay valid, and a multi-scalar cluster escapes as a unit (Swift rounds a
mid-grapheme index to its boundary, so the base letter is neither dropped nor
duplicated). The output is byte-identical.
Measured serialize ~23% faster on the 20k-item generated corpus via a same-state
A/B (~2.8x to ~2.2x slower than libxml2). Verified by EscapingBulkCopyTests
(the oracle differential extended with combining-mark, ZWJ, and flag clusters),
the Apache Xalan gold-output and C14N suites, the full suite, the WASM build,
and an adversarial grapheme-alignment review.
Escaping.text and Escaping.attribute rebuilt a value that needed escaping one
character at a time: result += String(character) for every unescaped character,
plus the escaped form for each marker. Re-escaping a text node that holds a
single '&' rebuilt the whole value grapheme by grapheme, and the per-character
String construction and append dominated the serialize path for entity-bearing
content.
Copy each run between escapable characters in a single append and rewrite only
the markers. The no-escaping-needed fast path (return the value unchanged) is
untouched. The output is byte-identical: matching is per-Character so multibyte
graphemes copy whole, and the escaped forms are the same.
Measured serialize ~7% faster on the 20k-item generated corpus via a same-state
A/B (~3.0x to ~2.8x slower than libxml2). Verified by EscapingBulkCopyTests (a
per-character-oracle differential with multibyte cases), the Apache Xalan
gold-output and C14N canonicalization suites, the full suite, the WASM build,
and an adversarial review that included a 400k-input fuzz.
parseTree ran TreeNode(parse(...)), which allocates two full trees: the parser
builds an immutable value Node tree, then TreeNode.init(_:) walks it to build
the mutable reference tree. The parsed tree is the deliverable, so building the
value tree first is pure overhead.
Parser.buildTree is the direct counterpart to build: the same iterative event
loop with the same element stack and the same well-formedness guards (an
unmatched end tag throws unexpectedEndOfInput; a document with no element throws
emptyDocument), but it accumulates TreeNode children per frame and adopts them
when an element closes, so the tree is allocated once. The produced tree is
identical: same kinds, names, attributes, values, child order, and parent links.
Measured parse ~8% faster on the 20k-item generated corpus via a same-state A/B
(6.1x to ~5.5x slower than libxml2). Verified by ParseTreeDirectBuildTests (a
structural differential against the convert-from-Node path, plus throw-parity
and a serialized round-trip), the full suite and the W3C XML conformance,
RELAX NG, and Xalan corpora (which all build trees through parseTree), the WASM
build, and an adversarial equivalence review.
EntityExpander.expand located the next '&', '<', and ';' with String.firstIndex
over graphemes, so every literal byte of a run was decoded into a grapheme just
to look for a one-byte marker. All three markers are ASCII, so a UTF-8 index is
a valid string index: scan over raw.utf8 instead, and the found index still
subscripts the string directly (the marker byte is never a continuation byte,
so its index lands on a scalar and grapheme boundary).
The decoded text is unchanged, including runs where the marker follows
multibyte UTF-8: a precomposed letter, a combining sequence, or an astral
character all leave the marker on a grapheme boundary.
Measured parse ~5.5% faster on the 20k-item generated corpus via a same-state
A/B (6.5x to ~6.1x slower than libxml2). Verified by EntityDecodeBulkTests
(extended with four multibyte-boundary cases), the W3C XML conformance corpora,
the Xalan XSLT gold-output corpus, the full suite, and the WASM build.
Building the mutable TreeNode tree from a parsed value Node ran append for every
child. append -> insert detaches the child from any previous parent, rejects a
self/ancestor cycle, and inserts into a children array that grows element by
element. The children the conversion passes come from element.children.map and
are freshly built and parentless, so all three guards are no-ops and the
incremental growth rebuilds an array that was already prepared.
TreeNode.init(adopting:) adopts the prepared array directly and wires each
child's parent. The tree is identical: the same children in the same order with
the same parent links. The adopting initializer is used only by the value-Node
conversion; every public factory keeps the detaching, cycle-checking append path
for externally supplied children that may already live in a tree.
Measured parse ~12% faster on the 20k-item generated corpus via a same-state
A/B (7.7x to ~6.5x slower than libxml2). Verified by TreeNodeAdoptingInitTests
(parent-wiring and post-adoption mutability), the W3C XML conformance corpora,
the Xalan XSLT gold-output corpus, the full suite, the WASM build, and an
adversarial equivalence review.
QualifiedName(_:) ran firstIndex(of: ":") over the whole name, grapheme by
grapheme, on every element and attribute. Most names have no prefix, so the
common case scanned the entire name to find no colon at all.
The byte scanner already visits every byte of an all-ASCII name, so have
takeASCIIName record the offset of the first colon (one comparison per byte),
and split at that known offset in QualifiedName(ascii:colonOffset:). The
unprefixed name (no colon) is then built with no scan; a prefixed name splits
at the recorded offset. The names are identical to the firstIndex path for
every colon placement: none, leading (:local), trailing (prefix:), middle,
doubled (::), and multiple (first colon wins). The hint is confined to the
all-ASCII fast path; non-ASCII names and every other construction site keep
QualifiedName(_:).
Measured parse ~12% faster on the 20k-item generated corpus via a same-state
A/B (8.5x to ~7.7x slower than libxml2). Verified by QualifiedNameColonHintTests
(colon-placement differential), the W3C XML conformance corpora, the Xalan XSLT
gold-output corpus, the full suite, the WASM build, and an adversarial
equivalence review.
EntityExpander.expand rebuilt every reference-bearing text run one character at
a time: result.append(character) per character, with a per-character charge
against the amplification budget. A text node holding a single & was copied
grapheme by grapheme, which is the bulk of the per-text-node cost in a document
whose content uses entities.
Scan to the next '&' or '<' and append the literal run in one operation,
charging the budget by the run length. The decoded text, the CDATA-verbatim
shielding, and the billion-laughs budget enforcement are unchanged: the same
overrun throws at the same input, since charging a run of length L against
budget B fails exactly when L > B, the same condition the per-character guard
enforced.
Also return early from recordReferenceContentFindings when there are no DTD
content models (the common no-DTD case): it otherwise scanned each text run for
'&' and rendered the enclosing element's qualified name before the model lookup
that would fail anyway.
Measured parse ~5% faster on the 20k-item generated corpus via a same-state A/B
(9.06x to ~8.5x slower than libxml2). Verified by the W3C XML conformance
corpora, the new EntityDecodeBulkTests six-case decode differential, the full
suite, the Xalan XSLT gold-output corpus, and the WASM build, with an
adversarial equivalence review of the rewrite.
The descendant and attribute walks tested every candidate node through a
closure that called matchesTree / matchesAttribute with the step's NodeTest and
the evaluation namespaces dictionary:
descendants(of: tree) { matchesTree($0, step.test, on: step.axis, context.namespaces) }
That passes the namespaces dictionary (and the NodeTest's associated string) by
value on every call, so a //name-style query over a wide document carried a
dictionary retain/release on each of tens of thousands of nodes, on the
traversal that dominates evaluation.
compiledTreeTest, compiledAttributeTest, and compiledNodeTest resolve the parts
that do not change as the walk moves (the axis principal kind, whether eval-time
bindings are in effect, the prefix-wildcard shape of the name) exactly once and
return a tight per-node closure capturing only the local name. The selected
nodes are identical to the unfused matchesTree / matchesAttribute / matches.
Measured xpath ~35% faster on the 20k-item generated corpus via a same-state
A/B, from ~2.0x to ~1.3x slower than libxml2. The full suite, the Apache Xalan
XSLT gold-output corpus, and the WASM build stay green. XPathCompiledNodeTestTests
is an exhaustive differential asserting each compiled closure equals the
function it specializes across every node, test, axis, and binding.
An EvaluationContext carried the variable bindings, the function library, the
namespace bindings, and the budget by value. Re-focusing the context onto each
node of a node-set (focused(on:position:size:), called once per node when a
predicate is applied) copied all three dictionaries every time, even though
none of them changes as the focus moves.
Bundle the focus-independent state in an Environment reference type. The
context now holds the node, position, size, and an Environment reference;
re-focusing copies the three integers and shares the environment by reference
instead of copying and reference-counting three dictionaries. The values and
behavior are identical.
Measured xpath ~16% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000). The full XPath suite, the Apache Xalan XSLT
gold-output corpus, and the WASM build stay green.
The attribute axis built an XPath node for every attribute of the element
and then filtered by the step's node test. A predicate like [@name='value']
runs that per matched element, so for a wide document it wrapped many
attributes only to discard all but one, and each wrap retains the owner
element and copies the attribute's qualified name and value.
Apply the same fusion the descendant axes use: test the raw attribute during
the walk and wrap only the ones that pass. matches delegates, for an
attribute node, to a new matchesAttribute (the attribute axis always carries
the attribute principal kind, so the reduction is exact), and the attribute
walk applies it as a keep filter. The node-set, declaration order, and
uniqueness are identical to build-then-filter.
Measured xpath ~20% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000); xpath is now ~2x libxml2, down from ~13x at
the start of this work. XPathFusedAttributeTests covers every node-test kind
through the fused path, and an adversarial equivalence review found no query
whose result changes. Full suite, Apache Xalan XSLT gold output, and the WASM
build stay green.
The descendant and descendant-or-self axes built the full subtree node list
and then filtered it by the step's node test. For a //name query over a wide
document most of the wrapped nodes were discarded immediately, and each wrap
retains a tree node whose weak parent forces the slow reference-counting path.
Run the node test against the raw tree node during the walk, so a rejected
node is never wrapped. matches(_:_:on:namespaces:) now delegates, for a tree
node, to a new tree-node matchesTree (one source of truth, exercised by the
full XPath suite), and the descendant walk applies it as a keep filter. The
node-set, document order, and uniqueness are identical to build-then-filter;
only non-matching nodes are no longer materialized.
Measured xpath ~11% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000). XPathFusedDescendantTests covers every
node-test kind through the fused path, and an adversarial equivalence review
found no query whose result changes. Full suite, Apache Xalan XSLT gold
output, and the WASM build stay green.
`descendants(of:)` recursed on wrapped XPath nodes and, at every node, built
a fresh array for that node's subtree and concatenated it
(`result.append(contentsOf: descendants(of: child))`). That is an array
allocation and copy per node, roughly O(n x depth) work over the subtree, and
the parse profile showed it dominating XPath evaluation for a `//`-style query
over a wide document.
Recurse on the raw tree node instead, appending into one shared inout
accumulator. The result is the identical node-set in the identical document
order; only the way the array is built changes, from O(n x depth) to O(n).
Measured xpath ~46% faster on the 20k-item generated corpus via a same-state
A/B (count unchanged at 10000). Full suite, the Apache Xalan XSLT gold-output
corpus, and the WASM build stay green.
A path that is a single forward-axis step from one context node returns its
nodes already in document order with no duplicates, so the document-order
sort is redundant. AxisNavigation produces the structural forward axes
(child, descendant, descendant-or-self, self, following, following-sibling)
in document order, deriving sibling order from the same children array the
order key compares, so the two can never disagree. This is the common shape
after //x compiles and fuses to a single descendant::x step.
The attribute axis (whose order interleaves with namespace declarations), the
namespace axis (produced in dictionary order), and every reverse axis
(nearest-first) are excluded and still sorted. Measured xpath ~7% faster on
the 20k-item generated corpus via a same-state A/B, identical results.
XPathSingleStepOrderTests pins document order for each included axis, that the
reverse axes still normalize, and that multi-step paths still sort. An
adversarial order-correctness review found no query whose order or uniqueness
the change breaks.
A path expression sorted its result into document order through orderUnique,
which first ran a Set-based de-duplication. But evaluateSteps already returns
a duplicate-free node-set: each step de-duplicates across its context nodes,
or uses an axis whose per-context results cannot overlap. So that Set copied
every node for nothing, and each copy retains the wrapped tree node, whose
weak parent reference forces a side table and the slow reference-counting
path.
Path and trailing-path evaluation now sort the steps' result directly.
Unions and filter primaries, which genuinely can carry duplicates, keep the
de-duplicating pass. Measured xpath ~9% faster on the 20k-item generated
corpus via a same-state A/B, identical results. The existing node-set
de-duplication and union tests cover that a path still yields each node once.
`//X` compiles to `descendant-or-self::node()/child::X`, which materializes
every node of the subtree as an intermediate context before selecting the X
children of each. Fuse that pair into a single `descendant::X` step at compile
time: the two select the same node-set, so evaluation skips the intermediate.
The fusion is transparent except for a positional predicate: `//x[1]` is the
first x child of EACH parent, not the first x in document order, so it must
not become `descendant::x[1]`. Apply the rewrite only when every predicate is
provably non-positional: its value is a boolean, node-set, or string (never a
number, which XPath reads as position() = value) and it nowhere references
position() or last(). Anything not provably both is left unfused, so a result
can never change.
Measured xpath ~10x to ~7x slower than libxml2 on the 20k-item generated
corpus via a same-state A/B, identical results. XPathDescendantFusionTests
pins the positional forms keeping their per-parent meaning, the non-positional
forms selecting across the subtree, and composition with trailing and nested
steps. An adversarial differential review (fused versus a spec-literal unfused
reference over nine attack suites) found no query whose result changes. The
boolean-function allowlist carries a documented invariant: every entry must be
position- and size-independent, since the position scan is syntactic.
Each location step accumulated its context nodes' axis results into a
Set<Node> to drop nodes reachable from more than one context, hashing and
copying every node. The wrapped TreeNode is reference-counted, so each copy
is ARC traffic, and the profile showed the Set insert, hashing, and Node
copies dominating evaluation.
De-duplication is needed only when two distinct context nodes can reach the
same node. Skip it on a single context and on the axes whose results are
disjoint per context (child, attribute, namespace, self): those produce the
identical node sequence the Set would, since every insertion succeeds, with
no hashing. Keep the Set on the axes that genuinely overlap (descendant,
ancestor, parent, the siblings, following/preceding). Predicates stay applied
per context and the path result still passes through the final
order-and-unique pass, so results are identical.
Measured xpath ~13x to ~10x slower than libxml2 on the 20k-item generated
corpus via a same-state A/B, ~28% faster, same result. XPathNodeSetDedupTests
pins that a node shared by many contexts through ancestor/parent/descendant
is still counted once, the disjoint axes return every distinct result, and an
overlapping union still de-duplicates. The full XPath suite and the Apache
Xalan XSLT gold-output corpus stay green.
The serializer's escaping functions rebuilt every text, attribute, comment,
and processing-instruction value character by character, even when the value
contained nothing to escape, which is the common case. Each now scans the
value's UTF-8 bytes for the bytes it would escape (the markup characters, the
active attribute quote, the whitespace characters, a comment hyphen, a PI
question mark, and any non-ASCII byte in ASCII-only mode) and returns the
value unchanged when none is present, taking the original character path only
when something must be escaped.
Measured serialize roughly halved, ~5.5x to ~2.7x slower than libxml2 on the
20k-item generated corpus, with byte-identical output. EscapingFastPathTests
pins both the unchanged fast path and the escaped slow path for each function,
and the Apache Xalan XSLT gold-output corpus (a serialize-heavy differential)
stays green. No output changed.
The attribute-value scanner ran the entity decoder on every value, and the
decoder re-scanned the whole value for '&' before doing nothing when none
was present. The byte-run scanner already visits every value byte, so it
now notes any '&' and an ampersand-free value (the common case) returns its
normalized run directly. Measured against a same-machine-state baseline,
every run came in below every baseline run, ~4% faster parse on the
20k-item generated corpus.
Move the bulk byte-run scanners (contentRunBytes, attributeRunBytes) into
ReaderByteRuns.swift and centralize their cursor mutation through a single
advanceByteRun method, so Reader's position fields stay write-encapsulated
in Reader.swift while both files stay under the length cap. The byte and
Character paths are held to identical decoded values by the differential
oracle, extended with ampersand-free, entity, and character-reference
attribute cases. No spec behavior changed.
The element-content markup dispatch ran up to five literal string
comparisons (</, <!--, <![CDATA[, <?, <!DOCTYPE) before the start-tag
branch won, a per-element cost the parse profile indicted as
Reader.matchesBytes. Branch instead on the single byte after '<' on the
byte fast path: '/' end tag, '?' processing instruction, '!' a
declaration, otherwise a start tag.
The Character path keeps the equivalent matches()-based dispatch for a
streaming source or spliced entity text, and a '<!' that matches none of
comment/CDATA/DOCTYPE still falls through to the start-tag scanner with
the same expectedName error. Measured parse ~9.5x -> ~9.0x slower than
libxml2 on the 20k-item corpus; the byte and Character paths are held
identical by ParserPathDifferentialTests over its adversarial corpus,
and the Xalan, W3C, and RELAX NG suites stay green. No spec behavior
changed.
Keep the reader on its byte fast path through dispatch and name/attribute
scanning so plain-ASCII documents avoid the per-Character String machinery
(String.Iterator.next, string comparison, small-string materialization,
bridge-object ARC) that a profile indicted as the parse bottleneck.
The root cause was that a single buffered lookahead Character defeated every
existing byte path, each of which requires an empty buffer, and the content,
markup, and attribute dispatch each peeked a Character before scanning. Add a
non-buffering peekByte for dispatch, a byte-mode skipSpace, an all-ASCII
takeASCIIName, and an attributeRunBytes bulk scanner (with the 3.3.3
whitespace-to-space transform, deferring any carriage return to the Character
path so CRLF is not double-counted). Detect a text run's closing '<' via
peekByte instead of buffering it, which had cascaded the next element's whole
dispatch and name scan onto the slow path. Skip reference decoding for
ampersand-free text. The Character path remains the exact fallback for a
streaming source, spliced entity text, and any non-ASCII byte; the two paths
share scanContentMarkup so they cannot diverge.
Measured parse 20.1x -> 9.5x slower than libxml2 on the 20k-item generated
corpus, roughly a 53 percent wall-clock reduction. A differential oracle pins
the byte and Character paths to identical event streams and error marks over
an adversarial corpus, alongside boundary tests. Verified against the full
local suite, the Apache Xalan XSLT gold-output corpus, the W3C XML conformance
corpora, the RELAX NG spec suite, and the WASM build. No spec behavior changed.
EventReader.next(), both initializers, and the Event alias are public
(an internal typealias in a split file had been shadowing the public
Parsing.Event); a new PureXMLPublicAPITests target compiles without
testable import so facade gaps fail CI. The large-corpus benchmarks
exposed document-order sorting as quadratic on flat fan-outs; the fix's
first attempt was itself worse (an eager sibling table built per
predicate evaluation), caught by sampling the live run, and replaced
with a lazy table: single-node sets never touch order keys and a
parent's table builds on its second lookup. XPath at 42 MB drops
27.1 s to 1.57 s (190x to 11.7x); at 210 MB PureXML completes in 9.4 s
the query libxml2 refuses at its 10M node-set cap (#143 tracks our
configurable budget). Benchmark harness: size series to 210 MB, CORPUS
mode for real files (NASA, SwissProt measured), SAX stage,
refusal-aware table, NULL-safe C driver. All conformance suites,
Linux, and WASM verified. Closes#142.
String input copies its UTF-8 once into owned storage (an unsafe
pointer encapsulated in a class lifetime, never escaping the parser)
and scalars decode straight off the pointer, eliminating the
per-character closure chain; ASCII literals match and consume at the
byte level; and plain-ASCII character data scans as a single byte run
with one String built per run. Parse drops 32 percent cumulative
(0.49 s to 0.34 s on the 4.2 MB corpus, libxml2 ratio 30x to 20x).
The W3C suite caught one boundary (a run starting with '>' could
swallow the ]]> check after the character path consumed ]]); the run
scanner leaves a leading '>' to the character path, pinned as a unit
regression. All conformance suites, Linux, and WASM verified.
matches() ran on every scan decision and materialized an Array of the
literal per call, the hottest comparison in the parse profile. It now
walks the literal directly. Parse time on the 4.2 MB benchmark corpus
drops 16 percent (0.49 s to 0.41 s, libxml2 ratio 30x to 26x); profile
findings and the optimization log recorded in docs/benchmarks.md.
scripts/benchmark.sh generates a deterministic corpus, builds PureXML
in release and a C driver against the SDK's libxml2, times parse,
serialize, and XPath over the same bytes (best-of-N, internal timing,
IO excluded), cross-checks the same XPath count on both sides, and
prints the comparison. First honest baseline recorded in
docs/benchmarks.md: parse ~30x, xpath ~36x, serialize ~5x slower than
libxml2 on a 4.2 MB corpus; these are the numbers #139's optimization
passes burn down. The SAX stage isolation is blocked on #142 (the
public streaming surface is internal, found by this harness).
