PureXML/docs/benchmarks.md
Mihaela Mihaljevic 7ffd1dd545 perf(parsing): remove redundant work in attribute-value and namespace scanning
Two profile-guided passes (#139), found by sample-profiling a 42MB generated
corpus after re-measuring the parse/libxml2 ratio fresh:

scanAttributeValue's closing-quote check re-derived an already-known answer
through Reader.consume's general String-iteration path, which allocates a
fresh String(quote) and iterates it via Character machinery twice, on every
single attribute value in the document. The content loop's own break
condition already guarantees the buffered peek is either the quote or end of
input, so checking it directly and advancing once is the same behavior for
less work.

NamespaceContext.enterElement, called once per element: checkNameShape's
name.localName.contains(":") resolved to Sequence.contains(_:), walking
extended grapheme clusters to find one ASCII byte; utf8.contains(0x3A) finds
the same colon via the UTF-8 byte view. Separately, attributes.map{
resolveAttribute } allocated a fresh backing array on every element even
though resolveAttribute returns its argument unchanged whenever the attribute
has no prefix or the xmlns prefix (the common case); checking first with no
allocation and reusing the input array when nothing would change gives the
identical result without it.

Both are same-observable-behavior changes, verified against the full local
suite (1852 tests) and the full W3C XSTS conformance suite (14383 groups),
both unchanged. Measured parse ~6.59x -> ~6.00x versus libxml2 at the
20000-item/4.2MB corpus, best of 5; see docs/benchmarks.md passes 25-26.
2026-07-12 11:40:33 +02:00

144 lines
18 KiB
Markdown

# Benchmarks: PureXML vs libxml2 (#139)
`bash scripts/benchmark.sh [items] [iterations]` generates a deterministic
corpus (never vendored), builds PureXML in release and the C driver against
the macOS SDK's libxml2, runs both over the same bytes with internal timing
(best-of-N, IO excluded), and prints the comparison. Both drivers also
cross-check the same XPath count, so every benchmark run doubles as a
cross-implementation correctness check.
## Baseline (first measurement)
20000 items, ~4.2 MB, best of 5, Apple Silicon (Studio), release builds:
| operation | PureXML | libxml2 | ratio |
|---|---|---|---|
| parse | 0.49 s | 0.016 s | ~30x |
| serialize | 0.059 s | 0.012 s | ~5x |
| xpath | 0.42 s | 0.012 s | ~36x |
These are the honest starting numbers the #139 optimization passes burn
down; the table updates as passes land, never silently.
## Current record (size series + real corpora, best of 3)
| corpus | parse | serialize | xpath |
|---|---|---|---|
| generated 4.2 MB | 0.34 s (18.7x) | 0.064 s (5.5x) | 0.15 s (11.9x) |
| generated 42 MB | 3.49 s (18.7x) | 0.63 s (5.6x) | 1.57 s (11.7x) |
| generated 210 MB | 19.3 s (19.7x) | 3.19 s (5.6x) | 9.4 s (libxml2: **refused**) |
| NASA ADC 25 MB (real) | 1.73 s (24.5x) | 0.42 s (8.0x) | 0.76 s (19.4x) |
| SwissProt 115 MB (real) | 11.0 s (22.0x) | 1.77 s (5.5x) | 5.0 s (18.7x) |
Two findings only the large corpora could surface:
- PureXML's document-order sort was quadratic on flat fan-outs (42 MB:
27 s in XPath, ratio 190x). Fixed; and the first fix attempt was itself
quadratic in a worse place (an eager sibling-table built per predicate
evaluation: one hour at 42 MB), caught by sampling the live run. The
cache now early-outs single-node sets and builds a parent's table only
on its second lookup. XPath at 42 MB: 27.1 s -> 1.57 s.
- libxml2 refuses XPath evaluation past its XPATH_MAX_NODESET_LENGTH cap
(10M nodes): the 210 MB corpus query returns NULL by design. PureXML
completes it in 9.4 s. The protective counterpart for hostile input is
tracked as #143.
Scaling: all three PureXML operations are linear in document size (ratios
flat from 4 MB to 210 MB), and there is no 2 GB integer cliff (64-bit
offsets throughout; libxml2's xmlReadMemory takes an int byte count).
## Optimization log
| pass | change | parse | xpath | serialize |
|---|---|---|---|---|
| baseline | - | 0.494 s (30x) | 0.413 s (37x) | 0.061 s (5.5x) |
| 1 | allocation-free Reader.matches() | 0.414 s (26x) | unchanged | unchanged |
| 2 | byte-backed Reader (owned UTF-8 storage, pointer-decoded scalars, byte-level literal match/consume) | 0.375 s (23x) | unchanged | unchanged |
| 3 | bulk content runs (plain-ASCII character data scanned and consumed at the byte level, String built once per run) | 0.335 s (20x) | unchanged | unchanged |
| 4 | byte-level dispatch + name/attribute scanning (non-buffering `peekByte`, byte-mode `skipSpace`, all-ASCII `takeASCIIName`, `attributeRunBytes`); keeps the lookahead buffer empty so the existing byte fast paths engage | 0.217 s (13x) | unchanged | unchanged |
| 5 | `sawAmpersand` short-circuit (ampersand-free text returns verbatim, skipping the reference-decode/split/findings pass) and detecting the text-run-closing `<` via `peekByte` instead of buffering it (the buffered `<` had cascaded the next element's whole dispatch and name scan onto the Character path) | 0.157 s (9.5x) | unchanged | unchanged |
| 6 | byte-level markup dispatch (the single byte after `<` selects end-tag/PI/declaration/start-tag, replacing up to five literal string comparisons per element with one byte peek) | 0.147 s (9.0x) | unchanged | unchanged |
| 7 | `sawAmpersand` short-circuit for attribute values (the byte scanner notes any `&`; an ampersand-free value skips the reference-decode pass and its full re-scan); measured against same-state baseline, every run below every baseline run | 0.143 s (~4% vs baseline) | unchanged | unchanged |
| 8 | serialize escaping fast path (text/attribute/comment/PI return the value unchanged when no character needs escaping, instead of rebuilding it character by character; byte-level scan for the escapable bytes) | unchanged | unchanged | 0.032 s (2.7x, from 5.5x) |
| 9 | skip cross-context node-set de-duplication on a single context or a disjoint axis (child/attribute/namespace/self); the `Set<Node>` hashing and copying was pure overhead where no duplicate can arise, same output (same-state A/B, ~28%) | unchanged | 0.097 s (10x, from 13x) | unchanged |
| 10 | `//` descendant fusion (`descendant-or-self::node()/child::X[P]` compiled to `descendant::X[P]` when `P` is non-positional), avoiding the whole-subtree intermediate context; guarded so positional predicates keep their per-parent meaning, adversarially reviewed for semantic transparency | unchanged | 0.063 s (~7x, from 10x) | unchanged |
| 11 | drop the redundant `Set` de-duplication when sorting a path result: `evaluateSteps` already yields a duplicate-free node-set, so only the document-order sort is needed (each skipped insert was a copy of a node wrapping a side-table-refcounted tree node); same-state A/B, ~9% | unchanged | 0.058 s (~6x, from ~7x) | unchanged |
| 12 | skip the document-order sort for a single forward-axis step from one context: `AxisNavigation` produces child/descendant/descendant-or-self/self/following/following-sibling already in document order with no duplicates, so the sort is redundant (adversarially reviewed for order correctness; same-state A/B, ~7%) | unchanged | 0.055 s (~6x) | unchanged |
| 13 | single-accumulator descendant traversal: `descendants(of:)` recursed on wrapped nodes and concatenated a fresh array at every node (`append(contentsOf: descendants(of: child))`), O(n x depth) allocation churn; rewritten to recurse on the raw tree node with one shared `inout` accumulator, O(n). Same output (same-state A/B, ~46%) | unchanged | 0.032 s (~3x, from ~6x) | unchanged |
| 14 | fuse the node test into the descendant walk: a node the step's test rejects is no longer wrapped in an `XPath.Node` (so never retained); `matches` delegates to a tree-node `matchesTree` so there is one source of truth, and the traversal applies it during the walk. Identical node-set (adversarially equivalence-reviewed; same-state A/B, ~11%) | unchanged | 0.028 s (~2.5x) | unchanged |
| 15 | the same test fusion for the attribute axis (`matchesAttribute`): a `[@name='...']` predicate built every attribute node per element and filtered to one; it now wraps only the matching attribute, saving the owner retain and the qualified-name/value copies for the rest. Identical node-set (adversarially equivalence-reviewed; same-state A/B, ~20%) | unchanged | 0.022 s (~2.0x) | unchanged |
| 16 | share the focus-independent evaluation state via an `Environment` reference: `focused(on:position:size:)` (called once per node of a predicate's input) copied the variables, functions, and namespace dictionaries every time; it now copies only node/position/size and shares one reference. Same semantics (same-state A/B, ~16%) | unchanged | 0.020 s (~2.0x) | unchanged |
| 17 | compile the step's node test once per query instead of re-evaluating it per node: the descendant/attribute walks called `{ matchesTree($0, step.test, on: step.axis, context.namespaces) }`, passing the `namespaces` dictionary (and the `NodeTest`'s associated string) by value on every one of tens of thousands of nodes, so the dominant traversal carried a dictionary retain/release per node. `compiledTreeTest`/`compiledAttributeTest`/`compiledNodeTest` resolve the axis principal kind, the binding state, and the name shape once and return a tight closure capturing only the local name. Equivalent to `matchesTree`/`matchesAttribute`/`matches` (exhaustive differential test + adversarial review; same-state A/B, ~35%) | unchanged | 0.013 s (~1.3x) | unchanged |
| 18 | bulk-copy literal runs in the entity decoder: `EntityExpander.expand` rebuilt every reference-bearing text run one character at a time (`result.append(character)` with a per-character budget charge), so a text node holding a single `&amp;` was copied grapheme by grapheme; it now scans to the next `&`/`<` and appends the literal run in one go (charging the budget by the run length). Also skips the element-content reference-findings pass (an `&` scan plus a qualified-name render per text node) when there is no DTD. Identical decoded text (W3C XML conformance corpora + 6-case differential + adversarial review; same-state A/B, ~5%) | 0.138 s (~8.5x) | unchanged | unchanged |
| 19 | carry the name's first-colon offset out of the byte scanner so `QualifiedName` splits the prefix without a second pass: `takeASCIIName` already visits every name byte, so it records the first `:` (one compare per byte); `QualifiedName(ascii:colonOffset:)` then splits at the known offset, and the common unprefixed name (no colon) takes no scan at all instead of a grapheme-by-grapheme `firstIndex(of:)` over the whole name. Identical names (colon-placement differential + W3C conformance + adversarial review; same-state A/B, ~12%) | 0.124 s (~7.7x) | unchanged | unchanged |
| 20 | adopt prepared children when materializing the tree instead of appending them one at a time: building the mutable `TreeNode` tree from the parsed value `Node` ran `append` per child, which detaches the child from any old parent, checks for a cycle, and grows the children array element by element; the converted children are freshly built and parentless, so `TreeNode.init(adopting:)` takes the prepared array directly and only wires each child's parent. Identical tree (parent-wiring differential + W3C conformance + adversarial review; same-state A/B, ~12%) | 0.106 s (~6.5x) | unchanged | unchanged |
| 21 | scan for the entity decoder's markers at the byte level: `EntityExpander.expand` found the next `&`, `<`, and `;` with `String.firstIndex` over graphemes; since all three are ASCII, a UTF-8 index is a valid string index, so the scan runs over `raw.utf8` and skips decoding every literal byte into a grapheme just to look for a marker. Same decoded text (the bulk + 4 multibyte-boundary cases in the decode differential + W3C conformance; same-state A/B, ~5.5%) | 0.099 s (~6.1x) | unchanged | unchanged |
| 22 | build the mutable `TreeNode` tree directly from the event stream instead of building the value `Node` tree and converting it: `parseTree` ran `TreeNode(parse(...))`, allocating two full trees (the value tree, then a converted reference tree). `Parser.buildTree` is the direct counterpart to `build` (same event loop, stack discipline, and well-formedness guards) that accumulates `TreeNode` children per frame and adopts them when an element closes, so the tree is allocated once. Identical tree (structural differential against the convert path + the whole suite and conformance corpora, which all run through `parseTree`, + adversarial review; same-state A/B, ~8%) | 0.090 s (~5.5x) | unchanged | unchanged |
| 23 | bulk-copy the verbatim runs when escaping output: `Escaping.text`/`attribute` rebuilt a value that needed escaping character by character (`result += String(character)` per unescaped character), so re-escaping a text node that holds a single `&` rebuilt it grapheme by grapheme; they now copy each run between escapable characters in one append and only the markers are rewritten. The `*NeedsEscaping` fast path (no rebuild when nothing needs escaping) is unchanged. Byte-identical output (per-character-oracle differential with multibyte cases + Apache Xalan gold output + C14N suite + adversarial 400k fuzz; same-state A/B, ~7%) | unchanged | unchanged | 0.029 s (~2.8x) |
| 24 | find the characters to escape by scanning bytes, not graphemes: the bulk-escape loop still iterated `Character`s and compared each to `&`/`<`/`>` (a grapheme compare per character); since every trigger is an ASCII marker, or in ASCII-only mode a non-ASCII lead byte, `value.utf8.firstIndex(where:)` locates the next one without decoding the run into graphemes (the marker byte lands on a grapheme boundary, so the string slice and subscript stay valid). Byte-identical output (the oracle differential extended with combining-mark, ZWJ, and flag clusters + the gold/C14N suites + adversarial grapheme-alignment review; same-state A/B, ~23%) | unchanged | unchanged | 0.022 s (~2.2x) |
| 25 | `scanAttributeValue`'s closing-quote check: after the content loop peeks the next character to decide whether it is the closing quote (breaking either on a match or on end of input, the only two cases), `reader.consume(String(quote))` re-derived the same answer through `matches`/`consume`'s general path, which allocates a fresh `String(quote)` and iterates it via `Sequence`/`Character` machinery twice (once to compare, once to count how many times to `advance()`), on every single attribute value in the document. Checking the already-buffered peek directly (`reader.peek() == quote`) and calling `advance()` once is the same observable behavior (same throw on end-of-input, same consumption on a match) for far less work. Verified: full local suite (1852 tests) and the full W3C XSTS suite (14383 groups, ratchet unchanged) pass unchanged. | 0.129 s (fresh session baseline; see note below) | unchanged | unchanged |
| 26 | `NamespaceContext.enterElement`, called once per element: (a) `checkNameShape`'s `name.localName.contains(":")` used a `Character` literal, resolving to `Sequence.contains(_:)`'s grapheme-by-grapheme walk to find one ASCII byte; `localName.utf8.contains(0x3A)` finds the same colon via the UTF-8 byte view, since 0x3A is never a lead or continuation byte of a multi-byte sequence. (b) `attributes.map { try resolveAttribute($0, at: mark) }` allocated a fresh backing array on every element even though `resolveAttribute` returns its argument completely unchanged whenever the attribute has no prefix or the `xmlns` prefix (the common case: most attributes in most documents are unprefixed); checking `attributes.contains { prefix != nil && prefix != "xmlns" }` first (no allocation) and reusing `attributes` directly when nothing would change gives the identical `resolvedAttributes` value for the common case without the allocation. Verified: full local suite and full W3C XSTS suite pass unchanged. | 0.121 s (~6.00x from 0.129 s at pass 25's baseline) | unchanged | unchanged |
Passes 25-26 were measured in a fresh session on different hardware from the original baseline and passes 1-24 (this doc does not know what machine recorded which historical entry), so the absolute seconds are not directly comparable across the gap; the ratio column (parse ~6.59x before pass 25, ~6.30x after pass 25, ~6.00x after pass 26, all at the 20000-item/4.2MB corpus, best of 5) is the honest same-session A/B this log's discipline requires. Reconciling the whole table against one pinned machine is open work, not assumed.
## Profile findings (first sample, parse path)
Top of stack, release build: String.Iterator.next (the per-character pull
closure chain), _stringCompareWithSmolCheck (Character equality in
matches/peek), String._uncheckedFromUTF8 and _allASCII (small-string
materialization), bridgeObjectRelease/swift_release (ARC on Character
buffers), UnicodeScalarView.distance. The Reader's architecture: one
closure call per scalar, [Character] lookahead, Character-typed
classification. The structural fix is a byte-level (UTF-8) scanner with
scalar classification tables; that is its own #139 pass, designed, not
patched.
## Known structure of the gap
- Parse and XPath dominate; serialize is closest.
- The scanner-vs-tree split needs the SAX stage, blocked on #142 (the
public streaming surface is internal today).
- Suspects to profile, in order: Character/grapheme-based scanning versus
byte scanning, per-scalar reader buffering (#135), ARC traffic in tree
construction, XPath node-set materialization on every axis step.
## Rules
- Optimize only what these numbers indict, one suspect per pass, with the
before/after recorded here.
- The corpus generator changes only with a note here, since ratios are
only comparable over identical corpora.
## XSLT transform (quadratic fix, not in the parse/serialize/xpath series)
A representative transform (select `item[@kind='even']` over the 20k-item
corpus, emit two attributes and two child values per item) was quadratic:
`firstInDocumentOrder()`, used to take the string-value of a single-node set
(`string(@id)`, `value-of select="name"`), computed a document-order key for
the one node, and that key scans the node's siblings, a linear pass over the
wide parent. Tens of thousands of single-node extractions each rescanned the
20k-wide parent. Returning the lone node directly (its order is irrelevant)
makes it linear: ~2.3 s -> ~0.65 s (~3.5x), output byte-identical (Xalan
gold-output verified). The fix is the same `count <= 1` early-out that
`sortedByDocumentOrder()` already had.
## XSD identity constraints (quadratic fix, found by per-operation scale-testing)
Validating a wide list against an `xs:unique`/`xs:key`/`xs:keyref` was
quadratic for two independent reasons, both invisible on the default corpus
(which has no identity constraints) and surfaced only by scale-testing the
operation on its own at 2x/4x/8x sizes. First, the duplicate check compared
each new value tuple against every tuple already collected (and a `keyref`
scanned every referenced key), so n targets cost O(n^2). Field equality is
value-space aware (`3.0` equals `3`, `true` equals `1`, a QName equals an
equivalent prefix), so the tuples cannot be hashed directly. The fix keys only
the tuples that provably compare as raw strings (whitespace-preserving lexical
types with no colon, which is most ids) in a `Set`, and keeps the exact
pairwise comparison for value-space, whitespace-collapsing, and QName-bearing
fields, so which documents pass is unchanged. Second, the error-location helper
rescanned the parent's whole child list to position each target (`item[3]`),
another O(n) per target; it now uses a per-parent, per-name index built once.
Together: a 16000-item `xs:unique` document drops from ~4.0 s to ~0.12 s
(~34x), and the cost is now linear. The full local suite and the W3C XSTS
datatype/identity conformance corpus are unchanged.