NodetteHQ/PureXML
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

perf(xpath): compile the step node test once per query (#289)

Browse Source 
The descendant and attribute walks tested every candidate node through a
closure that called matchesTree / matchesAttribute with the step's NodeTest and
the evaluation namespaces dictionary:

    descendants(of: tree) { matchesTree($0, step.test, on: step.axis, context.namespaces) }

That passes the namespaces dictionary (and the NodeTest's associated string) by
value on every call, so a //name-style query over a wide document carried a
dictionary retain/release on each of tens of thousands of nodes, on the
traversal that dominates evaluation.

compiledTreeTest, compiledAttributeTest, and compiledNodeTest resolve the parts
that do not change as the walk moves (the axis principal kind, whether eval-time
bindings are in effect, the prefix-wildcard shape of the name) exactly once and
return a tight per-node closure capturing only the local name. The selected
nodes are identical to the unfused matchesTree / matchesAttribute / matches.

Measured xpath ~35% faster on the 20k-item generated corpus via a same-state
A/B, from ~2.0x to ~1.3x slower than libxml2. The full suite, the Apache Xalan
XSLT gold-output corpus, and the WASM build stay green. XPathCompiledNodeTestTests
is an exhaustive differential asserting each compiled closure equals the
function it specializes across every node, test, axis, and binding.
This commit is contained in:
Mihaela Mihaljevic 2026-06-25 09:44:04 +02:00 committed by GitHub
parent 68f6680a66
commit f6a093fe4b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 209 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGELOG.md
View File

@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- XPath compiles a step's node test once per query instead of re-evaluating its fixed structure on every node. The descendant and attribute walks tested each candidate through a closure that called `matchesTree`/`matchesAttribute` with the step's `NodeTest` and the evaluation `namespaces` dictionary, so the dominant traversal of a `//name`-style query passed (and reference-counted) the namespace dictionary by value on every one of tens of thousands of nodes. `compiledTreeTest`, `compiledAttributeTest`, and `compiledNodeTest` resolve the parts that do not change as the walk moves (the axis principal kind, whether eval-time bindings are in effect, the prefix-wildcard shape of the test name) exactly once, returning a tight per-node closure that captures only the local name. The selected nodes are identical to the unfused `matchesTree`/`matchesAttribute`/`matches` path. Measured `xpath` ~35% faster on the 20k-item generated corpus via a same-machine-state A/B (from ~2.0x to ~1.3x slower than libxml2). Guarded by `XPathCompiledNodeTestTests`, an exhaustive differential over every node, test, axis, and binding asserting the compiled closures equal the functions they specialize, and confirmed by an adversarial equivalence review.
- XPath evaluation shares the focus-independent state (the variable bindings, the function library, the namespace bindings, and the budget) through a single `Environment` reference instead of carrying it by value in every context. Re-focusing the context on each node of a node-set, which happens once per node when a predicate is applied, copied three dictionaries every time; it now copies only the node, position, and size and shares the environment by reference. The values and behavior are unchanged. Measured `xpath` ~16% faster on the 20k-item generated corpus via a same-machine-state A/B; covered by the full XPath suite (the change is internal to the evaluator).
- The XPath attribute axis fuses the step's node test into the walk, the same way the descendant axes do, so an attribute the test rejects is never wrapped in an `XPath.Node`. A predicate like `[@name='value']` previously built an `XPath.Node` for every attribute of the element (each wrap retains the owner element and copies the attribute's qualified name and value) and then filtered to the one named attribute; it now wraps only the matching attribute. `matches` delegates, for an attribute node, to a new `matchesAttribute` (the attribute axis always has the attribute principal kind, so this is exact), and the attribute walk applies it. The node-set, declaration order, and uniqueness are identical to the build-then-filter path. Measured `xpath` ~20% faster on the 20k-item generated corpus via a same-machine-state A/B. Guarded by `XPathFusedAttributeTests` (name and namespaced-name tests, the principal-kind wildcard, `node()`, the kind tests that never select an attribute, and a predicate over the selection) and confirmed by an adversarial equivalence review.

110
Sources/XPath/EvaluatorSteps.swift
View File

@ -56,16 +56,19 @@ extension PureXML.XPath.Evaluator {
if case let .tree(tree) = contextNode {
switch step.axis {
case .descendant:
return PureXML.XPath.AxisNavigation.descendants(of: tree) { matchesTree($0, step.test, on: step.axis, context.namespaces) }
let keep = compiledTreeTest(step.test, on: step.axis, context.namespaces)
return PureXML.XPath.AxisNavigation.descendants(of: tree, where: keep)
case .descendantOrSelf:
return PureXML.XPath.AxisNavigation.descendantsOrSelf(of: tree) { matchesTree($0, step.test, on: step.axis, context.namespaces) }
let keep = compiledTreeTest(step.test, on: step.axis, context.namespaces)
return PureXML.XPath.AxisNavigation.descendantsOrSelf(of: tree, where: keep)
case .attribute:
return PureXML.XPath.AxisNavigation.attributes(of: tree) { matchesAttribute($0, step.test, context.namespaces) }
let keep = compiledAttributeTest(step.test, context.namespaces)
return PureXML.XPath.AxisNavigation.attributes(of: tree, where: keep)
default: break
}
}
return PureXML.XPath.AxisNavigation.nodes(on: step.axis, from: contextNode)
.filter { matches($0, step.test, on: step.axis, namespaces: context.namespaces) }
let keep = compiledNodeTest(step.test, on: step.axis, context.namespaces)
return PureXML.XPath.AxisNavigation.nodes(on: step.axis, from: contextNode).filter(keep)
}
/// Filters nodes through each predicate in turn. A numeric predicate is a
@ -131,6 +134,103 @@ extension PureXML.XPath.Evaluator {
}
}
/// Compiles a tree-node test into a per-node predicate, resolving everything
/// that is invariant across a traversal (the axis principal kind, whether
/// namespace bindings are in effect, the prefix-wildcard shape of the test
/// name) exactly once instead of on every node. A descendant walk over a
/// wide subtree applies the returned closure tens of thousands of times, so
/// hoisting the test's fixed structure out of the per-node path is the bulk
/// of the saving. The closure is equivalent to ``matchesTree`` for the same
/// arguments, the one source of truth it specializes.
static func compiledTreeTest(
_ test: NodeTest,
on axis: Axis,
_ namespaces: [String: String],
) -> (PureXML.Model.TreeNode) -> Bool {
switch test {
case let .name(name):
guard axis.principalKind == .element else { return { _ in false } }
// No eval-time bindings and no prefix wildcard: the match is the
// plain in-document-string comparison, with no per-node namespace or
// suffix work.
if namespaces.isEmpty, !name.hasSuffix(":*") {
return { tree in
tree.kind == .element && tree.name.map { $0.description == name || $0.localName == name } == true
}
}
return { tree in
tree.kind == .element && tree.name.map { qualifiedMatches($0, name, namespaces) } == true
}
case .wildcard:
guard axis.principalKind == .element else { return { _ in false } }
return { $0.kind == .element }
case .text:
return { $0.kind == .text || $0.kind == .cdata }
case .node:
return { _ in true }
case .comment:
return { $0.kind == .comment }
case let .processingInstruction(target):
return { tree in
tree.kind == .processingInstruction && (target == nil || tree.name?.description == target)
}
}
}
/// Compiles an attribute-node test into a per-attribute predicate, hoisting
/// the same invariants as ``compiledTreeTest`` (the binding state and the
/// prefix-wildcard shape) out of the per-attribute path. Equivalent to
/// ``matchesAttribute`` for the same arguments.
static func compiledAttributeTest(
_ test: NodeTest,
_ namespaces: [String: String],
) -> (PureXML.Model.Attribute) -> Bool {
switch test {
case let .name(name):
if namespaces.isEmpty, !name.hasSuffix(":*") {
return { $0.name.description == name || $0.name.localName == name }
}
return { qualifiedMatches($0.name, name, namespaces) }
case .wildcard, .node:
return { _ in true }
case .text, .comment, .processingInstruction:
return { _ in false }
}
}
/// Compiles a general node test (any node kind) into a per-node predicate,
/// for the axes that build a node list and filter it. A tree node is tested
/// by the compiled tree test; an attribute or namespace node keeps the
/// ``matches`` behavior. Equivalent to ``matches`` for the same arguments.
static func compiledNodeTest(
_ test: NodeTest,
on axis: Axis,
_ namespaces: [String: String],
) -> (Node) -> Bool {
let treeTest = compiledTreeTest(test, on: axis, namespaces)
switch test {
case let .name(name):
let kind = axis.principalKind
return { node in
if case let .tree(tree) = node { return treeTest(tree) }
return nameMatches(node, name, kind, namespaces)
}
case .wildcard:
let kind = axis.principalKind
return { node in
if case let .tree(tree) = node { return treeTest(tree) }
return wildcardMatches(node, kind)
}
case .node:
return { _ in true }
case .text, .comment, .processingInstruction:
return { node in
if case let .tree(tree) = node { return treeTest(tree) }
return false
}
}
}
/// The node test applied directly to a tree node, the single-node core of
/// ``matches`` for a `.tree` node. Taking the ``TreeNode`` rather than a
/// wrapped ``Node`` lets the descendant traversal test a candidate without

101
Tests/XPathCompiledNodeTestTests.swift Normal file
View File

@ -0,0 +1,101 @@
import Testing
@testable import PureXML
/// The compiled per-node tests (`compiledTreeTest`, `compiledAttributeTest`,
/// `compiledNodeTest`) hoist a step's invariant structure (the axis principal
/// kind, the binding state, the prefix-wildcard shape) out of the per-node path
/// so a wide traversal resolves it once, not on every node. They must stay
/// exactly equivalent to the unfused `matchesTree`, `matchesAttribute`, and
/// `matches` they specialize: a divergence would make a fused traversal select
/// different nodes than the filter fallback. This walks a document covering
/// every node kind and asserts equivalence across every test, axis, and
/// binding, so a future edit to a `matches*` function that forgets its compiled
/// twin fails here.
@Suite("XPath compiled node-test equivalence")
struct XPathCompiledNodeTestTests {
private static let source =
#"<r xmlns:k="urn:k"><e a="1" k:c="3">text<k:e b="2"/><!--c--><?pi go?></e><k:e/></r>"#
private typealias Eval = PureXML.XPath.Evaluator
private typealias Test = PureXML.XPath.NodeTest
private typealias Axis = PureXML.XPath.Axis
private static let tests: [Test] = [
.name("e"), .name("k:e"), .name("k:c"), .name("k:*"), .name("missing"),
.wildcard, .node, .text, .comment,
.processingInstruction(target: nil), .processingInstruction(target: "pi"),
]
private static let axes: [Axis] = [
.child, .descendant, .descendantOrSelf, .attribute, .namespace,
.selfAxis, .parent, .ancestor, .following, .preceding,
.followingSibling, .precedingSibling, .ancestorOrSelf,
]
private static let bindings: [[String: String]] = [[:], ["k": "urn:k"]]
private func allTreeNodes(_ root: PureXML.Model.TreeNode) -> [PureXML.Model.TreeNode] {
var out = [root]
for child in root.children {
out += allTreeNodes(child)
}
return out
}
@Test("compiledTreeTest equals matchesTree for every node, test, axis, binding")
func test_treeEquivalence() throws {
let nodes = try allTreeNodes(PureXML.parseTree(Self.source))
for namespaces in Self.bindings {
for axis in Self.axes {
for test in Self.tests {
let compiled = Eval.compiledTreeTest(test, on: axis, namespaces)
for node in nodes {
#expect(
compiled(node) == Eval.matchesTree(node, test, on: axis, namespaces),
"tree test=\(test) axis=\(axis) ns=\(namespaces)",
)
}
}
}
}
}
@Test("compiledNodeTest equals matches for tree, attribute, and namespace nodes")
func test_nodeEquivalence() throws {
let trees = try allTreeNodes(PureXML.parseTree(Self.source))
var nodes: [PureXML.XPath.Node] = trees.map { .tree($0) }
for tree in trees {
for attr in tree.attributes {
nodes.append(.attribute(owner: tree, attr))
}
}
nodes.append(.namespace(owner: trees[0], prefix: "k", uri: "urn:k"))
for namespaces in Self.bindings {
for axis in Self.axes {
for test in Self.tests {
let compiled = Eval.compiledNodeTest(test, on: axis, namespaces)
for node in nodes {
#expect(
compiled(node) == Eval.matches(node, test, on: axis, namespaces: namespaces),
"node test=\(test) axis=\(axis) ns=\(namespaces)",
)
}
}
}
}
}
@Test("compiledAttributeTest equals matchesAttribute for every attribute, test, binding")
func test_attributeEquivalence() throws {
let attrs = try allTreeNodes(PureXML.parseTree(Self.source)).flatMap(\.attributes)
for namespaces in Self.bindings {
for test in Self.tests {
let compiled = Eval.compiledAttributeTest(test, namespaces)
for attr in attrs {
#expect(
compiled(attr) == Eval.matchesAttribute(attr, test, namespaces),
"attr test=\(test) ns=\(namespaces)",
)
}
}
}
}
}

1
docs/benchmarks.md
View File

@ -68,6 +68,7 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count).
| 14 | fuse the node test into the descendant walk: a node the step's test rejects is no longer wrapped in an `XPath.Node` (so never retained); `matches` delegates to a tree-node `matchesTree` so there is one source of truth, and the traversal applies it during the walk. Identical node-set (adversarially equivalence-reviewed; same-state A/B, ~11%) | unchanged | 0.028 s (~2.5x) | unchanged |
| 15 | the same test fusion for the attribute axis (`matchesAttribute`): a `[@name='...']` predicate built every attribute node per element and filtered to one; it now wraps only the matching attribute, saving the owner retain and the qualified-name/value copies for the rest. Identical node-set (adversarially equivalence-reviewed; same-state A/B, ~20%) | unchanged | 0.022 s (~2.0x) | unchanged |
| 16 | share the focus-independent evaluation state via an `Environment` reference: `focused(on:position:size:)` (called once per node of a predicate's input) copied the variables, functions, and namespace dictionaries every time; it now copies only node/position/size and shares one reference. Same semantics (same-state A/B, ~16%) | unchanged | 0.020 s (~2.0x) | unchanged |
| 17 | compile the step's node test once per query instead of re-evaluating it per node: the descendant/attribute walks called `{ matchesTree($0, step.test, on: step.axis, context.namespaces) }`, passing the `namespaces` dictionary (and the `NodeTest`'s associated string) by value on every one of tens of thousands of nodes, so the dominant traversal carried a dictionary retain/release per node. `compiledTreeTest`/`compiledAttributeTest`/`compiledNodeTest` resolve the axis principal kind, the binding state, and the name shape once and return a tight closure capturing only the local name. Equivalent to `matchesTree`/`matchesAttribute`/`matches` (exhaustive differential test + adversarial review; same-state A/B, ~35%) | unchanged | 0.013 s (~1.3x) | unchanged |
## Profile findings (first sample, parse path)