From b35f4a17ef75c9c3b3ccbf0dda92272ab0105950 Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Thu, 25 Jun 2026 03:21:03 +0200 Subject: [PATCH] perf(xpath): fuse // into a descendant step for non-positional predicates (#281) `//X` compiles to `descendant-or-self::node()/child::X`, which materializes every node of the subtree as an intermediate context before selecting the X children of each. Fuse that pair into a single `descendant::X` step at compile time: the two select the same node-set, so evaluation skips the intermediate. The fusion is transparent except for a positional predicate: `//x[1]` is the first x child of EACH parent, not the first x in document order, so it must not become `descendant::x[1]`. Apply the rewrite only when every predicate is provably non-positional: its value is a boolean, node-set, or string (never a number, which XPath reads as position() = value) and it nowhere references position() or last(). Anything not provably both is left unfused, so a result can never change. Measured xpath ~10x to ~7x slower than libxml2 on the 20k-item generated corpus via a same-state A/B, identical results. XPathDescendantFusionTests pins the positional forms keeping their per-parent meaning, the non-positional forms selecting across the subtree, and composition with trailing and nested steps. An adversarial differential review (fused versus a spec-literal unfused reference over nine attack suites) found no query whose result changes. The boolean-function allowlist carries a documented invariant: every entry must be position- and size-independent, since the position scan is syntactic. --- CHANGELOG.md | 2 + Sources/XPath/CompilerDescendantFusion.swift | 112 +++++++++++++++++++ Sources/XPath/CompilerPaths.swift | 4 +- Tests/XPathDescendantFusionTests.swift | 51 +++++++++ docs/benchmarks.md | 1 + 5 files changed, 168 insertions(+), 2 deletions(-) create mode 100644 Sources/XPath/CompilerDescendantFusion.swift create mode 100644 Tests/XPathDescendantFusionTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 32862bb..117c8de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- XPath compiles `//` (the abbreviation for `descendant-or-self::node()/child::X`) to a single `descendant::X` step when the child step's predicates are non-positional, avoiding the materialization of every node in the subtree as an intermediate context (measured `xpath` ~10x -> ~7x slower than libxml2 on the 20k-item generated corpus via a same-machine-state A/B, identical results; ~54% faster than before this cycle's two XPath changes combined). The two node-sets are equal, so the fusion is transparent, EXCEPT for a positional predicate: `//x[1]` is the first `x` child of EACH parent, not the first `x` in document order, which `descendant::x[1]` would wrongly select. The rewrite is therefore applied only when every predicate is provably non-positional: its value is a boolean, node-set, or string (never a number, which XPath reads as `position() = value`) and it nowhere references `position()` or `last()`. Anything not provably both is left unfused, so a result can never change. Guarded by `XPathDescendantFusionTests` (positional forms keep their per-parent meaning, non-positional forms select across the subtree, composition with trailing and nested steps) and confirmed by an adversarial differential review (fused versus a spec-literal unfused reference) that found no query whose result the fusion changes. + - XPath location-step evaluation skips cross-context node-set de-duplication when no duplicate can arise (measured `xpath` ~13x -> ~10x slower than libxml2 on the 20k-item generated corpus via a same-machine-state A/B, ~28% faster, identical results). A step accumulated each context node's axis results into a `Set` to drop nodes reached from more than one context, which hashed and copied every node (the wrapped `TreeNode` reference is reference-counted, so each copy is ARC traffic). De-duplication is needed only when two distinct context nodes can reach the same node, so it is now skipped on a single context and on the axes whose results are disjoint per context (`child`, `attribute`, `namespace`, `self`); those produce the identical node sequence the `Set` would (every insertion succeeds) without the hashing. The `Set` is kept on the axes that genuinely overlap (`descendant`, `ancestor`, `parent`, the siblings, `following`/`preceding`). Predicates are applied per context (unchanged) and the path result still passes through the final order-and-unique pass, so semantics are preserved. Guarded by `XPathNodeSetDedupTests`, which checks that a node shared by many contexts through `ancestor`/`parent`/`descendant` is still counted once, that the disjoint axes return every distinct result, and that an overlapping union still de-duplicates; the full XPath suite and the Apache Xalan XSLT gold-output corpus stay green. - Serialization roughly halved (measured `serialize` ~5.5x -> ~2.7x slower than libxml2 on the 20k-item generated corpus, `bash scripts/benchmark.sh 20000 5`, with byte-identical output). The escaping functions (`text`, `attribute`, `comment`, `processingInstruction`) rebuilt every value character by character even when nothing needed escaping, which is the common case. They now scan the value's UTF-8 bytes for the escapable bytes (markup characters, the active attribute quote, the whitespace characters, a comment hyphen, a PI question mark, and any non-ASCII byte in ASCII-only mode) and return the value unchanged when none is present, falling back to the original character path only when something must be escaped. Guarded by `EscapingFastPathTests`, which pins both the unchanged fast path and the escaped slow path for each function, and verified against the Apache Xalan XSLT gold-output corpus (a serialize-heavy differential). No output changed. diff --git a/Sources/XPath/CompilerDescendantFusion.swift b/Sources/XPath/CompilerDescendantFusion.swift new file mode 100644 index 0000000..2135cc1 --- /dev/null +++ b/Sources/XPath/CompilerDescendantFusion.swift @@ -0,0 +1,112 @@ +extension PureXML.XPath.Compiler { + /// Fuses each `descendant-or-self::node()` step immediately followed by a + /// `child::X` step into a single `descendant::X` step. This is the standard + /// `//` rewrite: `descendant-or-self::node()/child::X` selects exactly the + /// `X` descendants of the context, the same node-set as `descendant::X`, but + /// without materializing every node of the subtree as an intermediate + /// context. The compiled step list is rewritten once, so every evaluation of + /// the query benefits. + /// + /// The rewrite is applied only when the child step's predicates are + /// non-positional. A positional predicate filters relative to each parent's + /// own child list (`//x[1]` is the first `x` child of EACH element, not the + /// first `x` in document order), which the `descendant` axis cannot + /// reproduce, so such a pair is left exactly as the grammar produced it. + static func fuseDescendantSteps(_ steps: [Step]) -> [Step] { + guard steps.count >= 2 else { return steps } + var result: [Step] = [] + result.reserveCapacity(steps.count) + var index = 0 + while index < steps.count { + let step = steps[index] + if index + 1 < steps.count, isDescendantOrSelfNode(step) { + let child = steps[index + 1] + if child.axis == .child, child.predicates.allSatisfy(isNonPositionalPredicate) { + result.append(Step(axis: .descendant, test: child.test, predicates: child.predicates)) + index += 2 + continue + } + } + result.append(step) + index += 1 + } + return result + } + + /// The bare `descendant-or-self::node()` step that `//` compiles to. + private static func isDescendantOrSelfNode(_ step: Step) -> Bool { + step.axis == .descendantOrSelf && step.test == .node && step.predicates.isEmpty + } + + /// Whether `predicate` is safe to carry from a `child` axis onto a + /// `descendant` axis. Safe exactly when it is not a positional filter: its + /// value is provably a boolean, node-set, or string (never a number, which + /// XPath reads as `position() = value`), AND it nowhere references + /// `position()` or `last()` (so `[position() < 3]`, a boolean, is still + /// excluded). Anything not provably both is treated as positional and left + /// unfused, so the optimization can never change a result. + private static func isNonPositionalPredicate(_ predicate: Expression) -> Bool { + isBooleanValued(predicate) && !referencesPosition(predicate) + } + + /// Whether the expression's value is provably a boolean, node-set, or string + /// (never a number). A relational or logical operator and the known + /// boolean-returning functions yield booleans; a path, filter, or union + /// yields a node-set; a literal string yields a string. A number, a + /// variable (type unknown at compile time), a unary minus, an arithmetic + /// operator, and an unrecognized function are not provably non-numeric. + private static func isBooleanValued(_ expression: Expression) -> Bool { + switch expression { + case let .binary(binaryOperator, _, _): + switch binaryOperator { + case .logicalOr, .logicalAnd, .equal, .notEqual, .lessThan, .lessOrEqual, .greaterThan, .greaterOrEqual: + true + case .add, .subtract, .multiply, .divide, .modulo: + false + } + case .union, .path, .filter, .string: + true + case let .function(name, _): + booleanReturningFunctions.contains(name) + case .number, .variable, .negate: + false + } + } + + /// The built-in functions that return a boolean (so a predicate calling one + /// is a boolean test, never a position test). INVARIANT: every name here must + /// be position- and size-independent. `referencesPosition` scans only for the + /// `position()`/`last()` syntax, so a function that reads the context position + /// or size INTERNALLY (without taking it as a visible argument) would defeat + /// the guard and let a positional predicate ride the descendant axis. Do not + /// add such a function to this set. + private static let booleanReturningFunctions: Set = [ + "boolean", "not", "true", "false", "lang", "contains", "starts-with", + ] + + /// Whether the expression references `position()` or `last()` anywhere, + /// scanned conservatively through every sub-expression (including nested + /// path and filter predicates, whose own context the functions actually + /// bind to). Over-inclusive on purpose: a false positive only declines a + /// fusion, never produces a wrong result. + private static func referencesPosition(_ expression: Expression) -> Bool { + switch expression { + case let .function(name, arguments): + name == "position" || name == "last" || arguments.contains(where: referencesPosition) + case let .binary(_, lhs, rhs): + referencesPosition(lhs) || referencesPosition(rhs) + case let .negate(operand): + referencesPosition(operand) + case let .union(lhs, rhs): + referencesPosition(lhs) || referencesPosition(rhs) + case let .path(_, steps): + steps.contains { $0.predicates.contains(where: referencesPosition) } + case let .filter(primary, predicates, steps): + referencesPosition(primary) + || predicates.contains(where: referencesPosition) + || steps.contains { $0.predicates.contains(where: referencesPosition) } + case .number, .string, .variable: + false + } + } +} diff --git a/Sources/XPath/CompilerPaths.swift b/Sources/XPath/CompilerPaths.swift index b9fce1e..4d46c57 100644 --- a/Sources/XPath/CompilerPaths.swift +++ b/Sources/XPath/CompilerPaths.swift @@ -11,7 +11,7 @@ extension PureXML.XPath.Compiler { skipSpace() guard try isPrimaryStart() else { let (absolute, steps) = try parseLocationPath() - return .path(absolute: absolute, steps: steps) + return .path(absolute: absolute, steps: Self.fuseDescendantSteps(steps)) } let primary = try parsePrimary() let predicates = try parsePredicates() @@ -28,7 +28,7 @@ extension PureXML.XPath.Compiler { if predicates.isEmpty, steps.isEmpty { return primary } - return .filter(primary: primary, predicates: predicates, steps: steps) + return .filter(primary: primary, predicates: predicates, steps: Self.fuseDescendantSteps(steps)) } private mutating func isPrimaryStart() throws -> Bool { diff --git a/Tests/XPathDescendantFusionTests.swift b/Tests/XPathDescendantFusionTests.swift new file mode 100644 index 0000000..2a68571 --- /dev/null +++ b/Tests/XPathDescendantFusionTests.swift @@ -0,0 +1,51 @@ +import Testing +@testable import PureXML + +/// `//X` compiles to `descendant-or-self::node()/child::X` and is fused to +/// `descendant::X` for speed. The fusion is valid only for non-positional +/// predicates: `//X[1]` is the first `X` child of EACH parent, which is NOT the +/// same node-set as `descendant::X[1]` (the first `X` in document order). These +/// tests pin that the positional forms keep their per-parent meaning (unfused) +/// while the non-positional forms select correctly (fused). +@Suite("XPath // descendant fusion") +struct XPathDescendantFusionTests { + /// Two `x` children under `a`, one under `b`; ids in document order 1,2,3. + private static let source = #""# + + private func count(_ path: String) throws -> Int { + try Int(PureXML.XPath.Query("count(\(path))").value(at: PureXML.parseTree(Self.source)).number) + } + + private func ids(_ path: String) throws -> [String] { + try PureXML.XPath.Query(path).evaluate(over: PureXML.parse(Self.source)).compactMap { + if case let .node(node) = $0 { return node.element?.attributes.first { $0.name.localName == "id" }?.value } + return nil + } + } + + @Test("a positional predicate under // keeps its per-parent meaning") + func test_positionalNotFused() throws { + // First x child of EACH parent: x1 (under a) and x3 (under b), not just x1. + #expect(try count("//x[1]") == 2) + #expect(try ids("//x[1]") == ["1", "3"]) + // Last x child of each parent: x2 (under a) and x3 (under b). + #expect(try ids("//x[last()]") == ["2", "3"]) + // position()=2 child of each parent: only a has a second x (x2). + #expect(try ids("//x[position()=2]") == ["2"]) + } + + @Test("a non-positional predicate under // selects across the whole subtree") + func test_nonPositionalFused() throws { + #expect(try ids("//x[@id='2']") == ["2"]) + #expect(try count("//x") == 3) + #expect(try ids("//x[@id='1' or @id='3']") == ["1", "3"]) + } + + @Test("// fusion composes with later steps and nested paths") + func test_fusionWithTrailingSteps() throws { + // //a/x is the x children of every a; a single a here, two x. + #expect(try count("//a/x") == 2) + // A nested positional predicate binds to its own context, not the outer. + #expect(try count("//a[x[1]]") == 1) + } +} diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 317f10f..1a86d45 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -61,6 +61,7 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count). | 7 | `sawAmpersand` short-circuit for attribute values (the byte scanner notes any `&`; an ampersand-free value skips the reference-decode pass and its full re-scan); measured against same-state baseline, every run below every baseline run | 0.143 s (~4% vs baseline) | unchanged | unchanged | | 8 | serialize escaping fast path (text/attribute/comment/PI return the value unchanged when no character needs escaping, instead of rebuilding it character by character; byte-level scan for the escapable bytes) | unchanged | unchanged | 0.032 s (2.7x, from 5.5x) | | 9 | skip cross-context node-set de-duplication on a single context or a disjoint axis (child/attribute/namespace/self); the `Set` hashing and copying was pure overhead where no duplicate can arise, same output (same-state A/B, ~28%) | unchanged | 0.097 s (10x, from 13x) | unchanged | +| 10 | `//` descendant fusion (`descendant-or-self::node()/child::X[P]` compiled to `descendant::X[P]` when `P` is non-positional), avoiding the whole-subtree intermediate context; guarded so positional predicates keep their per-parent meaning, adversarially reviewed for semantic transparency | unchanged | 0.063 s (~7x, from 10x) | unchanged | ## Profile findings (first sample, parse path)