perf(parse): build the TreeNode tree directly from the event stream (#295)
parseTree ran TreeNode(parse(...)), which allocates two full trees: the parser builds an immutable value Node tree, then TreeNode.init(_:) walks it to build the mutable reference tree. The parsed tree is the deliverable, so building the value tree first is pure overhead. Parser.buildTree is the direct counterpart to build: the same iterative event loop with the same element stack and the same well-formedness guards (an unmatched end tag throws unexpectedEndOfInput; a document with no element throws emptyDocument), but it accumulates TreeNode children per frame and adopts them when an element closes, so the tree is allocated once. The produced tree is identical: same kinds, names, attributes, values, child order, and parent links. Measured parse ~8% faster on the 20k-item generated corpus via a same-state A/B (6.1x to ~5.5x slower than libxml2). Verified by ParseTreeDirectBuildTests (a structural differential against the convert-from-Node path, plus throw-parity and a serialized round-trip), the full suite and the W3C XML conformance, RELAX NG, and Xalan corpora (which all build trees through parseTree), the WASM build, and an adversarial equivalence review.
This commit is contained in:
GitHub
parent
6be96e51ce
commit
7d22194614
@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Changed
|
||||
|
||||
- `parseTree` assembles the mutable `TreeNode` tree directly from the event stream instead of building the immutable value `Node` tree and converting it. The old path allocated two full trees: the parser built a value `Node` tree, then `TreeNode.init(_:)` walked it to build the reference tree. `Parser.buildTree` is the direct counterpart to the value-tree assembler, the same iterative event loop with the same element stack and the same well-formedness guards (it throws on an unmatched end tag and on a document with no element), but it accumulates `TreeNode` children per frame and adopts them when the element closes, so the tree is allocated once. The produced tree is identical, the same node kinds, names, attributes, values, child order, and parent links. Measured `parse` ~8% faster on the 20k-item generated corpus via a same-machine-state A/B. Verified by a structural differential against the convert-from-`Node` path (`ParseTreeDirectBuildTests`), the full suite and the W3C XML conformance, RELAX NG, and Apache Xalan corpora (which all materialize trees through `parseTree`), and an adversarial equivalence review.
|
||||
|
||||
- The entity decoder finds its markers at the byte level. `EntityExpander.expand` located the next `&`, `<`, and `;` with `String.firstIndex` over graphemes, decoding every literal byte of the run into a grapheme just to look for a one-byte marker. All three markers are ASCII, so a UTF-8 index is a valid string index: the scan now runs over `raw.utf8` and the found index still subscripts the string directly. The decoded text is unchanged, including text where the markers follow multibyte UTF-8 (precomposed, combining, and astral characters all keep the marker on a grapheme boundary). Measured `parse` ~5.5% faster on the 20k-item generated corpus via a same-machine-state A/B. Verified by the decode differential (`EntityDecodeBulkTests`, extended with multibyte-boundary cases) and the W3C XML conformance corpora.
|
||||
|
||||
- Materializing the mutable `TreeNode` tree from a parsed value `Node` adopts each node's prepared children directly instead of appending them one at a time. Building a node ran `append` for every child, which detaches the child from any previous parent, rejects a cycle, and grows the children array element by element. The children produced by the value-`Node` conversion are freshly built and parentless, so those guards are always no-ops and the repeated growth is wasted; `TreeNode.init(adopting:)` adopts the already-built array and only wires each child's parent. The resulting tree is identical (same children in the same order, the same parent links). The adopting initializer is used only by the conversion; every public factory keeps the detaching, cycle-checking `append` path for externally supplied children. Measured `parse` ~12% faster on the 20k-item generated corpus via a same-machine-state A/B. Verified by a parent-wiring differential (`TreeNodeAdoptingInitTests`), the W3C XML conformance corpora, and an adversarial equivalence review.
|
||||
|
||||
@ -165,7 +165,7 @@ public extension PureXML {
|
||||
limits: Parsing.Limits = .default,
|
||||
resolver: Parsing.EntityResolver = .refusing,
|
||||
) throws -> Model.TreeNode {
|
||||
try Model.TreeNode(parse(xml, limits: limits, resolver: resolver))
|
||||
try Parsing.Parser().parseTree(xml, limits: limits, resolver: resolver)
|
||||
}
|
||||
|
||||
/// Reads a possibly-invalid document without throwing, returning the maximal
|
||||
|
||||
84
Sources/Parsing/ParserTreeBuild.swift
Normal file
84
Sources/Parsing/ParserTreeBuild.swift
Normal file
@ -0,0 +1,84 @@
|
||||
/// A partially-built ``TreeNode`` element while its children stream in, the
|
||||
/// direct-tree counterpart to `ElementFrame`. File-scope and private.
|
||||
private struct TreeNodeFrame {
|
||||
let name: PureXML.Model.QualifiedName
|
||||
let attributes: [PureXML.Model.Attribute]
|
||||
var children: [PureXML.Model.TreeNode] = []
|
||||
}
|
||||
|
||||
public extension PureXML.Parsing.Parser {
|
||||
/// Parses a string straight into the mutable, parent-aware ``TreeNode`` tree,
|
||||
/// assembling it directly from the event stream instead of building the value
|
||||
/// ``Node`` tree and converting it. The result is identical to
|
||||
/// ``Model/TreeNode/init(_:)`` applied to ``parse(_:limits:resolver:)``, the
|
||||
/// same nodes with the same parent links, but the tree is allocated once
|
||||
/// rather than twice.
|
||||
func parseTree(
|
||||
_ xml: String,
|
||||
limits: PureXML.Parsing.Limits = .default,
|
||||
resolver: PureXML.Parsing.EntityResolver = .refusing,
|
||||
) throws -> PureXML.Model.TreeNode {
|
||||
try buildTree(PureXML.Parsing.EventReader(xml, limits: limits, resolver: resolver))
|
||||
}
|
||||
|
||||
/// The direct-``TreeNode`` counterpart to `build(_:)`: same event loop, same
|
||||
/// stack discipline, same well-formedness guards, but it accumulates
|
||||
/// ``TreeNode`` children per frame and adopts them when an element closes, so
|
||||
/// it never materializes the intermediate value ``Node`` tree. Each produced
|
||||
/// tree is identical to converting `build(_:)`'s node.
|
||||
internal func buildTree(_ source: PureXML.Parsing.EventReader) throws -> PureXML.Model.TreeNode {
|
||||
var reader = source
|
||||
var roots: [PureXML.Model.TreeNode] = []
|
||||
var stack: [TreeNodeFrame] = []
|
||||
var produced = false
|
||||
|
||||
while let event = try reader.next() {
|
||||
produced = true
|
||||
switch event {
|
||||
case let .startElement(name, attributes):
|
||||
stack.append(TreeNodeFrame(name: name, attributes: attributes))
|
||||
case .endElement:
|
||||
guard let frame = stack.popLast() else {
|
||||
throw PureXML.Parsing.ParseError.unexpectedEndOfInput(.start)
|
||||
}
|
||||
let element = PureXML.Model.TreeNode(
|
||||
adopting: .element,
|
||||
name: frame.name,
|
||||
attributes: frame.attributes,
|
||||
children: frame.children,
|
||||
)
|
||||
attachTree(element, to: &stack, roots: &roots)
|
||||
case let .characters(text):
|
||||
attachTree(PureXML.Model.TreeNode(kind: .text, value: text), to: &stack, roots: &roots)
|
||||
case let .cdata(text):
|
||||
attachTree(PureXML.Model.TreeNode(kind: .cdata, value: text), to: &stack, roots: &roots)
|
||||
case let .comment(text):
|
||||
attachTree(PureXML.Model.TreeNode(kind: .comment, value: text), to: &stack, roots: &roots)
|
||||
case let .processingInstruction(target, data):
|
||||
let instruction = PureXML.Model.TreeNode(
|
||||
kind: .processingInstruction,
|
||||
name: PureXML.Model.QualifiedName(target),
|
||||
value: data,
|
||||
)
|
||||
attachTree(instruction, to: &stack, roots: &roots)
|
||||
}
|
||||
}
|
||||
|
||||
guard produced, roots.contains(where: { $0.kind == .element }) else {
|
||||
throw PureXML.Parsing.ParseError.emptyDocument
|
||||
}
|
||||
return PureXML.Model.TreeNode(adopting: .document, children: roots)
|
||||
}
|
||||
|
||||
private func attachTree(
|
||||
_ node: PureXML.Model.TreeNode,
|
||||
to stack: inout [TreeNodeFrame],
|
||||
roots: inout [PureXML.Model.TreeNode],
|
||||
) {
|
||||
if stack.isEmpty {
|
||||
roots.append(node)
|
||||
} else {
|
||||
stack[stack.count - 1].children.append(node)
|
||||
}
|
||||
}
|
||||
}
|
||||
67
Tests/ParseTreeDirectBuildTests.swift
Normal file
67
Tests/ParseTreeDirectBuildTests.swift
Normal file
@ -0,0 +1,67 @@
|
||||
import Testing
|
||||
@testable import PureXML
|
||||
|
||||
/// `parseTree` now assembles the `TreeNode` tree directly from the event stream
|
||||
/// (`Parser.buildTree`) instead of building the value `Node` tree and converting
|
||||
/// it. These pin that the direct build is structurally identical to the old
|
||||
/// path, `TreeNode(parse(xml))`, for every node kind, attribute set, nesting,
|
||||
/// namespace, mixed-content, and entity shape, including the parent links, and
|
||||
/// that it still throws on the same malformed inputs.
|
||||
@Suite("parseTree direct-build equivalence")
|
||||
struct ParseTreeDirectBuildTests {
|
||||
private static let documents = [
|
||||
#"<r/>"#,
|
||||
#"<r></r>"#,
|
||||
#"<r a="1" b="2"/>"#,
|
||||
#"<r xmlns:m="urn:m"><m:c m:x="1"/><c/></r>"#,
|
||||
#"<r>text</r>"#,
|
||||
#"<r>mixed <b>bold</b> and <i>italic</i> tail</r>"#,
|
||||
#"<r>a & b < c A B</r>"#,
|
||||
#"<r><![CDATA[<raw> & stuff]]></r>"#,
|
||||
#"<r><!-- comment --><?pi data?><c/></r>"#,
|
||||
#"<?xml version="1.0"?><!-- pre --><r><deep><deeper><x>v</x></deeper></deep></r><!-- post -->"#,
|
||||
#"<r> <a/> <b/> \#n</r>"#,
|
||||
#"<doc><p>one</p><p>two</p><p>three</p></doc>"#,
|
||||
]
|
||||
|
||||
/// Recursively asserts two trees are structurally identical and that each
|
||||
/// child's parent points back to its container in both.
|
||||
private func assertSameTree(_ direct: PureXML.Model.TreeNode, _ converted: PureXML.Model.TreeNode) {
|
||||
#expect(direct.kind == converted.kind)
|
||||
#expect(direct.name == converted.name)
|
||||
#expect(direct.value == converted.value)
|
||||
#expect(direct.attributes == converted.attributes)
|
||||
#expect(direct.children.count == converted.children.count)
|
||||
for (directChild, convertedChild) in zip(direct.children, converted.children) {
|
||||
#expect(directChild.parent === direct)
|
||||
#expect(convertedChild.parent === converted)
|
||||
assertSameTree(directChild, convertedChild)
|
||||
}
|
||||
}
|
||||
|
||||
@Test("the direct build matches the convert-from-Node build for every document")
|
||||
func test_structuralEquivalence() throws {
|
||||
for document in Self.documents {
|
||||
let direct = try PureXML.parseTree(document)
|
||||
let converted = try PureXML.Model.TreeNode(PureXML.parse(document))
|
||||
assertSameTree(direct, converted)
|
||||
}
|
||||
}
|
||||
|
||||
@Test("the direct build rejects the same malformed inputs")
|
||||
func test_sameRejections() {
|
||||
for malformed in ["", "<r>", "</r>", "<!-- only a comment -->", "<a><b></a></b>", "<r></s>"] {
|
||||
let directThrew = (try? PureXML.parseTree(malformed)) == nil
|
||||
let oldThrew = (try? PureXML.Model.TreeNode(PureXML.parse(malformed))) == nil
|
||||
#expect(directThrew == oldThrew, "rejection mismatch for \(malformed.debugDescription)")
|
||||
}
|
||||
}
|
||||
|
||||
@Test("a serialized round-trip through the direct tree is unchanged")
|
||||
func test_roundTrip() throws {
|
||||
let document = #"<r xmlns:m="urn:m"><m:c a="1">x & y</m:c><d><![CDATA[z]]></d></r>"#
|
||||
let direct = try PureXML.parseTree(document)
|
||||
let converted = try PureXML.Model.TreeNode(PureXML.parse(document))
|
||||
#expect(PureXML.serialize(direct.node) == PureXML.serialize(converted.node))
|
||||
}
|
||||
}
|
||||
@ -73,6 +73,7 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count).
|
||||
| 19 | carry the name's first-colon offset out of the byte scanner so `QualifiedName` splits the prefix without a second pass: `takeASCIIName` already visits every name byte, so it records the first `:` (one compare per byte); `QualifiedName(ascii:colonOffset:)` then splits at the known offset, and the common unprefixed name (no colon) takes no scan at all instead of a grapheme-by-grapheme `firstIndex(of:)` over the whole name. Identical names (colon-placement differential + W3C conformance + adversarial review; same-state A/B, ~12%) | 0.124 s (~7.7x) | unchanged | unchanged |
|
||||
| 20 | adopt prepared children when materializing the tree instead of appending them one at a time: building the mutable `TreeNode` tree from the parsed value `Node` ran `append` per child, which detaches the child from any old parent, checks for a cycle, and grows the children array element by element; the converted children are freshly built and parentless, so `TreeNode.init(adopting:)` takes the prepared array directly and only wires each child's parent. Identical tree (parent-wiring differential + W3C conformance + adversarial review; same-state A/B, ~12%) | 0.106 s (~6.5x) | unchanged | unchanged |
|
||||
| 21 | scan for the entity decoder's markers at the byte level: `EntityExpander.expand` found the next `&`, `<`, and `;` with `String.firstIndex` over graphemes; since all three are ASCII, a UTF-8 index is a valid string index, so the scan runs over `raw.utf8` and skips decoding every literal byte into a grapheme just to look for a marker. Same decoded text (the bulk + 4 multibyte-boundary cases in the decode differential + W3C conformance; same-state A/B, ~5.5%) | 0.099 s (~6.1x) | unchanged | unchanged |
|
||||
| 22 | build the mutable `TreeNode` tree directly from the event stream instead of building the value `Node` tree and converting it: `parseTree` ran `TreeNode(parse(...))`, allocating two full trees (the value tree, then a converted reference tree). `Parser.buildTree` is the direct counterpart to `build` (same event loop, stack discipline, and well-formedness guards) that accumulates `TreeNode` children per frame and adopts them when an element closes, so the tree is allocated once. Identical tree (structural differential against the convert path + the whole suite and conformance corpora, which all run through `parseTree`, + adversarial review; same-state A/B, ~8%) | 0.090 s (~5.5x) | unchanged | unchanged |
|
||||
|
||||
## Profile findings (first sample, parse path)
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user