perf(parse): bulk-copy literal runs in the entity decoder (#291)
EntityExpander.expand rebuilt every reference-bearing text run one character at a time: result.append(character) per character, with a per-character charge against the amplification budget. A text node holding a single & was copied grapheme by grapheme, which is the bulk of the per-text-node cost in a document whose content uses entities. Scan to the next '&' or '<' and append the literal run in one operation, charging the budget by the run length. The decoded text, the CDATA-verbatim shielding, and the billion-laughs budget enforcement are unchanged: the same overrun throws at the same input, since charging a run of length L against budget B fails exactly when L > B, the same condition the per-character guard enforced. Also return early from recordReferenceContentFindings when there are no DTD content models (the common no-DTD case): it otherwise scanned each text run for '&' and rendered the enclosing element's qualified name before the model lookup that would fail anyway. Measured parse ~5% faster on the 20k-item generated corpus via a same-state A/B (9.06x to ~8.5x slower than libxml2). Verified by the W3C XML conformance corpora, the new EntityDecodeBulkTests six-case decode differential, the full suite, the Xalan XSLT gold-output corpus, and the WASM build, with an adversarial equivalence review of the rewrite.
This commit is contained in:
parent
c62e02f49d
commit
2cca4912ff
@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
|
||||
- The entity decoder copies the literal text between references in bulk instead of one character at a time. `EntityExpander.expand` rebuilt every reference-bearing run grapheme by grapheme (`result.append(character)` with a per-character amplification-budget charge), so a text node holding a single `&` was reconstructed character by character; it now scans to the next `&` or `<` and appends the run in one operation, charging the budget by the run length. The decoded text, the CDATA-verbatim handling, and the billion-laughs budget enforcement are unchanged (the same overrun throws at the same input). The element-content reference-findings pass, which scanned each text run for `&` and rendered the enclosing element's qualified name, now returns immediately when the document has no DTD content models, which is the common case. Measured `parse` ~5% faster on the 20k-item generated corpus (whose text uses `&`) via a same-machine-state A/B. Verified by the W3C XML conformance corpora, a six-case decode differential (`EntityDecodeBulkTests`: references in runs, decimal and hex character references, declared-entity replacement with budget overrun, CDATA in replacement text, an unterminated reference), and an adversarial equivalence review.
|
||||
|
||||
## [0.4.1] - 2026-06-25
|
||||
|
||||
### Changed
|
||||
|
||||
@ -182,27 +182,38 @@ private struct EntityExpander {
|
||||
mutating func expand(_ raw: String, visiting: Set<String>, counts: Bool) throws {
|
||||
var iterator = raw.startIndex
|
||||
while iterator < raw.endIndex {
|
||||
let character = raw[iterator]
|
||||
// A CDATA section inside replacement text shields its content from
|
||||
// reference recognition: copy it verbatim through the terminator.
|
||||
if character == "<", raw[iterator...].hasPrefix("<![CDATA[") {
|
||||
let openEnd = raw.index(iterator, offsetBy: 9)
|
||||
var end = raw.endIndex
|
||||
var cursor = openEnd
|
||||
while cursor < raw.endIndex {
|
||||
if raw[cursor...].hasPrefix("]]>") {
|
||||
end = raw.index(cursor, offsetBy: 3)
|
||||
break
|
||||
}
|
||||
cursor = raw.index(after: cursor)
|
||||
}
|
||||
for verbatim in raw[iterator ..< end] {
|
||||
try append(verbatim, counts: counts)
|
||||
}
|
||||
iterator = end
|
||||
continue
|
||||
// Copy the literal run up to the next reference or markup character
|
||||
// in bulk rather than one character at a time: only '&' (a
|
||||
// reference) and '<' (a possible CDATA section) need per-character
|
||||
// handling, and the text between them is the bulk of any run.
|
||||
guard let special = raw[iterator...].firstIndex(where: { $0 == "&" || $0 == "<" }) else {
|
||||
try appendRun(raw[iterator...], counts: counts)
|
||||
return
|
||||
}
|
||||
guard character == "&" else {
|
||||
if special > iterator {
|
||||
try appendRun(raw[iterator ..< special], counts: counts)
|
||||
iterator = special
|
||||
}
|
||||
let character = raw[iterator]
|
||||
if character == "<" {
|
||||
// A CDATA section inside replacement text shields its content
|
||||
// from reference recognition: copy it verbatim through the
|
||||
// terminator. A '<' that does not open CDATA is literal content.
|
||||
if raw[iterator...].hasPrefix("<![CDATA[") {
|
||||
let openEnd = raw.index(iterator, offsetBy: 9)
|
||||
var end = raw.endIndex
|
||||
var cursor = openEnd
|
||||
while cursor < raw.endIndex {
|
||||
if raw[cursor...].hasPrefix("]]>") {
|
||||
end = raw.index(cursor, offsetBy: 3)
|
||||
break
|
||||
}
|
||||
cursor = raw.index(after: cursor)
|
||||
}
|
||||
try appendRun(raw[iterator ..< end], counts: counts)
|
||||
iterator = end
|
||||
continue
|
||||
}
|
||||
try append(character, counts: counts)
|
||||
iterator = raw.index(after: iterator)
|
||||
continue
|
||||
@ -216,6 +227,22 @@ private struct EntityExpander {
|
||||
}
|
||||
}
|
||||
|
||||
/// Appends a verbatim run, charging its whole length against the
|
||||
/// amplification budget at once when counting (the per-character
|
||||
/// ``append(_:counts:)`` charges one at a time; for a run the effect is
|
||||
/// identical, the budget reaching the same value and the same overrun
|
||||
/// throwing, only without the per-character work).
|
||||
private mutating func appendRun(_ run: Substring, counts: Bool) throws {
|
||||
if counts {
|
||||
let length = run.count
|
||||
guard budget >= length else {
|
||||
throw PureXML.Parsing.ParseError.amplificationLimitExceeded(mark)
|
||||
}
|
||||
budget -= length
|
||||
}
|
||||
result += run
|
||||
}
|
||||
|
||||
private mutating func resolve(_ body: String, visiting: Set<String>, counts: Bool) throws {
|
||||
if let predefined = Self.predefinedValue(body) {
|
||||
try append(predefined, counts: counts)
|
||||
|
||||
@ -90,6 +90,11 @@ extension PureXML.Parsing.EventReader {
|
||||
/// (which is character data, not ignorable whitespace) inside element
|
||||
/// content.
|
||||
mutating func recordReferenceContentFindings(raw: String, decoded: String) {
|
||||
// Element-content reference findings exist only against a declared
|
||||
// content model. With no DTD (the common case) there is nothing to
|
||||
// check, so return before scanning `raw` for '&' or rendering the
|
||||
// element's qualified name.
|
||||
guard !documentType.elementModels.isEmpty else { return }
|
||||
guard raw.contains("&"), let current = open.last,
|
||||
let model = documentType.elementModels[current.localName] ?? documentType.elementModels[current.description]
|
||||
else { return }
|
||||
|
||||
77
Tests/EntityDecodeBulkTests.swift
Normal file
77
Tests/EntityDecodeBulkTests.swift
Normal file
@ -0,0 +1,77 @@
|
||||
import Testing
|
||||
@testable import PureXML
|
||||
|
||||
/// `EntityExpander.expand` copies the literal text between references in bulk
|
||||
/// rather than one character at a time. These pin that the bulk path produces
|
||||
/// exactly the per-character result it replaced: references at the start, end,
|
||||
/// and adjacent; literal runs of every length around them; decimal and hex
|
||||
/// character references; declared-entity replacement (the budget-counted path,
|
||||
/// including a bulk run that overflows the amplification budget); and a CDATA
|
||||
/// section in replacement text copied verbatim without reference recognition.
|
||||
@Suite("Entity decode bulk-copy equivalence")
|
||||
struct EntityDecodeBulkTests {
|
||||
private typealias Decoder = PureXML.Parsing.EntityDecoder
|
||||
private static let mark = PureXML.Parsing.Mark(line: 1, column: 1, offset: 0)
|
||||
|
||||
private func decode(_ raw: String, entities: [String: String] = [:], budget: Int = 1_000_000) throws -> String {
|
||||
var remaining = budget
|
||||
return try Decoder.decode(raw, entities: entities, budget: &remaining, at: Self.mark)
|
||||
}
|
||||
|
||||
@Test("predefined references inside literal runs decode in place")
|
||||
func test_predefinedRuns() throws {
|
||||
#expect(try decode("plain text with no references") == "plain text with no references")
|
||||
#expect(try decode("a&b") == "a&b")
|
||||
#expect(try decode("x & y < z > w "q" 'a'") == "x & y < z > w \"q\" 'a'")
|
||||
#expect(try decode("&<>") == "&<>")
|
||||
#expect(try decode("&tail") == "&tail")
|
||||
#expect(try decode("head&") == "head&")
|
||||
// A long literal run on each side of a single reference (the bulk case).
|
||||
let long = String(repeating: "abcde ", count: 200)
|
||||
#expect(try decode("\(long)&\(long)") == "\(long)&\(long)")
|
||||
}
|
||||
|
||||
@Test("character references decode decimal and hex")
|
||||
func test_characterReferences() throws {
|
||||
#expect(try decode("ABC") == "ABC")
|
||||
#expect(try decode("pre ☺ post") == "pre \u{263A} post")
|
||||
}
|
||||
|
||||
@Test("declared entity replacement expands and counts against the budget")
|
||||
func test_declaredReplacement() throws {
|
||||
#expect(try decode("<&e;>", entities: ["e": "REPLACEMENT"]) == "<REPLACEMENT>")
|
||||
// A replacement run longer than the remaining budget overflows: the
|
||||
// per-character path threw at the overrunning character, the bulk path
|
||||
// throws on the whole run; both reject the decode.
|
||||
#expect(throws: (any Error).self) {
|
||||
try decode("&e;", entities: ["e": "ABCDE"], budget: 3)
|
||||
}
|
||||
// Exactly the budget is allowed (the boundary the per-character guard
|
||||
// permitted).
|
||||
#expect(try decode("&e;", entities: ["e": "ABCDE"], budget: 5) == "ABCDE")
|
||||
}
|
||||
|
||||
@Test("a CDATA section in replacement text is copied verbatim")
|
||||
func test_cdataInReplacement() throws {
|
||||
// The '¬' inside CDATA must not be treated as a reference.
|
||||
#expect(try decode("&e;", entities: ["e": "<![CDATA[a¬b]]>"]) == "<![CDATA[a¬b]]>")
|
||||
}
|
||||
|
||||
@Test("an unterminated reference is rejected")
|
||||
func test_unterminatedReference() throws {
|
||||
#expect(throws: (any Error).self) {
|
||||
try decode("text & without a semicolon")
|
||||
}
|
||||
}
|
||||
|
||||
@Test("text with references decodes through the parser to one text node")
|
||||
func test_throughParser() throws {
|
||||
let node = try PureXML.parse("<r>x & y < z > w</r>")
|
||||
guard case let .document(children) = node, let root = children.compactMap(\.element).first else {
|
||||
Issue.record("no root")
|
||||
return
|
||||
}
|
||||
let text = root.children.compactMap { if case let .text(value) = $0 { value } else { nil } }.joined()
|
||||
#expect(text == "x & y < z > w")
|
||||
}
|
||||
}
|
||||
@ -69,6 +69,7 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count).
|
||||
| 15 | the same test fusion for the attribute axis (`matchesAttribute`): a `[@name='...']` predicate built every attribute node per element and filtered to one; it now wraps only the matching attribute, saving the owner retain and the qualified-name/value copies for the rest. Identical node-set (adversarially equivalence-reviewed; same-state A/B, ~20%) | unchanged | 0.022 s (~2.0x) | unchanged |
|
||||
| 16 | share the focus-independent evaluation state via an `Environment` reference: `focused(on:position:size:)` (called once per node of a predicate's input) copied the variables, functions, and namespace dictionaries every time; it now copies only node/position/size and shares one reference. Same semantics (same-state A/B, ~16%) | unchanged | 0.020 s (~2.0x) | unchanged |
|
||||
| 17 | compile the step's node test once per query instead of re-evaluating it per node: the descendant/attribute walks called `{ matchesTree($0, step.test, on: step.axis, context.namespaces) }`, passing the `namespaces` dictionary (and the `NodeTest`'s associated string) by value on every one of tens of thousands of nodes, so the dominant traversal carried a dictionary retain/release per node. `compiledTreeTest`/`compiledAttributeTest`/`compiledNodeTest` resolve the axis principal kind, the binding state, and the name shape once and return a tight closure capturing only the local name. Equivalent to `matchesTree`/`matchesAttribute`/`matches` (exhaustive differential test + adversarial review; same-state A/B, ~35%) | unchanged | 0.013 s (~1.3x) | unchanged |
|
||||
| 18 | bulk-copy literal runs in the entity decoder: `EntityExpander.expand` rebuilt every reference-bearing text run one character at a time (`result.append(character)` with a per-character budget charge), so a text node holding a single `&` was copied grapheme by grapheme; it now scans to the next `&`/`<` and appends the literal run in one go (charging the budget by the run length). Also skips the element-content reference-findings pass (an `&` scan plus a qualified-name render per text node) when there is no DTD. Identical decoded text (W3C XML conformance corpora + 6-case differential + adversarial review; same-state A/B, ~5%) | 0.138 s (~8.5x) | unchanged | unchanged |
|
||||
|
||||
## Profile findings (first sample, parse path)
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user