From 2cca4912ff7b694670fa572c5ace18eb83713e9b Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Thu, 25 Jun 2026 11:45:02 +0200 Subject: [PATCH] perf(parse): bulk-copy literal runs in the entity decoder (#291) EntityExpander.expand rebuilt every reference-bearing text run one character at a time: result.append(character) per character, with a per-character charge against the amplification budget. A text node holding a single & was copied grapheme by grapheme, which is the bulk of the per-text-node cost in a document whose content uses entities. Scan to the next '&' or '<' and append the literal run in one operation, charging the budget by the run length. The decoded text, the CDATA-verbatim shielding, and the billion-laughs budget enforcement are unchanged: the same overrun throws at the same input, since charging a run of length L against budget B fails exactly when L > B, the same condition the per-character guard enforced. Also return early from recordReferenceContentFindings when there are no DTD content models (the common no-DTD case): it otherwise scanned each text run for '&' and rendered the enclosing element's qualified name before the model lookup that would fail anyway. Measured parse ~5% faster on the 20k-item generated corpus via a same-state A/B (9.06x to ~8.5x slower than libxml2). Verified by the W3C XML conformance corpora, the new EntityDecodeBulkTests six-case decode differential, the full suite, the Xalan XSLT gold-output corpus, and the WASM build, with an adversarial equivalence review of the rewrite. --- CHANGELOG.md | 4 ++ Sources/Parsing/EntityDecoder.swift | 67 ++++++++++++++------ Sources/Parsing/EventReaderValidity.swift | 5 ++ Tests/EntityDecodeBulkTests.swift | 77 +++++++++++++++++++++++ docs/benchmarks.md | 1 + 5 files changed, 134 insertions(+), 20 deletions(-) create mode 100644 Tests/EntityDecodeBulkTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index debee06..257861b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed + +- The entity decoder copies the literal text between references in bulk instead of one character at a time. `EntityExpander.expand` rebuilt every reference-bearing run grapheme by grapheme (`result.append(character)` with a per-character amplification-budget charge), so a text node holding a single `&` was reconstructed character by character; it now scans to the next `&` or `<` and appends the run in one operation, charging the budget by the run length. The decoded text, the CDATA-verbatim handling, and the billion-laughs budget enforcement are unchanged (the same overrun throws at the same input). The element-content reference-findings pass, which scanned each text run for `&` and rendered the enclosing element's qualified name, now returns immediately when the document has no DTD content models, which is the common case. Measured `parse` ~5% faster on the 20k-item generated corpus (whose text uses `&`) via a same-machine-state A/B. Verified by the W3C XML conformance corpora, a six-case decode differential (`EntityDecodeBulkTests`: references in runs, decimal and hex character references, declared-entity replacement with budget overrun, CDATA in replacement text, an unterminated reference), and an adversarial equivalence review. + ## [0.4.1] - 2026-06-25 ### Changed diff --git a/Sources/Parsing/EntityDecoder.swift b/Sources/Parsing/EntityDecoder.swift index 756e722..021dc56 100644 --- a/Sources/Parsing/EntityDecoder.swift +++ b/Sources/Parsing/EntityDecoder.swift @@ -182,27 +182,38 @@ private struct EntityExpander { mutating func expand(_ raw: String, visiting: Set, counts: Bool) throws { var iterator = raw.startIndex while iterator < raw.endIndex { - let character = raw[iterator] - // A CDATA section inside replacement text shields its content from - // reference recognition: copy it verbatim through the terminator. - if character == "<", raw[iterator...].hasPrefix("") { - end = raw.index(cursor, offsetBy: 3) - break - } - cursor = raw.index(after: cursor) - } - for verbatim in raw[iterator ..< end] { - try append(verbatim, counts: counts) - } - iterator = end - continue + // Copy the literal run up to the next reference or markup character + // in bulk rather than one character at a time: only '&' (a + // reference) and '<' (a possible CDATA section) need per-character + // handling, and the text between them is the bulk of any run. + guard let special = raw[iterator...].firstIndex(where: { $0 == "&" || $0 == "<" }) else { + try appendRun(raw[iterator...], counts: counts) + return } - guard character == "&" else { + if special > iterator { + try appendRun(raw[iterator ..< special], counts: counts) + iterator = special + } + let character = raw[iterator] + if character == "<" { + // A CDATA section inside replacement text shields its content + // from reference recognition: copy it verbatim through the + // terminator. A '<' that does not open CDATA is literal content. + if raw[iterator...].hasPrefix("") { + end = raw.index(cursor, offsetBy: 3) + break + } + cursor = raw.index(after: cursor) + } + try appendRun(raw[iterator ..< end], counts: counts) + iterator = end + continue + } try append(character, counts: counts) iterator = raw.index(after: iterator) continue @@ -216,6 +227,22 @@ private struct EntityExpander { } } + /// Appends a verbatim run, charging its whole length against the + /// amplification budget at once when counting (the per-character + /// ``append(_:counts:)`` charges one at a time; for a run the effect is + /// identical, the budget reaching the same value and the same overrun + /// throwing, only without the per-character work). + private mutating func appendRun(_ run: Substring, counts: Bool) throws { + if counts { + let length = run.count + guard budget >= length else { + throw PureXML.Parsing.ParseError.amplificationLimitExceeded(mark) + } + budget -= length + } + result += run + } + private mutating func resolve(_ body: String, visiting: Set, counts: Bool) throws { if let predefined = Self.predefinedValue(body) { try append(predefined, counts: counts) diff --git a/Sources/Parsing/EventReaderValidity.swift b/Sources/Parsing/EventReaderValidity.swift index 5f3e2b1..0bc6bf5 100644 --- a/Sources/Parsing/EventReaderValidity.swift +++ b/Sources/Parsing/EventReaderValidity.swift @@ -90,6 +90,11 @@ extension PureXML.Parsing.EventReader { /// (which is character data, not ignorable whitespace) inside element /// content. mutating func recordReferenceContentFindings(raw: String, decoded: String) { + // Element-content reference findings exist only against a declared + // content model. With no DTD (the common case) there is nothing to + // check, so return before scanning `raw` for '&' or rendering the + // element's qualified name. + guard !documentType.elementModels.isEmpty else { return } guard raw.contains("&"), let current = open.last, let model = documentType.elementModels[current.localName] ?? documentType.elementModels[current.description] else { return } diff --git a/Tests/EntityDecodeBulkTests.swift b/Tests/EntityDecodeBulkTests.swift new file mode 100644 index 0000000..4afb7f3 --- /dev/null +++ b/Tests/EntityDecodeBulkTests.swift @@ -0,0 +1,77 @@ +import Testing +@testable import PureXML + +/// `EntityExpander.expand` copies the literal text between references in bulk +/// rather than one character at a time. These pin that the bulk path produces +/// exactly the per-character result it replaced: references at the start, end, +/// and adjacent; literal runs of every length around them; decimal and hex +/// character references; declared-entity replacement (the budget-counted path, +/// including a bulk run that overflows the amplification budget); and a CDATA +/// section in replacement text copied verbatim without reference recognition. +@Suite("Entity decode bulk-copy equivalence") +struct EntityDecodeBulkTests { + private typealias Decoder = PureXML.Parsing.EntityDecoder + private static let mark = PureXML.Parsing.Mark(line: 1, column: 1, offset: 0) + + private func decode(_ raw: String, entities: [String: String] = [:], budget: Int = 1_000_000) throws -> String { + var remaining = budget + return try Decoder.decode(raw, entities: entities, budget: &remaining, at: Self.mark) + } + + @Test("predefined references inside literal runs decode in place") + func test_predefinedRuns() throws { + #expect(try decode("plain text with no references") == "plain text with no references") + #expect(try decode("a&b") == "a&b") + #expect(try decode("x & y < z > w "q" 'a'") == "x & y < z > w \"q\" 'a'") + #expect(try decode("&<>") == "&<>") + #expect(try decode("&tail") == "&tail") + #expect(try decode("head&") == "head&") + // A long literal run on each side of a single reference (the bulk case). + let long = String(repeating: "abcde ", count: 200) + #expect(try decode("\(long)&\(long)") == "\(long)&\(long)") + } + + @Test("character references decode decimal and hex") + func test_characterReferences() throws { + #expect(try decode("ABC") == "ABC") + #expect(try decode("pre ☺ post") == "pre \u{263A} post") + } + + @Test("declared entity replacement expands and counts against the budget") + func test_declaredReplacement() throws { + #expect(try decode("<&e;>", entities: ["e": "REPLACEMENT"]) == "") + // A replacement run longer than the remaining budget overflows: the + // per-character path threw at the overrunning character, the bulk path + // throws on the whole run; both reject the decode. + #expect(throws: (any Error).self) { + try decode("&e;", entities: ["e": "ABCDE"], budget: 3) + } + // Exactly the budget is allowed (the boundary the per-character guard + // permitted). + #expect(try decode("&e;", entities: ["e": "ABCDE"], budget: 5) == "ABCDE") + } + + @Test("a CDATA section in replacement text is copied verbatim") + func test_cdataInReplacement() throws { + // The '¬' inside CDATA must not be treated as a reference. + #expect(try decode("&e;", entities: ["e": ""]) == "") + } + + @Test("an unterminated reference is rejected") + func test_unterminatedReference() throws { + #expect(throws: (any Error).self) { + try decode("text & without a semicolon") + } + } + + @Test("text with references decodes through the parser to one text node") + func test_throughParser() throws { + let node = try PureXML.parse("x & y < z > w") + guard case let .document(children) = node, let root = children.compactMap(\.element).first else { + Issue.record("no root") + return + } + let text = root.children.compactMap { if case let .text(value) = $0 { value } else { nil } }.joined() + #expect(text == "x & y < z > w") + } +} diff --git a/docs/benchmarks.md b/docs/benchmarks.md index fe0417f..6af35a8 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -69,6 +69,7 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count). | 15 | the same test fusion for the attribute axis (`matchesAttribute`): a `[@name='...']` predicate built every attribute node per element and filtered to one; it now wraps only the matching attribute, saving the owner retain and the qualified-name/value copies for the rest. Identical node-set (adversarially equivalence-reviewed; same-state A/B, ~20%) | unchanged | 0.022 s (~2.0x) | unchanged | | 16 | share the focus-independent evaluation state via an `Environment` reference: `focused(on:position:size:)` (called once per node of a predicate's input) copied the variables, functions, and namespace dictionaries every time; it now copies only node/position/size and shares one reference. Same semantics (same-state A/B, ~16%) | unchanged | 0.020 s (~2.0x) | unchanged | | 17 | compile the step's node test once per query instead of re-evaluating it per node: the descendant/attribute walks called `{ matchesTree($0, step.test, on: step.axis, context.namespaces) }`, passing the `namespaces` dictionary (and the `NodeTest`'s associated string) by value on every one of tens of thousands of nodes, so the dominant traversal carried a dictionary retain/release per node. `compiledTreeTest`/`compiledAttributeTest`/`compiledNodeTest` resolve the axis principal kind, the binding state, and the name shape once and return a tight closure capturing only the local name. Equivalent to `matchesTree`/`matchesAttribute`/`matches` (exhaustive differential test + adversarial review; same-state A/B, ~35%) | unchanged | 0.013 s (~1.3x) | unchanged | +| 18 | bulk-copy literal runs in the entity decoder: `EntityExpander.expand` rebuilt every reference-bearing text run one character at a time (`result.append(character)` with a per-character budget charge), so a text node holding a single `&` was copied grapheme by grapheme; it now scans to the next `&`/`<` and appends the literal run in one go (charging the budget by the run length). Also skips the element-content reference-findings pass (an `&` scan plus a qualified-name render per text node) when there is no DTD. Identical decoded text (W3C XML conformance corpora + 6-case differential + adversarial review; same-state A/B, ~5%) | 0.138 s (~8.5x) | unchanged | unchanged | ## Profile findings (first sample, parse path)