perf(serialize): scan bytes, not graphemes, to find characters to escape (#297)

The bulk-copy escape loop still iterated Characters and compared each to '&', '<', '>' (a grapheme comparison per character), the dominant remaining serialize cost for content that needs escaping. Every escaping trigger is an ASCII marker, or in ASCII-only mode a non-ASCII lead byte, so find the next one with value.utf8.firstIndex(where:) and decode no grapheme for the run between markers. The marker byte always lands on a grapheme boundary: an ASCII byte is its own scalar and forces a cluster break (so '&', '<', '>', the quote, and whitespace are never mid-cluster), and after a grapheme-aligned runStart the first byte at or above 0x80 is a lead byte. So value[marker] and value[runStart ..< marker] stay valid, and a multi-scalar cluster escapes as a unit (Swift rounds a mid-grapheme index to its boundary, so the base letter is neither dropped nor duplicated). The output is byte-identical. Measured serialize ~23% faster on the 20k-item generated corpus via a same-state A/B (~2.8x to ~2.2x slower than libxml2). Verified by EscapingBulkCopyTests (the oracle differential extended with combining-mark, ZWJ, and flag clusters), the Apache Xalan gold-output and C14N suites, the full suite, the WASM build, and an adversarial grapheme-alignment review.
2026-06-25 15:39:20 +02:00 · 2026-06-25 15:39:20 +02:00 · 0bda990f80
commit 0bda990f80
parent 1d4ec91922
4 changed files with 60 additions and 29 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

 ### Changed

+- Output escaping locates the characters that need escaping by scanning bytes rather than graphemes. The bulk-copy escape loop still iterated `Character`s and compared each to `&`, `<`, `>` (a grapheme comparison per character). Every escaping trigger is an ASCII marker, or in ASCII-only mode a non-ASCII lead byte, so `Escaping.text` and `Escaping.attribute` now find the next one with `value.utf8.firstIndex(where:)`, decoding no grapheme for the run between markers. The marker byte always lands on a grapheme boundary (an ASCII byte is its own scalar and forces a cluster break; the first byte at or above `0x80` after a boundary is a lead byte), so the string slice and subscript stay valid and a multi-scalar cluster escapes as a unit. The output is byte-identical. Measured `serialize` ~23% faster on the 20k-item generated corpus via a same-machine-state A/B. Verified by the oracle differential extended with combining-mark, ZWJ-sequence, and regional-indicator clusters (`EscapingBulkCopyTests`), the Apache Xalan gold-output and C14N suites, and an adversarial grapheme-alignment review.
+
 - Output escaping copies the verbatim runs between escapable characters in bulk instead of rebuilding the value one character at a time. `Escaping.text` and `Escaping.attribute` appended each character on its own (`result += String(character)` for every unescaped character) when a value needed escaping, so re-escaping a text node that holds a single `&` rebuilt it grapheme by grapheme. They now copy each run between escapable characters in a single append and rewrite only the markers (`&`, `<`, `>`, the attribute quote and whitespace, and, in ASCII-only mode, non-ASCII characters). The no-escaping-needed fast path is unchanged. The output is byte-identical. Measured `serialize` ~7% faster on the 20k-item generated corpus (whose decoded text re-escapes an `&`) via a same-machine-state A/B. Verified by a per-character-oracle differential with multibyte cases (`EscapingBulkCopyTests`), the Apache Xalan gold-output and C14N canonicalization suites, and an adversarial review including a 400k-input fuzz.

 - `parseTree` assembles the mutable `TreeNode` tree directly from the event stream instead of building the immutable value `Node` tree and converting it. The old path allocated two full trees: the parser built a value `Node` tree, then `TreeNode.init(_:)` walked it to build the reference tree. `Parser.buildTree` is the direct counterpart to the value-tree assembler, the same iterative event loop with the same element stack and the same well-formedness guards (it throws on an unmatched end tag and on a document with no element), but it accumulates `TreeNode` children per frame and adopts them when the element closes, so the tree is allocated once. The produced tree is identical, the same node kinds, names, attributes, values, child order, and parent links. Measured `parse` ~8% faster on the 20k-item generated corpus via a same-machine-state A/B. Verified by a structural differential against the convert-from-`Node` path (`ParseTreeDirectBuildTests`), the full suite and the W3C XML conformance, RELAX NG, and Apache Xalan corpora (which all materialize trees through `parseTree`), and an adversarial equivalence review.
--- a/Sources/Emitting/Escaping.swift
+++ b/Sources/Emitting/Escaping.swift
@ -13,30 +13,45 @@ extension PureXML.Emitting {
            }
            var result = ""
            var runStart = value.startIndex
-            var index = value.startIndex
-            while index < value.endIndex {
-                let character = value[index]
-                let escaped: String? = switch character {
-                case "&": "&amp;"
-                case "<": "&lt;"
-                case ">": "&gt;"
-                case "\r" where escapeCarriageReturn: "&#xD;"
-                default: asciiOnly ? plainIfNonASCII(character) : nil
-                }
-                let next = value.index(after: index)
-                if let escaped {
-                    // Copy the verbatim run before this character in one append
-                    // rather than appending each unescaped character on its own.
-                    if runStart < index { result += value[runStart ..< index] }
-                    result += escaped
-                    runStart = next
-                }
-                index = next
+            // Find each character that needs escaping by scanning bytes (every
+            // trigger is an ASCII marker, or in ASCII-only mode a non-ASCII lead
+            // byte, both of which fall on a grapheme boundary), copy the verbatim
+            // run before it in one append, and rewrite only that character.
+            while let marker = value.utf8[runStart...].firstIndex(where: {
+                textEscapableByte($0, asciiOnly: asciiOnly, escapeCarriageReturn: escapeCarriageReturn)
+            }) {
+                if runStart < marker { result += value[runStart ..< marker] }
+                let character = value[marker]
+                result += textEscape(character, asciiOnly: asciiOnly, escapeCarriageReturn: escapeCarriageReturn) ?? String(character)
+                runStart = value.index(after: marker)
            }
            if runStart < value.endIndex { result += value[runStart...] }
            return result
        }

+        /// Whether a byte can begin a character ``text`` must escape: an ASCII
+        /// markup byte, a carriage return when asked, or any non-ASCII lead byte
+        /// in ASCII-only mode (its character is reference-escaped whole).
+        private static func textEscapableByte(_ byte: UInt8, asciiOnly: Bool, escapeCarriageReturn: Bool) -> Bool {
+            switch byte {
+            case 0x26, 0x3C, 0x3E: true
+            case 0x0D where escapeCarriageReturn: true
+            case 0x80... where asciiOnly: true
+            default: false
+            }
+        }
+
+        /// The escaped form of one content character, or nil when it is verbatim.
+        private static func textEscape(_ character: Character, asciiOnly: Bool, escapeCarriageReturn: Bool) -> String? {
+            switch character {
+            case "&": "&amp;"
+            case "<": "&lt;"
+            case ">": "&gt;"
+            case "\r" where escapeCarriageReturn: "&#xD;"
+            default: asciiOnly ? plainIfNonASCII(character) : nil
+            }
+        }
+
        /// Whether ``text`` would change `value`: it escapes `&`, `<`, `>` always,
        /// a carriage return when asked, and (in ASCII-only mode) any non-ASCII
        /// byte. Scanned at the byte level, since every trigger is ASCII or, for
@ -80,21 +95,29 @@ extension PureXML.Emitting {
            }
            var result = ""
            var runStart = value.startIndex
-            var index = value.startIndex
-            while index < value.endIndex {
-                let character = value[index]
-                let next = value.index(after: index)
-                if let escaped = attributeEscape(character, quote: quote, asciiOnly: asciiOnly) {
-                    if runStart < index { result += value[runStart ..< index] }
-                    result += escaped
-                    runStart = next
-                }
-                index = next
+            while let marker = value.utf8[runStart...].firstIndex(where: {
+                attributeEscapableByte($0, quoteByte: quoteByte, asciiOnly: asciiOnly)
+            }) {
+                if runStart < marker { result += value[runStart ..< marker] }
+                let character = value[marker]
+                result += attributeEscape(character, quote: quote, asciiOnly: asciiOnly) ?? String(character)
+                runStart = value.index(after: marker)
            }
            if runStart < value.endIndex { result += value[runStart...] }
            return result
        }

+        /// Whether a byte can begin a character ``attribute`` must escape: the
+        /// quote, an ASCII markup or whitespace byte, or a non-ASCII lead byte in
+        /// ASCII-only mode.
+        private static func attributeEscapableByte(_ byte: UInt8, quoteByte: UInt8, asciiOnly: Bool) -> Bool {
+            switch byte {
+            case quoteByte, 0x26, 0x3C, 0x3E, 0x09, 0x0A, 0x0D: true
+            case 0x80... where asciiOnly: true
+            default: false
+            }
+        }
+
        /// The escaped form of one attribute-value character, or nil when it is
        /// copied verbatim. Split out so ``attribute`` stays a simple run loop.
        private static func attributeEscape(_ character: Character, quote: Character, asciiOnly: Bool) -> String? {
--- a/Tests/EscapingBulkCopyTests.swift
+++ b/Tests/EscapingBulkCopyTests.swift
@ -40,6 +40,11 @@ struct EscapingBulkCopyTests {
        "&<>", "&start", "end&", "&&&",
        "café & thé", "e\u{0301} < x", "\u{1F600}&\u{1F601}",
        "tab\tnewline\ncr\r", "naïve résumé",
+        // Combining marks: in ASCII-only mode the scan's first byte >= 0x80 is the
+        // mark's lead byte, mid-grapheme; the whole cluster must still escape as a
+        // unit, not duplicate or drop the base letter.
+        "e\u{0301}", "a\u{0301}b", "e\u{0301}\u{0301}", "x\u{0301}y\u{0301}z", "a\u{0301}<b",
+        "\u{1F468}\u{200D}\u{1F469}\u{200D}\u{1F467}", "\u{1F1FA}\u{1F1F8}",
        String(repeating: "x", count: 300) + "&" + String(repeating: "y", count: 300),
    ]

--- a/docs/benchmarks.md
+++ b/docs/benchmarks.md
@ -75,6 +75,7 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count).
 | 21 | scan for the entity decoder's markers at the byte level: `EntityExpander.expand` found the next `&`, `<`, and `;` with `String.firstIndex` over graphemes; since all three are ASCII, a UTF-8 index is a valid string index, so the scan runs over `raw.utf8` and skips decoding every literal byte into a grapheme just to look for a marker. Same decoded text (the bulk + 4 multibyte-boundary cases in the decode differential + W3C conformance; same-state A/B, ~5.5%) | 0.099 s (~6.1x) | unchanged | unchanged |
 | 22 | build the mutable `TreeNode` tree directly from the event stream instead of building the value `Node` tree and converting it: `parseTree` ran `TreeNode(parse(...))`, allocating two full trees (the value tree, then a converted reference tree). `Parser.buildTree` is the direct counterpart to `build` (same event loop, stack discipline, and well-formedness guards) that accumulates `TreeNode` children per frame and adopts them when an element closes, so the tree is allocated once. Identical tree (structural differential against the convert path + the whole suite and conformance corpora, which all run through `parseTree`, + adversarial review; same-state A/B, ~8%) | 0.090 s (~5.5x) | unchanged | unchanged |
 | 23 | bulk-copy the verbatim runs when escaping output: `Escaping.text`/`attribute` rebuilt a value that needed escaping character by character (`result += String(character)` per unescaped character), so re-escaping a text node that holds a single `&` rebuilt it grapheme by grapheme; they now copy each run between escapable characters in one append and only the markers are rewritten. The `*NeedsEscaping` fast path (no rebuild when nothing needs escaping) is unchanged. Byte-identical output (per-character-oracle differential with multibyte cases + Apache Xalan gold output + C14N suite + adversarial 400k fuzz; same-state A/B, ~7%) | unchanged | unchanged | 0.029 s (~2.8x) |
+| 24 | find the characters to escape by scanning bytes, not graphemes: the bulk-escape loop still iterated `Character`s and compared each to `&`/`<`/`>` (a grapheme compare per character); since every trigger is an ASCII marker, or in ASCII-only mode a non-ASCII lead byte, `value.utf8.firstIndex(where:)` locates the next one without decoding the run into graphemes (the marker byte lands on a grapheme boundary, so the string slice and subscript stay valid). Byte-identical output (the oracle differential extended with combining-mark, ZWJ, and flag clusters + the gold/C14N suites + adversarial grapheme-alignment review; same-state A/B, ~23%) | unchanged | unchanged | 0.022 s (~2.2x) |

 ## Profile findings (first sample, parse path)