From 2f8ca9b289e07f1825412a745f3ce3ad4b7ceafa Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Wed, 24 Jun 2026 18:46:58 +0200 Subject: [PATCH] perf(parsing): byte-level scanning fast paths halve parse time vs libxml2 (#275) Keep the reader on its byte fast path through dispatch and name/attribute scanning so plain-ASCII documents avoid the per-Character String machinery (String.Iterator.next, string comparison, small-string materialization, bridge-object ARC) that a profile indicted as the parse bottleneck. The root cause was that a single buffered lookahead Character defeated every existing byte path, each of which requires an empty buffer, and the content, markup, and attribute dispatch each peeked a Character before scanning. Add a non-buffering peekByte for dispatch, a byte-mode skipSpace, an all-ASCII takeASCIIName, and an attributeRunBytes bulk scanner (with the 3.3.3 whitespace-to-space transform, deferring any carriage return to the Character path so CRLF is not double-counted). Detect a text run's closing '<' via peekByte instead of buffering it, which had cascaded the next element's whole dispatch and name scan onto the slow path. Skip reference decoding for ampersand-free text. The Character path remains the exact fallback for a streaming source, spliced entity text, and any non-ASCII byte; the two paths share scanContentMarkup so they cannot diverge. Measured parse 20.1x -> 9.5x slower than libxml2 on the 20k-item generated corpus, roughly a 53 percent wall-clock reduction. A differential oracle pins the byte and Character paths to identical event streams and error marks over an adversarial corpus, alongside boundary tests. Verified against the full local suite, the Apache Xalan XSLT gold-output corpus, the W3C XML conformance corpora, the RELAX NG spec suite, and the WASM build. No spec behavior changed. --- CHANGELOG.md | 4 + Sources/Parsing/CharacterClassification.swift | 35 +++ Sources/Parsing/EventReader.swift | 57 ++-- Sources/Parsing/EventReaderAttributes.swift | 36 ++- Sources/Parsing/EventReaderContent.swift | 28 ++ Sources/Parsing/Reader.swift | 246 ++++++++++++------ Sources/Parsing/XMLCharacter.swift | 15 ++ Tests/ByteFastPathTests.swift | 117 +++++++++ Tests/ParserPathDifferentialTests.swift | 121 +++++++++ docs/benchmarks.md | 2 + 10 files changed, 558 insertions(+), 103 deletions(-) create mode 100644 Sources/Parsing/CharacterClassification.swift create mode 100644 Sources/Parsing/EventReaderContent.swift create mode 100644 Tests/ByteFastPathTests.swift create mode 100644 Tests/ParserPathDifferentialTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 80dc474..bba8dc8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed + +- Parser throughput on plain-ASCII documents roughly halved against the libxml2 reference (measured `parse` 20.1x -> 9.5x slower than libxml2 on the 20k-item generated corpus, `bash scripts/benchmark.sh 20000 5`; ~53% wall-clock reduction). The work was profile-driven (macOS `sample`, not guesswork): the scanner, not tree construction, dominated parse time (SAX is ~84% of parse), and within it the cost was per-`Character` `String` machinery (`String.Iterator.next`, `_stringCompareWithSmolCheck`, `_uncheckedFromUTF8`, bridge-object ARC). The root cause was that a single buffered lookahead `Character` defeated every existing byte fast path (each requires an empty buffer), and the content/markup/attribute dispatch all peeked a `Character` before scanning. The fix keeps the reader on the byte path: a non-buffering `peekByte` for dispatch, a byte-mode `skipSpace`, an all-ASCII `takeASCIIName`, an `attributeRunBytes` bulk scanner (with the 3.3.3 whitespace-to-space transform, deferring any carriage return to the Character path so CRLF is not double-counted), and a `sawAmpersand` flag so ampersand-free text skips reference decoding entirely. The single highest-leverage change was detecting the closing `<` of a text run via `peekByte` instead of buffering it, which had been cascading the next element's whole dispatch and name scan onto the slow path. The Character path remains the exact fallback for a streaming source, spliced entity text, and any non-ASCII byte; byte and Character paths are kept from diverging by `scanContentMarkup` (shared dispatch) and by `ByteFastPathTests`, which pins the fast-path/fallback boundaries (non-ASCII names, CRLF and character references in attribute values, ampersand-free vs entity-bearing text, over-long names, raw `<` in attributes). Verified against the full local suite (1668 tests) plus the Apache Xalan XSLT gold-output corpus, the W3C XML conformance corpora (xmltest, OASIS, Sun, Eduni, IBM, Japanese), and the RELAX NG spec suite, all green; no spec behavior changed. + ## [0.3.0] - 2026-06-24 ### Added diff --git a/Sources/Parsing/CharacterClassification.swift b/Sources/Parsing/CharacterClassification.swift new file mode 100644 index 0000000..d9aeeaa --- /dev/null +++ b/Sources/Parsing/CharacterClassification.swift @@ -0,0 +1,35 @@ +extension StringProtocol { + /// Trims leading and trailing XML whitespace without Foundation. + func trimmingXMLWhitespace() -> String { + var slice = self[...] + while let first = slice.first, first.isXMLWhitespace { + slice = slice.dropFirst() + } + while let last = slice.last, last.isXMLWhitespace { + slice = slice.dropLast() + } + return String(slice) + } +} + +extension Character { + private typealias XML = PureXML.Parsing.XMLCharacter + + /// XML S production: space, tab, carriage return, line feed. + var isXMLWhitespace: Bool { + unicodeScalars.count == 1 && unicodeScalars.first.map(XML.isWhitespace) == true + } + + /// Whether this character may start an XML name. A grapheme qualifies when its + /// first scalar is a NameStartChar and any trailing scalars (combining marks) + /// are NameChar. + var isXMLNameStart: Bool { + guard let first = unicodeScalars.first, XML.isNameStart(first) else { return false } + return unicodeScalars.dropFirst().allSatisfy(XML.isNameChar) + } + + /// Whether this character may continue an XML name (every scalar a NameChar). + var isXMLNameContinuation: Bool { + !unicodeScalars.isEmpty && unicodeScalars.allSatisfy(XML.isNameChar) + } +} diff --git a/Sources/Parsing/EventReader.swift b/Sources/Parsing/EventReader.swift index ceceb1a..564b088 100644 --- a/Sources/Parsing/EventReader.swift +++ b/Sources/Parsing/EventReader.swift @@ -145,34 +145,28 @@ public extension PureXML.Parsing { // replacement into the reader with no text before it: the next // construct is then read from the spliced replacement. while true { + // Fast byte dispatch: when the next input is a raw byte (an + // empty lookahead buffer over owned storage), route on it + // without buffering a Character, so every matches()/consume()/ + // scan call below stays on its byte fast path. The Character + // path is the identical fallback for a streaming source, + // spliced entity text, or a non-ASCII lead. + if let byte = reader.peekByte() { + eventStart = reader.mark + if byte == 0x3C { return try scanContentMarkup() } // '<' + if let text = try scanText() { return text } + continue + } guard let lead = reader.peek() else { throw ParseError.unexpectedEndOfInput(reader.mark) } eventStart = reader.mark - if reader.matches(" Event { + mutating func scanStartTag() throws -> Event { guard open.count < limits.maxDepth else { throw ParseError.nestingTooDeep(limit: limits.maxDepth, reader.mark) } @@ -207,17 +201,26 @@ public extension PureXML.Parsing { let mark = reader.mark var raw = "" var length = 0 + // Whether any '&' was seen: without one the text has no references + // and no markup-entity boundary, so the whole decode/split/findings + // pass is skipped. The scanners visit every byte already, so this + // costs nothing beyond a compare they were doing anyway. + var sawAmpersand = false while true { // Bulk byte runs first: plain ASCII content (the bulk of any // document) skips the per-character machinery entirely; the // run never spans '<', CR, ']', or invalid bytes, so the // character path below keeps exact error marks. - if let run = reader.contentRunBytes() { + if let run = reader.contentRunBytes(sawAmpersand: &sawAmpersand) { length += run.utf8.count try checkContent(length, mark) raw += run continue } + // Detect the closing '<' on the byte path so the buffer stays + // empty: a buffered '<' would force the next element's dispatch, + // matches(), and name scan onto the slow Character path. + if reader.peekByte() == 0x3C { break } guard let character = reader.peek(), character != "<" else { break } length += 1 try checkContent(length, mark) @@ -227,10 +230,12 @@ public extension PureXML.Parsing { guard character.unicodeScalars.allSatisfy(XMLCharacter.isChar) else { throw ParseError.invalidCharacter(reader.mark) } + if character == "&" { sawAmpersand = true } raw.append(character) reader.advance() } - let split = raw.contains("&") ? EntityDecoder.splitAtMarkupEntity(raw, entities: referencableEntities) : nil + guard sawAmpersand else { return .characters(raw) } + let split = EntityDecoder.splitAtMarkupEntity(raw, entities: referencableEntities) if let split { let replacement = try EntityDecoder.includeForContent( split.name, @@ -247,7 +252,7 @@ public extension PureXML.Parsing { return .characters(decoded) } - private mutating func scanProcessingInstruction() throws -> (target: String, data: String) { + mutating func scanProcessingInstruction() throws -> (target: String, data: String) { let mark = reader.mark reader.consume(" PureXML.Model.QualifiedName { let mark = reader.mark + // Fast path: an all-ASCII name scanned directly from the bytes, with + // no per-character buffering. The Character loop below handles a + // non-ASCII name, the non-fast-path, and the too-long error. + if let fast = reader.takeASCIIName(maxLength: limits.maxNameLength) { + return PureXML.Model.QualifiedName(fast) + } guard let first = reader.peek(), first.isXMLNameStart else { throw ParseError.expectedName(reader.mark) } diff --git a/Sources/Parsing/EventReaderAttributes.swift b/Sources/Parsing/EventReaderAttributes.swift index 202e4b9..d0e9e39 100644 --- a/Sources/Parsing/EventReaderAttributes.swift +++ b/Sources/Parsing/EventReaderAttributes.swift @@ -6,8 +6,14 @@ extension PureXML.Parsing.EventReader { var seen: Set = [] while true { reader.skipSpace() - guard let next = reader.peek(), next != ">", next != "/" else { - return attributes + // Byte dispatch keeps the buffer empty so scanName() reaches its + // byte fast path; the Character peek is the off-fast-path fallback. + if let byte = reader.peekByte() { + if byte == 0x3E || byte == 0x2F { return attributes } // '>' or '/' + } else { + guard let next = reader.peek(), next != ">", next != "/" else { + return attributes + } } let mark = reader.mark let name = try scanName() @@ -27,8 +33,17 @@ extension PureXML.Parsing.EventReader { throw PureXML.Parsing.ParseError.duplicateAttribute(name: name.description, mark) } attributes.append(PureXML.Model.Attribute(name: name, value: value)) - // A following attribute needs whitespace before it. - if let next = reader.peek(), next != ">", next != "/", next != "?", !next.isWhitespace { + // A following attribute needs whitespace before it. Check the raw + // byte when on the fast path to avoid buffering the next name char. + if let byte = reader.peekByte() { + // '>', '/', '?', or whitespace may legally follow a value; any + // other byte means a missing separator before the next attribute. + let isSeparator = byte == 0x3E || byte == 0x2F || byte == 0x3F + || byte == 0x20 || byte == 0x09 || byte == 0x0A || byte == 0x0D + if !isSeparator { + throw PureXML.Parsing.ParseError.missingSpaceBeforeAttribute(reader.mark) + } + } else if let next = reader.peek(), next != ">", next != "/", next != "?", !next.isWhitespace { throw PureXML.Parsing.ParseError.missingSpaceBeforeAttribute(reader.mark) } } @@ -40,9 +55,20 @@ extension PureXML.Parsing.EventReader { throw PureXML.Parsing.ParseError.unquotedAttributeValue(reader.mark) } reader.advance() + let quoteByte: UInt8 = quote == "\"" ? 0x22 : 0x27 var raw = "" var length = 0 - while let character = reader.peek(), character != quote { + while true { + // Bulk byte runs first: plain ASCII value text (the common case) + // skips the per-character machinery; the run stops before the + // quote, '<', CR, or any byte the Character path must inspect. + if let run = reader.attributeRunBytes(quote: quoteByte) { + length += run.utf8.count + try checkContent(length, mark) + raw += run + continue + } + guard let character = reader.peek(), character != quote else { break } length += 1 try checkContent(length, mark) if character == "<" { diff --git a/Sources/Parsing/EventReaderContent.swift b/Sources/Parsing/EventReaderContent.swift new file mode 100644 index 0000000..fd3b6ce --- /dev/null +++ b/Sources/Parsing/EventReaderContent.swift @@ -0,0 +1,28 @@ +/// Content markup dispatch, in its own file to keep the reader's primary type +/// body under the length cap. Behavior is part of EventReader proper. +extension PureXML.Parsing.EventReader { + /// At a '<' in element content, dispatches to the matching markup scanner. + /// The caller has confirmed the lead is '<' and set the event start. Shared + /// by the byte and Character dispatch paths so the two cannot diverge. + mutating func scanContentMarkup() throws -> Event { + if reader.matches("', quotes) while leaving + /// the buffer empty, so the byte scanners below stay engaged, and fall + /// back to the Character path for any byte it does not recognize. + func peekByte(_ ahead: Int = 0) -> UInt8? { + guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil } + let index = byteIndex + ahead + return index < storage.count ? storage.pointer[index] : nil + } + var mark: Mark { Mark(line: line, column: column, offset: offset) } @@ -182,6 +195,34 @@ extension PureXML.Parsing { return true } + /// On the byte fast path (an empty lookahead buffer over owned bytes), + /// consumes an all-ASCII XML name and returns it, advancing past it. + /// Returns nil when not on the fast path, when the first byte is not an + /// ASCII name-start, when the name reaches a non-ASCII byte, or when it + /// exceeds `maxLength`, so the caller falls back to the Character scanner + /// (which handles the full Unicode name grammar and the length error). + /// A name has no newlines, so position tracking is plain arithmetic. + mutating func takeASCIIName(maxLength: Int) -> String? { + guard buffer.isEmpty, pendingRaw == nil, let storage, byteIndex < storage.count else { return nil } + let pointer = storage.pointer + guard PureXML.Parsing.XMLCharacter.isASCIINameStart(pointer[byteIndex]) else { return nil } + var index = byteIndex + 1 + let limit = byteIndex + maxLength + while index < storage.count { + let byte = pointer[index] + if byte >= 0x80 { return nil } + if !PureXML.Parsing.XMLCharacter.isASCIINameChar(byte) { break } + if index >= limit { return nil } + index += 1 + } + let length = index - byteIndex + let run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: length), as: UTF8.self) + byteIndex = index + offset += length + column += length + return run + } + /// If the upcoming characters equal `literal`, consume them and return true. @discardableResult mutating func consume(_ literal: String) -> Bool { @@ -203,58 +244,35 @@ extension PureXML.Parsing { } mutating func skipSpace() { + if skipSpaceBytes() { return } while let character = peek(), character.isXMLWhitespace { advance() } } - /// Bulk-scans character data in byte mode: consumes and returns the - /// longest upcoming run of plain ASCII content bytes (no '<', no - /// carriage return, no "]]" pair, every byte a valid XML character), - /// or nil when the fast path does not apply. Anything subtle, an - /// entity boundary aside ('&' is plain content here), is left for - /// the character path so error marks stay exact. Returns nil rather - /// than an empty run so callers can alternate with the slow loop. - mutating func contentRunBytes() -> String? { - guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil } + /// Byte-mode whitespace skip. Returns true when it stopped at a clean + /// non-whitespace byte (so the buffer is empty and the caller needs no + /// Character loop); false when off the fast path or stopped at a + /// carriage return that the Character path must fold per 2.11. + private mutating func skipSpaceBytes() -> Bool { + guard buffer.isEmpty, pendingRaw == nil, let storage else { return false } let pointer = storage.pointer - let count = storage.count - // A run never contains ']', so "]]>" can only straddle the - // boundary where the slow path consumed "]]" and the run would - // begin with '>': leave a leading '>' to the slow path and its - // cdataCloseInContent check (W3C ibm14n01). - if byteIndex < count, pointer[byteIndex] == 0x3E { return nil } - var index = byteIndex - var newlines = 0 - var lastLineStart = -1 - while index < count { - let byte = pointer[index] - if byte == 0x3C { break } // '<' - // Valid plain ASCII content only; CR, ']' and non-ASCII go - // to the character path. - guard byte < 0x80, byte != 0x0D, byte != 0x5D, - byte >= 0x20 || byte == 0x09 || byte == 0x0A - else { - if index == byteIndex { return nil } - break + while byteIndex < storage.count { + switch pointer[byteIndex] { + case 0x20, 0x09: + column += 1 + case 0x0A: + line += 1 + column = 1 + case 0x0D: + return false + default: + return true } - if byte == 0x0A { - newlines += 1 - lastLineStart = index - } - index += 1 + byteIndex += 1 + offset += 1 } - guard index > byteIndex else { return nil } - let run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: index - byteIndex), as: UTF8.self) - offset += index - byteIndex - if newlines > 0 { - line += newlines - column = index - lastLineStart - } else { - column += index - byteIndex - } - byteIndex = index - return run + return true } /// Prepends `text` to the unread stream: used to splice a general @@ -267,38 +285,116 @@ extension PureXML.Parsing { } } -extension StringProtocol { - /// Trims leading and trailing XML whitespace without Foundation. - func trimmingXMLWhitespace() -> String { - var slice = self[...] - while let first = slice.first, first.isXMLWhitespace { - slice = slice.dropFirst() +/// The bulk byte-run scanners, in an extension so the reader's primary type +/// body stays under the length cap. They consume plain ASCII runs of content +/// or attribute-value text directly from the owned storage (the common case in +/// any real document) and hand subtle bytes back to the Character path. Both +/// stay on the byte fast path only when the lookahead buffer is empty. +extension PureXML.Parsing.Reader { + /// Bulk-scans character data in byte mode: consumes and returns the + /// longest upcoming run of plain ASCII content bytes (no '<', no + /// carriage return, no "]]" pair, every byte a valid XML character), + /// or nil when the fast path does not apply. Anything subtle, an + /// entity boundary aside ('&' is plain content here), is left for + /// the character path so error marks stay exact. Returns nil rather + /// than an empty run so callers can alternate with the slow loop. + mutating func contentRunBytes(sawAmpersand: inout Bool) -> String? { + guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil } + let pointer = storage.pointer + let count = storage.count + // A run never contains ']', so "]]>" can only straddle the + // boundary where the slow path consumed "]]" and the run would + // begin with '>': leave a leading '>' to the slow path and its + // cdataCloseInContent check (W3C ibm14n01). + if byteIndex < count, pointer[byteIndex] == 0x3E { return nil } + var index = byteIndex + var newlines = 0 + var lastLineStart = -1 + while index < count { + let byte = pointer[index] + if byte == 0x3C { break } // '<' + // Valid plain ASCII content only; CR, ']' and non-ASCII go + // to the character path. + guard byte < 0x80, byte != 0x0D, byte != 0x5D, + byte >= 0x20 || byte == 0x09 || byte == 0x0A + else { + if index == byteIndex { return nil } + break + } + if byte == 0x0A { + newlines += 1 + lastLineStart = index + } else if byte == 0x26 { + sawAmpersand = true + } + index += 1 } - while let last = slice.last, last.isXMLWhitespace { - slice = slice.dropLast() + guard index > byteIndex else { return nil } + let run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: index - byteIndex), as: UTF8.self) + offset += index - byteIndex + if newlines > 0 { + line += newlines + column = index - lastLineStart + } else { + column += index - byteIndex } - return String(slice) - } -} - -extension Character { - private typealias XML = PureXML.Parsing.XMLCharacter - - /// XML S production: space, tab, carriage return, line feed. - var isXMLWhitespace: Bool { - unicodeScalars.count == 1 && unicodeScalars.first.map(XML.isWhitespace) == true - } - - /// Whether this character may start an XML name. A grapheme qualifies when its - /// first scalar is a NameStartChar and any trailing scalars (combining marks) - /// are NameChar. - var isXMLNameStart: Bool { - guard let first = unicodeScalars.first, XML.isNameStart(first) else { return false } - return unicodeScalars.dropFirst().allSatisfy(XML.isNameChar) - } - - /// Whether this character may continue an XML name (every scalar a NameChar). - var isXMLNameContinuation: Bool { - !unicodeScalars.isEmpty && unicodeScalars.allSatisfy(XML.isNameChar) + byteIndex = index + return run + } + + /// Bulk-scans an attribute value in byte mode: consumes and returns the + /// longest upcoming run of plain ASCII value bytes, stopping at the + /// closing `quote`, a raw '<', a carriage return (2.11 folding then + /// 3.3.3 normalization must run on the Character path), a control + /// character, or a non-ASCII byte. Tab and line feed inside the run + /// normalize to a single space per 3.3.3; '&' stays raw for later + /// reference decoding. Returns nil when the fast path does not apply or + /// the run would be empty, so callers alternate with the slow loop. + mutating func attributeRunBytes(quote: UInt8) -> String? { + guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil } + let pointer = storage.pointer + let count = storage.count + var index = byteIndex + var newlines = 0 + var lastLineStart = -1 + var needsTransform = false + while index < count { + let byte = pointer[index] + if byte == quote || byte == 0x3C || byte == 0x0D || byte >= 0x80 { break } + if byte < 0x20 { + if byte == 0x09 || byte == 0x0A { + needsTransform = true + if byte == 0x0A { + newlines += 1 + lastLineStart = index + } + } else { + break + } + } + index += 1 + } + guard index > byteIndex else { return nil } + let length = index - byteIndex + let run: String + if needsTransform { + var bytes = [UInt8](repeating: 0, count: length) + for position in 0 ..< length { + let byte = pointer[byteIndex + position] + bytes[position] = (byte == 0x09 || byte == 0x0A) ? 0x20 : byte + } + run = String(decoding: bytes, as: UTF8.self) + } else { + run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: length), as: UTF8.self) + } + offset += length + if newlines > 0 { + line += newlines + column = index - lastLineStart + } else { + column += length + } + byteIndex = index + return run } } diff --git a/Sources/Parsing/XMLCharacter.swift b/Sources/Parsing/XMLCharacter.swift index ed69b0d..f3ace93 100644 --- a/Sources/Parsing/XMLCharacter.swift +++ b/Sources/Parsing/XMLCharacter.swift @@ -50,6 +50,21 @@ public extension PureXML.Parsing { } } + /// The ASCII subset of `NameStartChar`, tested on a raw byte for the + /// parser's byte-level fast path. A non-ASCII name-start (byte >= 0x80) + /// is left to the scalar predicate via the Character scanner. + static func isASCIINameStart(_ byte: UInt8) -> Bool { + switch byte { + case 0x3A, 0x41 ... 0x5A, 0x5F, 0x61 ... 0x7A: true + default: false + } + } + + /// The ASCII subset of `NameChar` (after the first), on a raw byte. + static func isASCIINameChar(_ byte: UInt8) -> Bool { + isASCIINameStart(byte) || (byte >= 0x30 && byte <= 0x39) || byte == 0x2D || byte == 0x2E + } + /// `NameChar`: a name character after the first. public static func isNameChar(_ scalar: Unicode.Scalar) -> Bool { if isNameStart(scalar) { return true } diff --git a/Tests/ByteFastPathTests.swift b/Tests/ByteFastPathTests.swift new file mode 100644 index 0000000..ee94882 --- /dev/null +++ b/Tests/ByteFastPathTests.swift @@ -0,0 +1,117 @@ +import Testing +@testable import PureXML + +/// Locks the parser's byte-level fast paths against their Character-path +/// fallbacks. Each case is chosen to land exactly on a boundary where the byte +/// scanner must hand off to the scalar scanner (a non-ASCII byte, a carriage +/// return, an entity reference) so that a future change to either path cannot +/// silently diverge. The expected values are the spec-correct results, +/// independent of which path produced them. +@Suite("Byte fast-path boundaries") +struct ByteFastPathTests { + @Test("An all-ASCII name uses the byte path; a non-ASCII name falls back") + func test_nameFallback() throws { + let node = try PureXML.parse("y") + let element = node.firstElement + #expect(element?.name.localName == "caf\u{e9}") + #expect(element?.attributes == [.init("na\u{ef}ve", "x")]) + #expect(element?.text == "y") + } + + @Test("A name whose ASCII prefix precedes a non-ASCII byte stays whole") + func test_mixedNameStaysWhole() throws { + // The byte scanner consumes "ab", meets the non-ASCII byte, returns nil, + // and the Character path re-scans the entire "ab\u{e9}" as one name. + let node = try PureXML.parse("") + #expect(node.firstElement?.name.localName == "ab\u{e9}") + } + + @Test("A prefixed name splits at the colon on the byte path") + func test_prefixedName() throws { + let node = try PureXML.parse("") + let element = node.firstElement + #expect(element?.name.prefix == "m") + #expect(element?.name.localName == "row") + #expect(element?.attributes.contains { $0.name.localName == "k" && $0.value == "v" } == true) + } + + @Test("Literal whitespace in an attribute value normalizes to single spaces") + func test_attributeWhitespaceNormalizes() throws { + // Tab and line feed each become one space per 3.3.3; these run through + // the byte scanner's whitespace transform. + let node = try PureXML.parse("") + #expect(node.firstElement?.attributes == [.init("v", "x y z")]) + } + + @Test("A CRLF in an attribute value collapses to a single space") + func test_attributeCRLF() throws { + // 2.11 folds CRLF to one LF before 3.3.3 turns it into one space. The + // byte scanner defers any carriage return to the Character path so the + // pair is not double-counted as two spaces. + let node = try PureXML.parse("") + #expect(node.firstElement?.attributes == [.init("v", "x y")]) + } + + @Test("A character reference in an attribute value survives normalization") + func test_attributeCharReferenceSurvives() throws { + // A referenced line feed is NOT subject to whitespace normalization: it + // stays a line feed while a literal line feed would have become a space. + let node = try PureXML.parse("") + #expect(node.firstElement?.attributes == [.init("v", "x\ny")]) + } + + @Test("Entities and bare ampersands in text decode through the slow path") + func test_textEntities() throws { + let node = try PureXML.parse("

plain text then 1 < 2 & ok

") + #expect(node.firstElement?.text == "plain text then 1 < 2 & ok") + } + + @Test("Ampersand-free text is returned verbatim") + func test_textWithoutEntities() throws { + let node = try PureXML.parse("

just plain ascii content, no references

") + #expect(node.firstElement?.text == "just plain ascii content, no references") + } + + @Test("A CRLF inside text content folds to a single line feed") + func test_textCRLF() throws { + let node = try PureXML.parse("

line one\r\nline two

") + #expect(node.firstElement?.text == "line one\nline two") + } + + @Test("Non-ASCII text content round-trips unchanged") + func test_nonASCIIText() throws { + let node = try PureXML.parse("

caf\u{e9} na\u{ef}ve r\u{e9}sum\u{e9}

") + #expect(node.firstElement?.text == "caf\u{e9} na\u{ef}ve r\u{e9}sum\u{e9}") + } + + @Test("An over-long ASCII name is rejected, not silently truncated") + func test_longNameRejected() { + let limits = PureXML.Parsing.Limits(maxNameLength: 4) + #expect(throws: PureXML.Parsing.ParseError.self) { + try PureXML.parse("", limits: limits) + } + } + + @Test("A raw less-than inside an attribute value is rejected") + func test_rawLessThanInAttribute() { + #expect(throws: PureXML.Parsing.ParseError.self) { + try PureXML.parse("") + } + } + + @Test("A leading greater-than in text content is preserved") + func test_textLeadingGreaterThan() throws { + let node = try PureXML.parse("

a > b

") + #expect(node.firstElement?.text == "a > b") + } +} + +private extension PureXML.Model.Node { + var firstElement: PureXML.Model.Element? { + guard case let .document(children) = self else { return element } + for child in children { + if case let .element(element) = child { return element } + } + return nil + } +} diff --git a/Tests/ParserPathDifferentialTests.swift b/Tests/ParserPathDifferentialTests.swift new file mode 100644 index 0000000..41a46d1 --- /dev/null +++ b/Tests/ParserPathDifferentialTests.swift @@ -0,0 +1,121 @@ +import Testing +@testable import PureXML + +/// A differential oracle for the parser's byte-level fast paths. The byte path +/// (a reader built from a `String`, which owns its UTF-8 and scans straight off +/// the bytes) and the streaming Character path (a reader built from a pulling +/// closure, whose `storage` is nil so every byte method returns nil) are two +/// implementations of the same grammar. They must therefore emit the identical +/// event stream AND, on malformed input, fail at the identical source mark, for +/// every input. Any divergence is a fast-path bug. The corpus is chosen to land +/// on the boundaries where the two paths differ in mechanism: carriage-return +/// folding, attribute-value normalization, multi-line position tracking, the +/// `]]>` guard, name limits, and the ASCII/non-ASCII handoff. +@Suite("Parser byte/Character path differential") +struct ParserPathDifferentialTests { + private func drain(_ reader: inout PureXML.Parsing.EventReader) -> String { + var out = "" + do { + while let event = try reader.next() { + let mark = reader.eventStart + out += "\(event)@\(mark.line):\(mark.column):\(mark.offset)|" + } + out += "EOF" + } catch { + out += "ERR:\(error)" + } + return out + } + + private func bytePath(_ input: String, _ limits: PureXML.Parsing.Limits) -> String { + var reader = PureXML.Parsing.EventReader(input, limits: limits) + return drain(&reader) + } + + private func characterPath(_ input: String, _ limits: PureXML.Parsing.Limits) -> String { + var iterator = input.makeIterator() + var reader = PureXML.Parsing.EventReader(pulling: { iterator.next() }, limits: limits) + return drain(&reader) + } + + private func expectAgreement(_ label: String, _ input: String, limits: PureXML.Parsing.Limits = .default) { + let viaBytes = bytePath(input, limits) + let viaCharacters = characterPath(input, limits) + #expect( + viaBytes == viaCharacters, + "byte and Character paths diverged for \(label) on \(input.debugDescription)\n bytes: \(viaBytes)\n chars: \(viaCharacters)", + ) + } + + @Test("Carriage-return and line-feed folding in content agree on both paths") + func test_contentLineEndings() { + expectAgreement("lone CR", "x\ry") + expectAgreement("CRLF", "x\r\ny") + expectAgreement("CR then CRLF", "x\r\r\ny") + expectAgreement("only CR", "\r") + expectAgreement("CR then element", "x\r") + expectAgreement("leading CRs then content and element", "\r\r\rxy") + expectAgreement("multi-line then element mark", "line1\nline2\nline3") + expectAgreement("blank lines then element mark", "\n\n\n") + } + + @Test("Attribute-value normalization agrees on both paths") + func test_attributeNormalization() { + expectAgreement("lone CR attr", "") + expectAgreement("only CR attr", "") + expectAgreement("CR then CRLF attr", "") + expectAgreement("two lone CR attr", "") + expectAgreement("mixed whitespace attr", "") + expectAgreement("char-ref CR survives", "") + expectAgreement("char-ref LF survives", "") + expectAgreement("literal tab plus ref tab", "") + expectAgreement("multi-line attr then attr", "") + } + + @Test("Bracket and CDATA-close boundaries agree on both paths") + func test_bracketBoundaries() { + expectAgreement("trailing ]]", "]]") + expectAgreement("]]> error mark", "x]]>y") + expectAgreement("] then ]]>", "]x]]>y") + expectAgreement("triple ] gt", "]]]>x") + expectAgreement("scattered brackets", "a]b]c") + expectAgreement("newline then ]]>", "line1\n]]>x") + expectAgreement("leading greater-than", "

a > b

") + } + + @Test("Name boundaries and the ASCII/non-ASCII handoff agree on both paths") + func test_nameBoundaries() { + let short = PureXML.Parsing.Limits(maxNameLength: 3) + expectAgreement("over-long name", "", limits: short) + expectAgreement("over-long attribute name", "", limits: short) + expectAgreement("double colon name", "") + expectAgreement("leading colon name", "<:x/>") + expectAgreement("non-ASCII name start", "<\u{e9}x/>") + expectAgreement("ASCII then non-ASCII name", "") + expectAgreement("nested non-ASCII name", "") + expectAgreement("prefix then non-ASCII local", "") + } + + @Test("Error marks after multi-line byte runs agree on both paths") + func test_errorMarksAfterRuns() { + expectAgreement("raw less-than in attribute", "") + expectAgreement("invalid char in content", "x\u{0}y") + expectAgreement("invalid char in attribute", "") + expectAgreement("invalid char after run crossing newline", "hello\nworld\u{0}x") + expectAgreement("invalid char after two newlines", "ab\ncd\nef\u{0}g") + expectAgreement("missing space before attribute", "") + expectAgreement("missing space after newline", "") + expectAgreement("truncated content after newline", "line1\nline2<") + } + + @Test("Entity and ampersand handling agrees on both paths") + func test_entitiesAndAmpersands() { + expectAgreement("ampersand-free text", "just plain text") + expectAgreement("decoded entities", "

1 < 2 & ok

") + expectAgreement("undefined entity mark", "&unknown;") + expectAgreement("entity after run", "text & more &bad;") + expectAgreement("ampersand truncation after newlines", "\n\nbad &x") + expectAgreement("whitespace-only content", " ") + expectAgreement("tabs then element", "\t\t") + } +} diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 565bf25..3095a97 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -55,6 +55,8 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count). | 1 | allocation-free Reader.matches() | 0.414 s (26x) | unchanged | unchanged | | 2 | byte-backed Reader (owned UTF-8 storage, pointer-decoded scalars, byte-level literal match/consume) | 0.375 s (23x) | unchanged | unchanged | | 3 | bulk content runs (plain-ASCII character data scanned and consumed at the byte level, String built once per run) | 0.335 s (20x) | unchanged | unchanged | +| 4 | byte-level dispatch + name/attribute scanning (non-buffering `peekByte`, byte-mode `skipSpace`, all-ASCII `takeASCIIName`, `attributeRunBytes`); keeps the lookahead buffer empty so the existing byte fast paths engage | 0.217 s (13x) | unchanged | unchanged | +| 5 | `sawAmpersand` short-circuit (ampersand-free text returns verbatim, skipping the reference-decode/split/findings pass) and detecting the text-run-closing `<` via `peekByte` instead of buffering it (the buffered `<` had cascaded the next element's whole dispatch and name scan onto the Character path) | 0.157 s (9.5x) | unchanged | unchanged | ## Profile findings (first sample, parse path)