NodetteHQ/PureXML
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

perf(parsing): byte-level scanning fast paths halve parse time vs libxml2 (#275)

Browse Source 
Keep the reader on its byte fast path through dispatch and name/attribute
scanning so plain-ASCII documents avoid the per-Character String machinery
(String.Iterator.next, string comparison, small-string materialization,
bridge-object ARC) that a profile indicted as the parse bottleneck.

The root cause was that a single buffered lookahead Character defeated every
existing byte path, each of which requires an empty buffer, and the content,
markup, and attribute dispatch each peeked a Character before scanning. Add a
non-buffering peekByte for dispatch, a byte-mode skipSpace, an all-ASCII
takeASCIIName, and an attributeRunBytes bulk scanner (with the 3.3.3
whitespace-to-space transform, deferring any carriage return to the Character
path so CRLF is not double-counted). Detect a text run's closing '<' via
peekByte instead of buffering it, which had cascaded the next element's whole
dispatch and name scan onto the slow path. Skip reference decoding for
ampersand-free text. The Character path remains the exact fallback for a
streaming source, spliced entity text, and any non-ASCII byte; the two paths
share scanContentMarkup so they cannot diverge.

Measured parse 20.1x -> 9.5x slower than libxml2 on the 20k-item generated
corpus, roughly a 53 percent wall-clock reduction. A differential oracle pins
the byte and Character paths to identical event streams and error marks over
an adversarial corpus, alongside boundary tests. Verified against the full
local suite, the Apache Xalan XSLT gold-output corpus, the W3C XML conformance
corpora, the RELAX NG spec suite, and the WASM build. No spec behavior changed.
This commit is contained in:
Mihaela Mihaljevic 2026-06-24 18:46:58 +02:00 committed by GitHub
parent 3aca6c393a
commit 2f8ca9b289
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
10 changed files with 558 additions and 103 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
CHANGELOG.md
View File

@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
- Parser throughput on plain-ASCII documents roughly halved against the libxml2 reference (measured `parse` 20.1x -> 9.5x slower than libxml2 on the 20k-item generated corpus, `bash scripts/benchmark.sh 20000 5`; ~53% wall-clock reduction). The work was profile-driven (macOS `sample`, not guesswork): the scanner, not tree construction, dominated parse time (SAX is ~84% of parse), and within it the cost was per-`Character` `String` machinery (`String.Iterator.next`, `_stringCompareWithSmolCheck`, `_uncheckedFromUTF8`, bridge-object ARC). The root cause was that a single buffered lookahead `Character` defeated every existing byte fast path (each requires an empty buffer), and the content/markup/attribute dispatch all peeked a `Character` before scanning. The fix keeps the reader on the byte path: a non-buffering `peekByte` for dispatch, a byte-mode `skipSpace`, an all-ASCII `takeASCIIName`, an `attributeRunBytes` bulk scanner (with the 3.3.3 whitespace-to-space transform, deferring any carriage return to the Character path so CRLF is not double-counted), and a `sawAmpersand` flag so ampersand-free text skips reference decoding entirely. The single highest-leverage change was detecting the closing `<` of a text run via `peekByte` instead of buffering it, which had been cascading the next element's whole dispatch and name scan onto the slow path. The Character path remains the exact fallback for a streaming source, spliced entity text, and any non-ASCII byte; byte and Character paths are kept from diverging by `scanContentMarkup` (shared dispatch) and by `ByteFastPathTests`, which pins the fast-path/fallback boundaries (non-ASCII names, CRLF and character references in attribute values, ampersand-free vs entity-bearing text, over-long names, raw `<` in attributes). Verified against the full local suite (1668 tests) plus the Apache Xalan XSLT gold-output corpus, the W3C XML conformance corpora (xmltest, OASIS, Sun, Eduni, IBM, Japanese), and the RELAX NG spec suite, all green; no spec behavior changed.
## [0.3.0] - 2026-06-24
### Added

35
Sources/Parsing/CharacterClassification.swift Normal file
View File

@ -0,0 +1,35 @@
extension StringProtocol {
/// Trims leading and trailing XML whitespace without Foundation.
func trimmingXMLWhitespace() -> String {
var slice = self[...]
while let first = slice.first, first.isXMLWhitespace {
slice = slice.dropFirst()
}
while let last = slice.last, last.isXMLWhitespace {
slice = slice.dropLast()
}
return String(slice)
}
}
extension Character {
private typealias XML = PureXML.Parsing.XMLCharacter
/// XML S production: space, tab, carriage return, line feed.
var isXMLWhitespace: Bool {
unicodeScalars.count == 1 && unicodeScalars.first.map(XML.isWhitespace) == true
}
/// Whether this character may start an XML name. A grapheme qualifies when its
/// first scalar is a NameStartChar and any trailing scalars (combining marks)
/// are NameChar.
var isXMLNameStart: Bool {
guard let first = unicodeScalars.first, XML.isNameStart(first) else { return false }
return unicodeScalars.dropFirst().allSatisfy(XML.isNameChar)
}
/// Whether this character may continue an XML name (every scalar a NameChar).
var isXMLNameContinuation: Bool {
!unicodeScalars.isEmpty && unicodeScalars.allSatisfy(XML.isNameChar)
}
}

57
Sources/Parsing/EventReader.swift
View File

@ -145,34 +145,28 @@ public extension PureXML.Parsing {
// replacement into the reader with no text before it: the next
// construct is then read from the spliced replacement.
while true {
// Fast byte dispatch: when the next input is a raw byte (an
// empty lookahead buffer over owned storage), route on it
// without buffering a Character, so every matches()/consume()/
// scan call below stays on its byte fast path. The Character
// path is the identical fallback for a streaming source,
// spliced entity text, or a non-ASCII lead.
if let byte = reader.peekByte() {
eventStart = reader.mark
if byte == 0x3C { return try scanContentMarkup() } // '<'
if let text = try scanText() { return text }
continue
}
guard let lead = reader.peek() else {
throw ParseError.unexpectedEndOfInput(reader.mark)
}
eventStart = reader.mark
if reader.matches("</") { return try scanEndTag() }
if reader.matches("<!--") {
recordEmptyElementContent("a comment")
return try scanComment()
}
if reader.matches("<![CDATA[") { return try scanCDATA() }
if reader.matches("<?") {
let mark = reader.mark
recordEmptyElementContent("a processing instruction")
let instruction = try scanProcessingInstruction()
if instruction.target.lowercased() == "xml" {
throw ParseError.reservedProcessingInstructionTarget(mark)
}
return .processingInstruction(target: instruction.target, data: instruction.data)
}
if reader.matches("<!DOCTYPE") {
throw ParseError.unsupportedDoctype(reader.mark)
}
if lead == "<" { return try scanStartTag() }
if lead == "<" { return try scanContentMarkup() }
if let text = try scanText() { return text }
}
}
private mutating func scanStartTag() throws -> Event {
mutating func scanStartTag() throws -> Event {
guard open.count < limits.maxDepth else {
throw ParseError.nestingTooDeep(limit: limits.maxDepth, reader.mark)
}
@ -207,17 +201,26 @@ public extension PureXML.Parsing {
let mark = reader.mark
var raw = ""
var length = 0
// Whether any '&' was seen: without one the text has no references
// and no markup-entity boundary, so the whole decode/split/findings
// pass is skipped. The scanners visit every byte already, so this
// costs nothing beyond a compare they were doing anyway.
var sawAmpersand = false
while true {
// Bulk byte runs first: plain ASCII content (the bulk of any
// document) skips the per-character machinery entirely; the
// run never spans '<', CR, ']', or invalid bytes, so the
// character path below keeps exact error marks.
if let run = reader.contentRunBytes() {
if let run = reader.contentRunBytes(sawAmpersand: &sawAmpersand) {
length += run.utf8.count
try checkContent(length, mark)
raw += run
continue
}
// Detect the closing '<' on the byte path so the buffer stays
// empty: a buffered '<' would force the next element's dispatch,
// matches(), and name scan onto the slow Character path.
if reader.peekByte() == 0x3C { break }
guard let character = reader.peek(), character != "<" else { break }
length += 1
try checkContent(length, mark)
@ -227,10 +230,12 @@ public extension PureXML.Parsing {
guard character.unicodeScalars.allSatisfy(XMLCharacter.isChar) else {
throw ParseError.invalidCharacter(reader.mark)
}
if character == "&" { sawAmpersand = true }
raw.append(character)
reader.advance()
}
let split = raw.contains("&") ? EntityDecoder.splitAtMarkupEntity(raw, entities: referencableEntities) : nil
guard sawAmpersand else { return .characters(raw) }
let split = EntityDecoder.splitAtMarkupEntity(raw, entities: referencableEntities)
if let split {
let replacement = try EntityDecoder.includeForContent(
split.name,
@ -247,7 +252,7 @@ public extension PureXML.Parsing {
return .characters(decoded)
}
private mutating func scanProcessingInstruction() throws -> (target: String, data: String) {
mutating func scanProcessingInstruction() throws -> (target: String, data: String) {
let mark = reader.mark
reader.consume("<?")
let target = try scanName()
@ -280,6 +285,12 @@ public extension PureXML.Parsing {
mutating func scanName() throws -> PureXML.Model.QualifiedName {
let mark = reader.mark
// Fast path: an all-ASCII name scanned directly from the bytes, with
// no per-character buffering. The Character loop below handles a
// non-ASCII name, the non-fast-path, and the too-long error.
if let fast = reader.takeASCIIName(maxLength: limits.maxNameLength) {
return PureXML.Model.QualifiedName(fast)
}
guard let first = reader.peek(), first.isXMLNameStart else {
throw ParseError.expectedName(reader.mark)
}

36
Sources/Parsing/EventReaderAttributes.swift
View File

@ -6,8 +6,14 @@ extension PureXML.Parsing.EventReader {
var seen: Set<String> = []
while true {
reader.skipSpace()
guard let next = reader.peek(), next != ">", next != "/" else {
return attributes
// Byte dispatch keeps the buffer empty so scanName() reaches its
// byte fast path; the Character peek is the off-fast-path fallback.
if let byte = reader.peekByte() {
if byte == 0x3E || byte == 0x2F { return attributes } // '>' or '/'
} else {
guard let next = reader.peek(), next != ">", next != "/" else {
return attributes
}
}
let mark = reader.mark
let name = try scanName()
@ -27,8 +33,17 @@ extension PureXML.Parsing.EventReader {
throw PureXML.Parsing.ParseError.duplicateAttribute(name: name.description, mark)
}
attributes.append(PureXML.Model.Attribute(name: name, value: value))
// A following attribute needs whitespace before it.
if let next = reader.peek(), next != ">", next != "/", next != "?", !next.isWhitespace {
// A following attribute needs whitespace before it. Check the raw
// byte when on the fast path to avoid buffering the next name char.
if let byte = reader.peekByte() {
// '>', '/', '?', or whitespace may legally follow a value; any
// other byte means a missing separator before the next attribute.
let isSeparator = byte == 0x3E || byte == 0x2F || byte == 0x3F
|| byte == 0x20 || byte == 0x09 || byte == 0x0A || byte == 0x0D
if !isSeparator {
throw PureXML.Parsing.ParseError.missingSpaceBeforeAttribute(reader.mark)
}
} else if let next = reader.peek(), next != ">", next != "/", next != "?", !next.isWhitespace {
throw PureXML.Parsing.ParseError.missingSpaceBeforeAttribute(reader.mark)
}
}
@ -40,9 +55,20 @@ extension PureXML.Parsing.EventReader {
throw PureXML.Parsing.ParseError.unquotedAttributeValue(reader.mark)
}
reader.advance()
let quoteByte: UInt8 = quote == "\"" ? 0x22 : 0x27
var raw = ""
var length = 0
while let character = reader.peek(), character != quote {
while true {
// Bulk byte runs first: plain ASCII value text (the common case)
// skips the per-character machinery; the run stops before the
// quote, '<', CR, or any byte the Character path must inspect.
if let run = reader.attributeRunBytes(quote: quoteByte) {
length += run.utf8.count
try checkContent(length, mark)
raw += run
continue
}
guard let character = reader.peek(), character != quote else { break }
length += 1
try checkContent(length, mark)
if character == "<" {

28
Sources/Parsing/EventReaderContent.swift Normal file
View File

@ -0,0 +1,28 @@
/// Content markup dispatch, in its own file to keep the reader's primary type
/// body under the length cap. Behavior is part of EventReader proper.
extension PureXML.Parsing.EventReader {
/// At a '<' in element content, dispatches to the matching markup scanner.
/// The caller has confirmed the lead is '<' and set the event start. Shared
/// by the byte and Character dispatch paths so the two cannot diverge.
mutating func scanContentMarkup() throws -> Event {
if reader.matches("</") { return try scanEndTag() }
if reader.matches("<!--") {
recordEmptyElementContent("a comment")
return try scanComment()
}
if reader.matches("<![CDATA[") { return try scanCDATA() }
if reader.matches("<?") {
let mark = reader.mark
recordEmptyElementContent("a processing instruction")
let instruction = try scanProcessingInstruction()
if instruction.target.lowercased() == "xml" {
throw PureXML.Parsing.ParseError.reservedProcessingInstructionTarget(mark)
}
return .processingInstruction(target: instruction.target, data: instruction.data)
}
if reader.matches("<!DOCTYPE") {
throw PureXML.Parsing.ParseError.unsupportedDoctype(reader.mark)
}
return try scanStartTag()
}
}

246
Sources/Parsing/Reader.swift
View File

@ -130,6 +130,19 @@ extension PureXML.Parsing {
return ahead < buffer.count ? buffer[ahead] : nil
}
/// The raw byte `ahead` positions past the cursor, on the byte fast path
/// only (an empty lookahead buffer, no pending raw character, owned
/// storage). Returns nil off the fast path or at end of input. It does
/// not normalize line endings and does not advance: callers use it to
/// dispatch on fixed ASCII bytes ('<', '/', '>', quotes) while leaving
/// the buffer empty, so the byte scanners below stay engaged, and fall
/// back to the Character path for any byte it does not recognize.
func peekByte(_ ahead: Int = 0) -> UInt8? {
guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil }
let index = byteIndex + ahead
return index < storage.count ? storage.pointer[index] : nil
}
var mark: Mark {
Mark(line: line, column: column, offset: offset)
}
@ -182,6 +195,34 @@ extension PureXML.Parsing {
return true
}
/// On the byte fast path (an empty lookahead buffer over owned bytes),
/// consumes an all-ASCII XML name and returns it, advancing past it.
/// Returns nil when not on the fast path, when the first byte is not an
/// ASCII name-start, when the name reaches a non-ASCII byte, or when it
/// exceeds `maxLength`, so the caller falls back to the Character scanner
/// (which handles the full Unicode name grammar and the length error).
/// A name has no newlines, so position tracking is plain arithmetic.
mutating func takeASCIIName(maxLength: Int) -> String? {
guard buffer.isEmpty, pendingRaw == nil, let storage, byteIndex < storage.count else { return nil }
let pointer = storage.pointer
guard PureXML.Parsing.XMLCharacter.isASCIINameStart(pointer[byteIndex]) else { return nil }
var index = byteIndex + 1
let limit = byteIndex + maxLength
while index < storage.count {
let byte = pointer[index]
if byte >= 0x80 { return nil }
if !PureXML.Parsing.XMLCharacter.isASCIINameChar(byte) { break }
if index >= limit { return nil }
index += 1
}
let length = index - byteIndex
let run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: length), as: UTF8.self)
byteIndex = index
offset += length
column += length
return run
}
/// If the upcoming characters equal `literal`, consume them and return true.
@discardableResult
mutating func consume(_ literal: String) -> Bool {
@ -203,58 +244,35 @@ extension PureXML.Parsing {
}
mutating func skipSpace() {
if skipSpaceBytes() { return }
while let character = peek(), character.isXMLWhitespace {
advance()
}
}
/// Bulk-scans character data in byte mode: consumes and returns the
/// longest upcoming run of plain ASCII content bytes (no '<', no
/// carriage return, no "]]" pair, every byte a valid XML character),
/// or nil when the fast path does not apply. Anything subtle, an
/// entity boundary aside ('&' is plain content here), is left for
/// the character path so error marks stay exact. Returns nil rather
/// than an empty run so callers can alternate with the slow loop.
mutating func contentRunBytes() -> String? {
guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil }
/// Byte-mode whitespace skip. Returns true when it stopped at a clean
/// non-whitespace byte (so the buffer is empty and the caller needs no
/// Character loop); false when off the fast path or stopped at a
/// carriage return that the Character path must fold per 2.11.
private mutating func skipSpaceBytes() -> Bool {
guard buffer.isEmpty, pendingRaw == nil, let storage else { return false }
let pointer = storage.pointer
let count = storage.count
// A run never contains ']', so "]]>" can only straddle the
// boundary where the slow path consumed "]]" and the run would
// begin with '>': leave a leading '>' to the slow path and its
// cdataCloseInContent check (W3C ibm14n01).
if byteIndex < count, pointer[byteIndex] == 0x3E { return nil }
var index = byteIndex
var newlines = 0
var lastLineStart = -1
while index < count {
let byte = pointer[index]
if byte == 0x3C { break } // '<'
// Valid plain ASCII content only; CR, ']' and non-ASCII go
// to the character path.
guard byte < 0x80, byte != 0x0D, byte != 0x5D,
byte >= 0x20 || byte == 0x09 || byte == 0x0A
else {
if index == byteIndex { return nil }
break
while byteIndex < storage.count {
switch pointer[byteIndex] {
case 0x20, 0x09:
column += 1
case 0x0A:
line += 1
column = 1
case 0x0D:
return false
default:
return true
}
if byte == 0x0A {
newlines += 1
lastLineStart = index
}
index += 1
byteIndex += 1
offset += 1
}
guard index > byteIndex else { return nil }
let run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: index - byteIndex), as: UTF8.self)
offset += index - byteIndex
if newlines > 0 {
line += newlines
column = index - lastLineStart
} else {
column += index - byteIndex
}
byteIndex = index
return run
return true
}
/// Prepends `text` to the unread stream: used to splice a general
@ -267,38 +285,116 @@ extension PureXML.Parsing {
}
}
extension StringProtocol {
/// Trims leading and trailing XML whitespace without Foundation.
func trimmingXMLWhitespace() -> String {
var slice = self[...]
while let first = slice.first, first.isXMLWhitespace {
slice = slice.dropFirst()
/// The bulk byte-run scanners, in an extension so the reader's primary type
/// body stays under the length cap. They consume plain ASCII runs of content
/// or attribute-value text directly from the owned storage (the common case in
/// any real document) and hand subtle bytes back to the Character path. Both
/// stay on the byte fast path only when the lookahead buffer is empty.
extension PureXML.Parsing.Reader {
/// Bulk-scans character data in byte mode: consumes and returns the
/// longest upcoming run of plain ASCII content bytes (no '<', no
/// carriage return, no "]]" pair, every byte a valid XML character),
/// or nil when the fast path does not apply. Anything subtle, an
/// entity boundary aside ('&' is plain content here), is left for
/// the character path so error marks stay exact. Returns nil rather
/// than an empty run so callers can alternate with the slow loop.
mutating func contentRunBytes(sawAmpersand: inout Bool) -> String? {
guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil }
let pointer = storage.pointer
let count = storage.count
// A run never contains ']', so "]]>" can only straddle the
// boundary where the slow path consumed "]]" and the run would
// begin with '>': leave a leading '>' to the slow path and its
// cdataCloseInContent check (W3C ibm14n01).
if byteIndex < count, pointer[byteIndex] == 0x3E { return nil }
var index = byteIndex
var newlines = 0
var lastLineStart = -1
while index < count {
let byte = pointer[index]
if byte == 0x3C { break } // '<'
// Valid plain ASCII content only; CR, ']' and non-ASCII go
// to the character path.
guard byte < 0x80, byte != 0x0D, byte != 0x5D,
byte >= 0x20 || byte == 0x09 || byte == 0x0A
else {
if index == byteIndex { return nil }
break
}
if byte == 0x0A {
newlines += 1
lastLineStart = index
} else if byte == 0x26 {
sawAmpersand = true
}
index += 1
}
while let last = slice.last, last.isXMLWhitespace {
slice = slice.dropLast()
guard index > byteIndex else { return nil }
let run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: index - byteIndex), as: UTF8.self)
offset += index - byteIndex
if newlines > 0 {
line += newlines
column = index - lastLineStart
} else {
column += index - byteIndex
}
return String(slice)
}
}
extension Character {
private typealias XML = PureXML.Parsing.XMLCharacter
/// XML S production: space, tab, carriage return, line feed.
var isXMLWhitespace: Bool {
unicodeScalars.count == 1 && unicodeScalars.first.map(XML.isWhitespace) == true
}
/// Whether this character may start an XML name. A grapheme qualifies when its
/// first scalar is a NameStartChar and any trailing scalars (combining marks)
/// are NameChar.
var isXMLNameStart: Bool {
guard let first = unicodeScalars.first, XML.isNameStart(first) else { return false }
return unicodeScalars.dropFirst().allSatisfy(XML.isNameChar)
}
/// Whether this character may continue an XML name (every scalar a NameChar).
var isXMLNameContinuation: Bool {
!unicodeScalars.isEmpty && unicodeScalars.allSatisfy(XML.isNameChar)
byteIndex = index
return run
}
/// Bulk-scans an attribute value in byte mode: consumes and returns the
/// longest upcoming run of plain ASCII value bytes, stopping at the
/// closing `quote`, a raw '<', a carriage return (2.11 folding then
/// 3.3.3 normalization must run on the Character path), a control
/// character, or a non-ASCII byte. Tab and line feed inside the run
/// normalize to a single space per 3.3.3; '&' stays raw for later
/// reference decoding. Returns nil when the fast path does not apply or
/// the run would be empty, so callers alternate with the slow loop.
mutating func attributeRunBytes(quote: UInt8) -> String? {
guard buffer.isEmpty, pendingRaw == nil, let storage else { return nil }
let pointer = storage.pointer
let count = storage.count
var index = byteIndex
var newlines = 0
var lastLineStart = -1
var needsTransform = false
while index < count {
let byte = pointer[index]
if byte == quote || byte == 0x3C || byte == 0x0D || byte >= 0x80 { break }
if byte < 0x20 {
if byte == 0x09 || byte == 0x0A {
needsTransform = true
if byte == 0x0A {
newlines += 1
lastLineStart = index
}
} else {
break
}
}
index += 1
}
guard index > byteIndex else { return nil }
let length = index - byteIndex
let run: String
if needsTransform {
var bytes = [UInt8](repeating: 0, count: length)
for position in 0 ..< length {
let byte = pointer[byteIndex + position]
bytes[position] = (byte == 0x09 || byte == 0x0A) ? 0x20 : byte
}
run = String(decoding: bytes, as: UTF8.self)
} else {
run = String(decoding: UnsafeBufferPointer(start: pointer + byteIndex, count: length), as: UTF8.self)
}
offset += length
if newlines > 0 {
line += newlines
column = index - lastLineStart
} else {
column += length
}
byteIndex = index
return run
}
}

15
Sources/Parsing/XMLCharacter.swift
View File

@ -50,6 +50,21 @@ public extension PureXML.Parsing {
}
}
/// The ASCII subset of `NameStartChar`, tested on a raw byte for the
/// parser's byte-level fast path. A non-ASCII name-start (byte >= 0x80)
/// is left to the scalar predicate via the Character scanner.
static func isASCIINameStart(_ byte: UInt8) -> Bool {
switch byte {
case 0x3A, 0x41 ... 0x5A, 0x5F, 0x61 ... 0x7A: true
default: false
}
}
/// The ASCII subset of `NameChar` (after the first), on a raw byte.
static func isASCIINameChar(_ byte: UInt8) -> Bool {
isASCIINameStart(byte) || (byte >= 0x30 && byte <= 0x39) || byte == 0x2D || byte == 0x2E
}
/// `NameChar`: a name character after the first.
public static func isNameChar(_ scalar: Unicode.Scalar) -> Bool {
if isNameStart(scalar) { return true }

117
Tests/ByteFastPathTests.swift Normal file
View File

@ -0,0 +1,117 @@
import Testing
@testable import PureXML
/// Locks the parser's byte-level fast paths against their Character-path
/// fallbacks. Each case is chosen to land exactly on a boundary where the byte
/// scanner must hand off to the scalar scanner (a non-ASCII byte, a carriage
/// return, an entity reference) so that a future change to either path cannot
/// silently diverge. The expected values are the spec-correct results,
/// independent of which path produced them.
@Suite("Byte fast-path boundaries")
struct ByteFastPathTests {
@Test("An all-ASCII name uses the byte path; a non-ASCII name falls back")
func test_nameFallback() throws {
let node = try PureXML.parse("<caf\u{e9} na\u{ef}ve=\"x\">y</caf\u{e9}>")
let element = node.firstElement
#expect(element?.name.localName == "caf\u{e9}")
#expect(element?.attributes == [.init("na\u{ef}ve", "x")])
#expect(element?.text == "y")
}
@Test("A name whose ASCII prefix precedes a non-ASCII byte stays whole")
func test_mixedNameStaysWhole() throws {
// The byte scanner consumes "ab", meets the non-ASCII byte, returns nil,
// and the Character path re-scans the entire "ab\u{e9}" as one name.
let node = try PureXML.parse("<ab\u{e9}/>")
#expect(node.firstElement?.name.localName == "ab\u{e9}")
}
@Test("A prefixed name splits at the colon on the byte path")
func test_prefixedName() throws {
let node = try PureXML.parse("<m:row xmlns:m=\"urn:x\" m:k=\"v\"/>")
let element = node.firstElement
#expect(element?.name.prefix == "m")
#expect(element?.name.localName == "row")
#expect(element?.attributes.contains { $0.name.localName == "k" && $0.value == "v" } == true)
}
@Test("Literal whitespace in an attribute value normalizes to single spaces")
func test_attributeWhitespaceNormalizes() throws {
// Tab and line feed each become one space per 3.3.3; these run through
// the byte scanner's whitespace transform.
let node = try PureXML.parse("<a v=\"x\ty\nz\"/>")
#expect(node.firstElement?.attributes == [.init("v", "x y z")])
}
@Test("A CRLF in an attribute value collapses to a single space")
func test_attributeCRLF() throws {
// 2.11 folds CRLF to one LF before 3.3.3 turns it into one space. The
// byte scanner defers any carriage return to the Character path so the
// pair is not double-counted as two spaces.
let node = try PureXML.parse("<a v=\"x\r\ny\"/>")
#expect(node.firstElement?.attributes == [.init("v", "x y")])
}
@Test("A character reference in an attribute value survives normalization")
func test_attributeCharReferenceSurvives() throws {
// A referenced line feed is NOT subject to whitespace normalization: it
// stays a line feed while a literal line feed would have become a space.
let node = try PureXML.parse("<a v=\"x&#10;y\"/>")
#expect(node.firstElement?.attributes == [.init("v", "x\ny")])
}
@Test("Entities and bare ampersands in text decode through the slow path")
func test_textEntities() throws {
let node = try PureXML.parse("<p>plain text then 1 &lt; 2 &amp; ok</p>")
#expect(node.firstElement?.text == "plain text then 1 < 2 & ok")
}
@Test("Ampersand-free text is returned verbatim")
func test_textWithoutEntities() throws {
let node = try PureXML.parse("<p>just plain ascii content, no references</p>")
#expect(node.firstElement?.text == "just plain ascii content, no references")
}
@Test("A CRLF inside text content folds to a single line feed")
func test_textCRLF() throws {
let node = try PureXML.parse("<p>line one\r\nline two</p>")
#expect(node.firstElement?.text == "line one\nline two")
}
@Test("Non-ASCII text content round-trips unchanged")
func test_nonASCIIText() throws {
let node = try PureXML.parse("<p>caf\u{e9} na\u{ef}ve r\u{e9}sum\u{e9}</p>")
#expect(node.firstElement?.text == "caf\u{e9} na\u{ef}ve r\u{e9}sum\u{e9}")
}
@Test("An over-long ASCII name is rejected, not silently truncated")
func test_longNameRejected() {
let limits = PureXML.Parsing.Limits(maxNameLength: 4)
#expect(throws: PureXML.Parsing.ParseError.self) {
try PureXML.parse("<abcdefgh/>", limits: limits)
}
}
@Test("A raw less-than inside an attribute value is rejected")
func test_rawLessThanInAttribute() {
#expect(throws: PureXML.Parsing.ParseError.self) {
try PureXML.parse("<a v=\"x<y\"/>")
}
}
@Test("A leading greater-than in text content is preserved")
func test_textLeadingGreaterThan() throws {
let node = try PureXML.parse("<p>a > b</p>")
#expect(node.firstElement?.text == "a > b")
}
}
private extension PureXML.Model.Node {
var firstElement: PureXML.Model.Element? {
guard case let .document(children) = self else { return element }
for child in children {
if case let .element(element) = child { return element }
}
return nil
}
}

121
Tests/ParserPathDifferentialTests.swift Normal file
View File

@ -0,0 +1,121 @@
import Testing
@testable import PureXML
/// A differential oracle for the parser's byte-level fast paths. The byte path
/// (a reader built from a `String`, which owns its UTF-8 and scans straight off
/// the bytes) and the streaming Character path (a reader built from a pulling
/// closure, whose `storage` is nil so every byte method returns nil) are two
/// implementations of the same grammar. They must therefore emit the identical
/// event stream AND, on malformed input, fail at the identical source mark, for
/// every input. Any divergence is a fast-path bug. The corpus is chosen to land
/// on the boundaries where the two paths differ in mechanism: carriage-return
/// folding, attribute-value normalization, multi-line position tracking, the
/// `]]>` guard, name limits, and the ASCII/non-ASCII handoff.
@Suite("Parser byte/Character path differential")
struct ParserPathDifferentialTests {
private func drain(_ reader: inout PureXML.Parsing.EventReader) -> String {
var out = ""
do {
while let event = try reader.next() {
let mark = reader.eventStart
out += "\(event)@\(mark.line):\(mark.column):\(mark.offset)|"
}
out += "EOF"
} catch {
out += "ERR:\(error)"
}
return out
}
private func bytePath(_ input: String, _ limits: PureXML.Parsing.Limits) -> String {
var reader = PureXML.Parsing.EventReader(input, limits: limits)
return drain(&reader)
}
private func characterPath(_ input: String, _ limits: PureXML.Parsing.Limits) -> String {
var iterator = input.makeIterator()
var reader = PureXML.Parsing.EventReader(pulling: { iterator.next() }, limits: limits)
return drain(&reader)
}
private func expectAgreement(_ label: String, _ input: String, limits: PureXML.Parsing.Limits = .default) {
let viaBytes = bytePath(input, limits)
let viaCharacters = characterPath(input, limits)
#expect(
viaBytes == viaCharacters,
"byte and Character paths diverged for \(label) on \(input.debugDescription)\n bytes: \(viaBytes)\n chars: \(viaCharacters)",
)
}
@Test("Carriage-return and line-feed folding in content agree on both paths")
func test_contentLineEndings() {
expectAgreement("lone CR", "<a>x\ry</a>")
expectAgreement("CRLF", "<a>x\r\ny</a>")
expectAgreement("CR then CRLF", "<a>x\r\r\ny</a>")
expectAgreement("only CR", "<a>\r</a>")
expectAgreement("CR then element", "<a>x\r<b/></a>")
expectAgreement("leading CRs then content and element", "<a>\r\r\rx<b/>y</a>")
expectAgreement("multi-line then element mark", "<a>line1\nline2\nline3<b/></a>")
expectAgreement("blank lines then element mark", "<a>\n\n\n<b/></a>")
}
@Test("Attribute-value normalization agrees on both paths")
func test_attributeNormalization() {
expectAgreement("lone CR attr", "<a v=\"x\ry\"/>")
expectAgreement("only CR attr", "<a v=\"\r\"/>")
expectAgreement("CR then CRLF attr", "<a v=\"x\r\r\ny\"/>")
expectAgreement("two lone CR attr", "<a v=\"x\ry\rz\"/>")
expectAgreement("mixed whitespace attr", "<a v=\"\t\r\n \"/>")
expectAgreement("char-ref CR survives", "<a v=\"x&#13;y\"/>")
expectAgreement("char-ref LF survives", "<a v=\"x&#10;y\"/>")
expectAgreement("literal tab plus ref tab", "<a v=\"a\tb&#9;c\"/>")
expectAgreement("multi-line attr then attr", "<a b=\"\nval\nval2\" c=\"y\"/>")
}
@Test("Bracket and CDATA-close boundaries agree on both paths")
func test_bracketBoundaries() {
expectAgreement("trailing ]]", "<a>]]</a>")
expectAgreement("]]> error mark", "<a>x]]>y</a>")
expectAgreement("] then ]]>", "<a>]x]]>y</a>")
expectAgreement("triple ] gt", "<a>]]]>x</a>")
expectAgreement("scattered brackets", "<a>a]b]c</a>")
expectAgreement("newline then ]]>", "<a>line1\n]]>x</a>")
expectAgreement("leading greater-than", "<p>a > b</p>")
}
@Test("Name boundaries and the ASCII/non-ASCII handoff agree on both paths")
func test_nameBoundaries() {
let short = PureXML.Parsing.Limits(maxNameLength: 3)
expectAgreement("over-long name", "<abcd/>", limits: short)
expectAgreement("over-long attribute name", "<aaa bcde=\"x\"/>", limits: short)
expectAgreement("double colon name", "<a:b:c/>")
expectAgreement("leading colon name", "<:x/>")
expectAgreement("non-ASCII name start", "<\u{e9}x/>")
expectAgreement("ASCII then non-ASCII name", "<ab\u{e9}/>")
expectAgreement("nested non-ASCII name", "<a><b\u{e9}c/></a>")
expectAgreement("prefix then non-ASCII local", "<x:\u{e9}/>")
}
@Test("Error marks after multi-line byte runs agree on both paths")
func test_errorMarksAfterRuns() {
expectAgreement("raw less-than in attribute", "<a v=\"x<y\"/>")
expectAgreement("invalid char in content", "<a>x\u{0}y</a>")
expectAgreement("invalid char in attribute", "<a v=\"x\u{0}y\"/>")
expectAgreement("invalid char after run crossing newline", "<a>hello\nworld\u{0}x</a>")
expectAgreement("invalid char after two newlines", "<a>ab\ncd\nef\u{0}g</a>")
expectAgreement("missing space before attribute", "<a b=\"1\"c=\"2\"/>")
expectAgreement("missing space after newline", "<a\nb=\"1\"c=\"2\"/>")
expectAgreement("truncated content after newline", "<a>line1\nline2<")
}
@Test("Entity and ampersand handling agrees on both paths")
func test_entitiesAndAmpersands() {
expectAgreement("ampersand-free text", "<a>just plain text</a>")
expectAgreement("decoded entities", "<p>1 &lt; 2 &amp; ok</p>")
expectAgreement("undefined entity mark", "<a>&unknown;</a>")
expectAgreement("entity after run", "<a>text &amp; more &bad;</a>")
expectAgreement("ampersand truncation after newlines", "<a>\n\nbad &x</a>")
expectAgreement("whitespace-only content", "<a> </a>")
expectAgreement("tabs then element", "<a>\t\t<b/></a>")
}
}

2
docs/benchmarks.md
View File

@ -55,6 +55,8 @@ offsets throughout; libxml2's xmlReadMemory takes an int byte count).
| 1 | allocation-free Reader.matches() | 0.414 s (26x) | unchanged | unchanged |
| 2 | byte-backed Reader (owned UTF-8 storage, pointer-decoded scalars, byte-level literal match/consume) | 0.375 s (23x) | unchanged | unchanged |
| 3 | bulk content runs (plain-ASCII character data scanned and consumed at the byte level, String built once per run) | 0.335 s (20x) | unchanged | unchanged |
| 4 | byte-level dispatch + name/attribute scanning (non-buffering `peekByte`, byte-mode `skipSpace`, all-ASCII `takeASCIIName`, `attributeRunBytes`); keeps the lookahead buffer empty so the existing byte fast paths engage | 0.217 s (13x) | unchanged | unchanged |
| 5 | `sawAmpersand` short-circuit (ampersand-free text returns verbatim, skipping the reference-decode/split/findings pass) and detecting the text-run-closing `<` via `peekByte` instead of buffering it (the buffered `<` had cascaded the next element's whole dispatch and name scan onto the Character path) | 0.157 s (9.5x) | unchanged | unchanged |
## Profile findings (first sample, parse path)