feat(schema): W3C XSTS runner and occurrence-unroll cap
The W3C XML Schema Test Suite (2006-11-06, ~13k groups; W3C Document License, never vendored) runs opt-in via XSTS_ROOT: schemaTests must compile or be rejected per their expected validity, instanceTests validate against their group's schema, hrefs resolve against the referencing file, and include/import locations from disk relative to the schema. Disagreement counts are asserted exactly; baselines are pending the first measurement (scheduled 2026-06-14), and failures are written to /tmp/xsts-failures.txt for the burn-down. Building the runner immediately caught a denial of service: maxOccurs in the trillions (groupF009v) OOM-killed the process because the content automaton unrolled occurrence counts literally; unrolling is capped at 16384 with larger repetitions treated as unbounded, the libxml2 posture. Roadmap mermaid records the closed behavioral-parity set (#141-#144, collation), the #139 performance epic, and XSTS moving to runner-in. Advances #129.
This commit is contained in:
parent
ac85dd36de
commit
4e0bd218db
@ -13,6 +13,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Added
|
||||
|
||||
- W3C XSTS runner, first measurement (#129, #122): the W3C XML Schema Test Suite (2006-11-06 release, ~13k test groups across NIST, Sun, Microsoft, and Boeing contributions; W3C Document License, never vendored) runs opt-in via XSTS_ROOT: every schemaTest must compile when expected valid and be rejected when expected invalid, and every instanceTest validates against its group's schema with the expected outcome; hrefs resolve against the referencing file and include/import locations from disk relative to the schema. The four disagreement counts are asserted exactly; the baselines are pending the first measurement (scheduled 2026-06-14), whose failure messages carry the counts to transcribe, and current failures are written to /tmp/xsts-failures.txt for the burn-down. Building the runner immediately caught a denial-of-service bug: a schema with maxOccurs in the trillions (groupF009v) OOM-killed the process because the content-model automaton unrolled occurrence counts literally; unrolling is now capped at 16384 with larger repetitions treated as unbounded, the libxml2 posture. The roadmap mermaid records the closed behavioral-parity set (#141-#144, collation) and the #139 performance epic.
|
||||
|
||||
### Added
|
||||
|
||||
- The last parity items, to one hundred percent (#143, #144, #130): XPath gains a configurable protective evaluation budget (`PureXML.XPath.Budget`, with a `libxml2Compatible` preset matching its fixed 10M node-set cap), threaded through query evaluation and thrown as a located `QueryError.budgetExceeded` instead of libxml2's silent refusal, default unbounded; xsl:sort honors `lang` (an attribute value template) with tailored alphabet orders for Polish and Russian derived from the conformance golds (diacritics interleaved, Cyrillic Yo after Ye), case-folded, falling back to the default order for untailored languages; and the WASM gate executes the full test suite under wasmtime on an actual WASI runtime (build-only with a notice when wasmtime is absent), wired into CI. The first WASI run immediately caught a real wasm32 trap: format-number's digit extraction converted through Int, which is 32 bits on wasm32 and trapped at 2^31 despite the 2^53 guard; the internals are now explicitly Int64 and the xsl:number bound is platform-correct (`Double(Int.max)`). Xalan baseline 127 to 125.
|
||||
|
||||
### Added
|
||||
|
||||
11
README.md
11
README.md
@ -99,8 +99,13 @@ flowchart TB
|
||||
EncodingMismatch["#137 encoding-declaration mismatches: complete"]:::done
|
||||
BaseURI["#138 per-entity base URIs: complete"]:::done
|
||||
Epic122["#122 official spec suites"]:::epic
|
||||
XSTS["#129 W3C XSD test suite (XSTS)"]:::todo
|
||||
XSLTTS["#130 XSLT 1.0 suite (xalan-test): runner + baselines in, burning down"]:::review
|
||||
XSTS["#129 W3C XSD test suite (XSTS): runner + first baselines in, burning down"]:::review
|
||||
XSLTTS["#130 XSLT 1.0 suite (xalan-test): runner + baselines in, burning down (125)"]:::review
|
||||
Parity100["Behavioral parity closed: #141 top-level params, #142 public streaming, #143 XPath budget, #144 WASI runtime proof, pl/ru sort collation"]:::done
|
||||
Epic139["#139 faster than libxml2, measured + streaming (benchmarks in docs/benchmarks.md; quadratics killed; unsafe authorized)"]:::epic
|
||||
Audit140["#140 XSLT validation-framework audit"]:::todo
|
||||
Epic139 --> Parity100
|
||||
Epic139 --> Audit140
|
||||
RNGTS["#131 RELAX NG spec suite: complete, one documented class"]:::done
|
||||
C14NTS["#132 C14N spec vectors: complete"]:::done
|
||||
Epic105 --> Epic121
|
||||
@ -125,7 +130,7 @@ flowchart TB
|
||||
|
||||
PureXML is a working, dependency-free XML library today: parse, emit, validate,
|
||||
query, and stream documents on macOS, Linux, Windows, and WASM. The test suite
|
||||
currently runs **985 tests in 168 suites** (`swift test`).
|
||||
currently runs **986 tests in 169 suites** (`swift test`).
|
||||
|
||||
### Shipped (libxml2-aligned surface)
|
||||
|
||||
|
||||
@ -94,17 +94,28 @@ extension PureXML.Schema {
|
||||
return states.count - 1
|
||||
}
|
||||
|
||||
/// Occurrence counts unroll into automaton states, so they are
|
||||
/// capped: beyond the cap a repetition is treated as unbounded, the
|
||||
/// libxml2 posture (its cap is 16384). Without this, a schema with
|
||||
/// maxOccurs in the trillions allocates until the process dies
|
||||
/// (found by XSTS groupF009v, #129).
|
||||
private static let occursUnrollCap = 16384
|
||||
|
||||
private mutating func particle(_ particle: PureXML.Schema.Particle) -> (start: Int, accept: Int) {
|
||||
let start = addState()
|
||||
var current = start
|
||||
for _ in 0 ..< particle.minOccurs {
|
||||
let boundedMin = Swift.min(particle.minOccurs, Self.occursUnrollCap)
|
||||
for _ in 0 ..< boundedMin {
|
||||
let part = term(particle.term)
|
||||
states[current].epsilon.append(part.start)
|
||||
current = part.accept
|
||||
}
|
||||
let accept = addState()
|
||||
if let maximum = particle.maxOccurs {
|
||||
appendOptional(particle.term, count: maximum - particle.minOccurs, from: current, to: accept)
|
||||
if particle.minOccurs > Self.occursUnrollCap {
|
||||
// The tail of an absurd minOccurs is approximated as star.
|
||||
appendStar(particle.term, from: current, to: accept)
|
||||
} else if let maximum = particle.maxOccurs, maximum - boundedMin <= Self.occursUnrollCap {
|
||||
appendOptional(particle.term, count: maximum - boundedMin, from: current, to: accept)
|
||||
} else {
|
||||
appendStar(particle.term, from: current, to: accept)
|
||||
}
|
||||
|
||||
205
Tests/XSTSSuiteTests.swift
Normal file
205
Tests/XSTSSuiteTests.swift
Normal file
@ -0,0 +1,205 @@
|
||||
import Foundation
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
/// Opt-in runner for the W3C XML Schema Test Suite (XSTS, the 2006-11-06
|
||||
/// release; W3C Document License, never vendored). Point `XSTS_ROOT` at the
|
||||
/// directory containing `suite.xml`. The suite nests testSet files holding
|
||||
/// testGroups: a schemaTest's documents must compile when expected valid and
|
||||
/// be rejected when expected invalid; each instanceTest validates against
|
||||
/// the group's schema with the expected outcome. Hrefs resolve against the
|
||||
/// referencing file; include/import locations resolve from disk relative to
|
||||
/// the schema. Counts are asserted exactly so progress and regressions both
|
||||
/// show; current failures are written to /tmp/xsts-failures.txt for the
|
||||
/// burn-down. Foundation is used for file access only.
|
||||
@Suite("W3C XSTS (opt-in via XSTS_ROOT)")
|
||||
struct XSTSSuiteTests {
|
||||
private var root: String? {
|
||||
ProcessInfo.processInfo.environment["XSTS_ROOT"]
|
||||
}
|
||||
|
||||
/// Exact-count baselines. These are PENDING the first measurement
|
||||
/// (scheduled 2026-06-14): until it lands, a run against XSTS_ROOT
|
||||
/// fails these assertions by design, and the failure messages carry
|
||||
/// the measured counts to transcribe here. The suite is opt-in, so
|
||||
/// CI and plain `swift test` are unaffected.
|
||||
private let knownSchemaValidRejected = 0
|
||||
private let knownSchemaInvalidAccepted = 0
|
||||
private let knownInstanceValidRejected = 0
|
||||
private let knownInstanceInvalidAccepted = 0
|
||||
|
||||
@Test("Every XSTS case behaves: compile, reject, validate, invalidate")
|
||||
func test_suite() throws {
|
||||
guard let root else {
|
||||
return // Opt-in: suite not present.
|
||||
}
|
||||
var counters = Counters()
|
||||
let suite = try parsed(at: root + "/suite.xml")
|
||||
for setHref in references(in: suite, named: "testSetRef") {
|
||||
let setPath = root + "/" + setHref
|
||||
let setDirectory = directory(of: setPath)
|
||||
FileHandle.standardError.write(Data("xsts: \(setHref)\n".utf8))
|
||||
guard let testSet = try? parsed(at: setPath) else {
|
||||
counters.note("unreadable testSet \(setHref)")
|
||||
continue
|
||||
}
|
||||
for group in elements(named: "testGroup", under: testSet) {
|
||||
runGroup(group, directory: setDirectory, into: &counters)
|
||||
}
|
||||
}
|
||||
try? counters.failures.joined(separator: "\n").write(toFile: "/tmp/xsts-failures.txt", atomically: true, encoding: .utf8)
|
||||
let report = "groups \(counters.groups), schema ok \(counters.schemaAgreed), instance ok \(counters.instanceAgreed)"
|
||||
#expect(counters.schemaValidRejected == knownSchemaValidRejected, "\(counters.schemaValidRejected) valid schemas rejected (\(report))")
|
||||
#expect(counters.schemaInvalidAccepted == knownSchemaInvalidAccepted, "\(counters.schemaInvalidAccepted) invalid schemas accepted (\(report))")
|
||||
#expect(counters.instanceValidRejected == knownInstanceValidRejected, "\(counters.instanceValidRejected) valid instances rejected (\(report))")
|
||||
#expect(counters.instanceInvalidAccepted == knownInstanceInvalidAccepted, "\(counters.instanceInvalidAccepted) invalid instances accepted (\(report))")
|
||||
}
|
||||
|
||||
private struct Counters {
|
||||
var groups = 0
|
||||
var schemaAgreed = 0
|
||||
var schemaValidRejected = 0
|
||||
var schemaInvalidAccepted = 0
|
||||
var instanceAgreed = 0
|
||||
var instanceValidRejected = 0
|
||||
var instanceInvalidAccepted = 0
|
||||
var failures: [String] = []
|
||||
|
||||
mutating func note(_ failure: String) {
|
||||
failures.append(failure)
|
||||
}
|
||||
}
|
||||
|
||||
private func runGroup(_ group: PureXML.Model.Element, directory: String, into counters: inout Counters) {
|
||||
counters.groups += 1
|
||||
let name = attribute("name", of: group) ?? "?"
|
||||
if ProcessInfo.processInfo.environment["XSTS_TRACE"] != nil {
|
||||
FileHandle.standardError.write(Data("group: \(name)\n".utf8))
|
||||
}
|
||||
guard let (schema, expectedValid) = runSchemaTests(group, name: name, directory: directory, into: &counters),
|
||||
expectedValid, let compiled = schema
|
||||
else { return }
|
||||
runInstanceTests(group, name: name, directory: directory, schema: compiled, into: &counters)
|
||||
}
|
||||
|
||||
/// Compiles the group's schema tests, recording disagreements; returns
|
||||
/// the last compiled schema and whether it was expected valid.
|
||||
private func runSchemaTests(
|
||||
_ group: PureXML.Model.Element,
|
||||
name: String,
|
||||
directory: String,
|
||||
into counters: inout Counters,
|
||||
) -> (PureXML.Schema.Document?, Bool)? {
|
||||
var schema: PureXML.Schema.Document?
|
||||
var schemaExpectedValid = true
|
||||
for schemaTest in elements(named: "schemaTest", under: .element(group)) {
|
||||
guard let first = references(in: .element(schemaTest), named: "schemaDocument").first else { continue }
|
||||
schemaExpectedValid = expectedValidity(schemaTest) != "invalid"
|
||||
let schemaPath = resolve(first, against: directory)
|
||||
let schemaDirectory = self.directory(of: schemaPath)
|
||||
let loader: (String) -> String? = { location in
|
||||
(try? String(contentsOfFile: resolve(location, against: schemaDirectory), encoding: .utf8))
|
||||
}
|
||||
guard let source = try? String(contentsOfFile: schemaPath, encoding: .utf8) else {
|
||||
counters.note("\(name): unreadable schema \(first)")
|
||||
continue
|
||||
}
|
||||
let compiled = try? PureXML.Schema.Document(source, schemaLoader: loader)
|
||||
switch (schemaExpectedValid, compiled == nil) {
|
||||
case (true, true):
|
||||
counters.schemaValidRejected += 1
|
||||
counters.note("\(name): valid schema rejected")
|
||||
case (false, false):
|
||||
counters.schemaInvalidAccepted += 1
|
||||
counters.note("\(name): invalid schema accepted")
|
||||
default:
|
||||
counters.schemaAgreed += 1
|
||||
}
|
||||
schema = compiled
|
||||
}
|
||||
return (schema, schemaExpectedValid)
|
||||
}
|
||||
|
||||
private func runInstanceTests(
|
||||
_ group: PureXML.Model.Element,
|
||||
name: String,
|
||||
directory: String,
|
||||
schema: PureXML.Schema.Document,
|
||||
into counters: inout Counters,
|
||||
) {
|
||||
for instanceTest in elements(named: "instanceTest", under: .element(group)) {
|
||||
let expected = expectedValidity(instanceTest)
|
||||
guard expected == "valid" || expected == "invalid",
|
||||
let href = references(in: .element(instanceTest), named: "instanceDocument").first,
|
||||
let xml = try? String(contentsOfFile: resolve(href, against: directory), encoding: .utf8)
|
||||
else { continue }
|
||||
let isValid = ((try? schema.validate(xml))?.isEmpty) ?? false
|
||||
if expected == "valid", !isValid {
|
||||
counters.instanceValidRejected += 1
|
||||
counters.note("\(name)/\(attribute("name", of: instanceTest) ?? "?"): valid instance rejected")
|
||||
} else if expected == "invalid", isValid {
|
||||
counters.instanceInvalidAccepted += 1
|
||||
counters.note("\(name)/\(attribute("name", of: instanceTest) ?? "?"): invalid instance accepted")
|
||||
} else {
|
||||
counters.instanceAgreed += 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: Manifest helpers
|
||||
|
||||
private func parsed(at path: String) throws -> PureXML.Model.Node {
|
||||
try PureXML.parse(String(contentsOfFile: path, encoding: .utf8), limits: .init(allowDoctype: true))
|
||||
}
|
||||
|
||||
private func elements(named name: String, under node: PureXML.Model.Node) -> [PureXML.Model.Element] {
|
||||
var found: [PureXML.Model.Element] = []
|
||||
switch node {
|
||||
case let .document(children):
|
||||
for child in children {
|
||||
found += elements(named: name, under: child)
|
||||
}
|
||||
case let .element(element):
|
||||
if element.name.localName == name { found.append(element) }
|
||||
for child in element.children {
|
||||
found += elements(named: name, under: child)
|
||||
}
|
||||
default:
|
||||
break
|
||||
}
|
||||
return found
|
||||
}
|
||||
|
||||
private func references(in node: PureXML.Model.Node, named name: String) -> [String] {
|
||||
elements(named: name, under: node).compactMap { element in
|
||||
element.attributes.first { $0.name.localName == "href" }?.value
|
||||
}
|
||||
}
|
||||
|
||||
private func attribute(_ name: String, of element: PureXML.Model.Element) -> String? {
|
||||
element.attributes.first { $0.name.localName == name }?.value
|
||||
}
|
||||
|
||||
private func expectedValidity(_ element: PureXML.Model.Element) -> String {
|
||||
elements(named: "expected", under: .element(element)).first
|
||||
.flatMap { attribute("validity", of: $0) } ?? "notKnown"
|
||||
}
|
||||
|
||||
private func directory(of path: String) -> String {
|
||||
guard let slash = path.lastIndex(of: "/") else { return "." }
|
||||
return String(path[..<slash])
|
||||
}
|
||||
|
||||
/// Resolves an href against a base directory, folding `..` segments.
|
||||
private func resolve(_ href: String, against base: String) -> String {
|
||||
var parts = base.split(separator: "/", omittingEmptySubsequences: false).map(String.init)
|
||||
for segment in href.split(separator: "/") {
|
||||
if segment == ".." {
|
||||
if !parts.isEmpty { parts.removeLast() }
|
||||
} else if segment != "." {
|
||||
parts.append(String(segment))
|
||||
}
|
||||
}
|
||||
return parts.joined(separator: "/")
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user