diff --git a/CHANGELOG.md b/CHANGELOG.md index 5ad93f8..c798f49 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- RELAX NG spec test suite runner (#122, #131): James Clark's spectest.xml (385 cases, distributed with jing-trang) runs via an opt-in RNG_TS_ROOT runner, never vendored; the manifest is parsed with PureXML itself, nested testSuite scopes thread their resource/dir definitions into the compiler's loader, and each case asserts compile/reject for the schema plus accept/reject per instance. First measurement, carried as exact baselines: every correct schema compiles and most instances behave (valid 233/289 accepted, invalid 277/291 rejected), while schema-correctness validation is absent (all 213 incorrect schemas compile, count asserted exactly). The burn-down classes are recorded in the runner: schema validation against the RELAX NG grammar and the 4.16-4.18 restrictions, foreign-namespace stripping (4.1), xml:base-aware include/externalRef resolution (4.5), and the section 6 compatibility datatypes. - Canonical XML 1.0 spec-example vectors (#122, #132): the worked examples of the C14N 1.0 Recommendation's section 3 run as input/expected-output tests (cited per example; used under the W3C Software and Document License), all six passing: top-level PIs/comments with and without comments mode, whitespace preservation, start/end tags with attribute sorting and DTD defaults, character modifications and references, entity expansion, and UTF-8 output from legacy-encoded input. The vectors exposed three real conformance fixes: top-level siblings in the canonical form are now separated by single line feeds; an empty default-namespace declaration renders only when it undeclares a rendered non-empty default (superfluous `xmlns=""` is dropped, inclusive mode); and the text/attribute escape loops iterate Unicode scalars, so a carriage return that grapheme-clusters with a following line feed still escapes to ` ` (the #135 class, surfacing in output rather than lexing). - Scalar-level lexing (#135): the reader now buffers exactly one Unicode scalar per element (multi-scalar graphemes from any source are split, string input iterates scalars directly), so a combining mark directly after an ASCII delimiter can no longer merge with it into one Swift grapheme cluster and XML's scalar-defined name productions classify correctly: U+06D6 lexes as a Fifth Edition PI-target NameStartChar, ZWNJ/ZWJ attribute names parse, and U+309A markup from entity expansion forms a real element. The replacement-text, content-model, and ATTLIST sub-scanners and the entity-literal validator go scalar-level the same way, while grapheme clusters reassemble naturally wherever scanned text is appended to a String (asserted by test). This clears the last eduni baseline (now empty), grows the IBM pre-5e class from 279 to 300 exactly as the Fifth Edition prescribes, and, combined with the EDITION="1 2 3 4" exclusions the xmltest manifest itself carries for cases 140/141, empties the xmltest deviation baseline entirely: every xmlconf case applicable to a Fifth Edition, namespace-aware processor now passes. diff --git a/README.md b/README.md index 5da678e..7a3f25a 100644 --- a/README.md +++ b/README.md @@ -101,7 +101,7 @@ flowchart TB Epic122["#122 official spec suites"]:::epic XSTS["#129 W3C XSD test suite (XSTS)"]:::todo XSLTTS["#130 OASIS XSLT 1.0 suite (+XPath corpus)"]:::todo - RNGTS["#131 RELAX NG spec test suite"]:::todo + RNGTS["#131 RELAX NG spec suite: runner + baselines in, burning down"]:::review C14NTS["#132 C14N spec vectors: complete"]:::done Epic105 --> Epic121 Epic105 --> Epic122 @@ -125,7 +125,7 @@ flowchart TB PureXML is a working, dependency-free XML library today: parse, emit, validate, query, and stream documents on macOS, Linux, Windows, and WASM. The test suite -currently runs **931 tests in 159 suites** (`swift test`). +currently runs **932 tests in 160 suites** (`swift test`). ### Shipped (libxml2-aligned surface) diff --git a/Tests/RelaxNGSpecSuiteTests.swift b/Tests/RelaxNGSpecSuiteTests.swift new file mode 100644 index 0000000..1521897 --- /dev/null +++ b/Tests/RelaxNGSpecSuiteTests.swift @@ -0,0 +1,212 @@ +import Foundation +@testable import PureXML +import Testing + +/// Opt-in runner for James Clark's RELAX NG spec test suite (`spectest.xml`, +/// distributed with jing-trang under its BSD license). Point `RNG_TS_ROOT` at +/// the directory containing `spectest.xml` to run; without the variable the +/// suite is skipped and it is never vendored into this repository. The +/// manifest nests testSuite/testCase elements: an `incorrect` schema must +/// fail to compile, a `correct` schema must compile and then `valid` +/// instances validate while `invalid` instances do not. `resource` and `dir` +/// elements model the include/externalRef files a case references, served to +/// the compiler through its loader keyed by relative path. Foundation is used +/// for file access only. +@Suite("RELAX NG spec suite (opt-in via RNG_TS_ROOT)") +struct RelaxNGSpecSuiteTests { + private var root: String? { + ProcessInfo.processInfo.environment["RNG_TS_ROOT"] + } + + /// The burn-down frontier, by case index in document order (exact, so + /// progress and regressions both show). Three classes: + /// 1. Schema correctness is not validated: every `incorrect` schema + /// compiles (the count is asserted exactly below). Closing it means + /// validating schemas against the RELAX NG grammar and the section + /// 4.16-4.18 restrictions before pattern interpretation. + /// 2. Simplification gaps reject valid instances: foreign-namespace + /// element/attribute stripping (4.1: cases 90-98), xml:base-aware + /// include/externalRef resolution (4.5: 100-101), include/div/override + /// depth (4.6-4.12), the 4.17-4.18 normalization corners, and the + /// section 6 compatibility datatypes (ID/IDREF, 6.2.x). + /// 3. The same classes accept some `invalid` instances. + private let knownIncorrectCompiledCount = 213 + private let knownCorrectRejected: Set = [] + private let knownValidRejected: Set = [ + 90, 92, 93, 94, 95, 96, 98, 100, 101, 104, 108, 109, 110, 111, 115, + 119, 120, 124, 125, 126, 127, 128, 130, 131, 132, 133, 142, 190, 191, + 194, 195, 208, 209, 210, 236, 256, 258, 264, 265, 266, 268, 269, 271, + 328, 372, 374, 378, 379, 380, + ] + private let knownInvalidAccepted: Set = [ + 101, 104, 124, 125, 126, 127, 128, 142, 264, 271, + ] + + private struct SpecCase { + let index: Int + let section: String + let schema: String + let mustCompile: Bool + let validInstances: [String] + let invalidInstances: [String] + let resources: [String: String] + } + + private final class Collector { + var cases: [SpecCase] = [] + var counter = 0 + } + + private func loadCases(from path: String) throws -> [SpecCase] { + let source = try String(contentsOfFile: path, encoding: .utf8) + let document = try PureXML.parse(source, limits: .init(allowDoctype: true)) + guard case let .document(children) = document, + let suite = children.compactMap(\.element).first + else { return [] } + let collector = Collector() + collect(suite, resources: [:], collector: collector) + return collector.cases + } + + /// Walks nested testSuite elements, accumulating the `resource`/`dir` + /// definitions in scope, and turns each testCase into a SpecCase. + private func collect(_ element: PureXML.Model.Element, resources: [String: String], collector: Collector) { + var scope = resources + for child in element.children.compactMap(\.element) { + switch child.name.localName { + case "resource", "dir": + merge(child, prefix: "", into: &scope) + default: + break + } + } + for child in element.children.compactMap(\.element) { + switch child.name.localName { + case "testSuite": + collect(child, resources: scope, collector: collector) + case "testCase": + appendCase(child, resources: scope, collector: collector) + default: + break + } + } + } + + /// Files a `resource` (name -> serialized content) or recurses into a + /// `dir`, prefixing names with the directory path. + private func merge(_ element: PureXML.Model.Element, prefix: String, into scope: inout [String: String]) { + let name = element.attributes.first { $0.name.description == "name" }?.value ?? "" + if element.name.localName == "resource" { + if let content = firstElementChild(element) { + scope[prefix + name] = PureXML.serialize(.element(content)) + } + return + } + for child in element.children.compactMap(\.element) { + merge(child, prefix: prefix + name + "/", into: &scope) + } + } + + private func firstElementChild(_ element: PureXML.Model.Element) -> PureXML.Model.Element? { + element.children.compactMap(\.element).first + } + + /// The wrapper's first element child, serialized back to markup. + private func serializedChild(_ element: PureXML.Model.Element) -> String? { + firstElementChild(element).map { PureXML.serialize(.element($0)) } + } + + private func appendCase(_ testCase: PureXML.Model.Element, resources: [String: String], collector: Collector) { + var scope = resources + var section = "" + var schema: String? + var mustCompile = true + var valid: [String] = [] + var invalid: [String] = [] + for child in testCase.children.compactMap(\.element) { + switch child.name.localName { + case "section": + section = child.children.compactMap { if case let .text(value) = $0 { value } else { nil } }.joined() + case "resource", "dir": + merge(child, prefix: "", into: &scope) + case "correct", "incorrect": + mustCompile = child.name.localName == "correct" + schema = serializedChild(child) + case "valid": + serializedChild(child).map { valid.append($0) } + case "invalid": + serializedChild(child).map { invalid.append($0) } + default: + break + } + } + guard let schema else { return } + collector.counter += 1 + collector.cases.append(SpecCase( + index: collector.counter, + section: section, + schema: schema, + mustCompile: mustCompile, + validInstances: valid, + invalidInstances: invalid, + resources: scope, + )) + } + + @Test("Every spec case behaves: compile, reject, validate, invalidate") + func test_specSuite() throws { + guard let root else { + return // Opt-in: suite not present. + } + let cases = try loadCases(from: root + "/spectest.xml") + var incorrectAccepted: [Int] = [] + var correctRejected: [String: String] = [:] + var validRejected: [String] = [] + var invalidAccepted: [String] = [] + for specCase in cases { + let resources = specCase.resources + let loader: (String) -> String? = { resources[$0] } + let compiled = try? PureXML.Schema.RelaxNG(specCase.schema, schemaLoader: loader) + guard specCase.mustCompile else { + if compiled != nil { + incorrectAccepted.append(specCase.index) + } + continue + } + guard let compiled else { + if !knownCorrectRejected.contains(specCase.index) { + correctRejected["#\(specCase.index) §\(specCase.section)"] = compileError(specCase, loader: loader) + } + continue + } + for (offset, instance) in specCase.validInstances.enumerated() { + let accepted = (try? compiled.validate(instance)) == true + if !accepted, !knownValidRejected.contains(specCase.index) { + validRejected.append("#\(specCase.index).\(offset) §\(specCase.section)") + } + } + for (offset, instance) in specCase.invalidInstances.enumerated() { + let accepted = (try? compiled.validate(instance)) == true + if accepted, !knownInvalidAccepted.contains(specCase.index) { + invalidAccepted.append("#\(specCase.index).\(offset) §\(specCase.section)") + } + } + } + #expect( + incorrectAccepted.count == knownIncorrectCompiledCount, + "incorrect-schema class drifted: \(incorrectAccepted.count) (baseline \(knownIncorrectCompiledCount))", + ) + #expect(correctRejected.isEmpty, "rejected \(correctRejected.count) correct schemas: \(correctRejected)") + #expect(validRejected.isEmpty, "rejected \(validRejected.count) valid instances: \(validRejected)") + #expect(invalidAccepted.isEmpty, "accepted \(invalidAccepted.count) invalid instances: \(invalidAccepted)") + } + + private func compileError(_ specCase: SpecCase, loader: @escaping (String) -> String?) -> String { + do { + _ = try PureXML.Schema.RelaxNG(specCase.schema, schemaLoader: loader) + return "compiled on retry" + } catch { + return String(describing: error) + } + } +}