feat(relaxng): QName values with dual-context resolution, instance level clean

A value of type QName compiles to its (namespace, local) pair via the
schema's xmlns/ns scope; the derivative engine threads the instance
element's in-scope bindings to value comparison in both the tree walk
and the streaming state (frames carry their bindings), so instance
prefixes and the default namespace resolve against the instance
document and unbound prefixes match nothing. Streaming also gains the
tree walk's empty-element rule. The spec suite's instance level is
fully clean: valid 289/289, invalid 291/291; only the asserted
schema-correctness class remains.
This commit is contained in:
Mihaela Mihaljevic 2026-06-10 10:57:14 +02:00
parent d0175d5ca0
commit c628682086
7 changed files with 152 additions and 53 deletions

View File

@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- RELAX NG QName values, instance level clean (#131): `<value type="QName">` compiles to its (namespace, local-name) value pair via the schema's xmlns/ns scope, and the derivative engine threads the instance element's in-scope namespace bindings down to value comparison (in both the tree walk and the streaming state, whose frames now carry their bindings), so an instance QName resolves its prefix, or the default namespace when unprefixed, against the instance document itself; an unbound prefix matches nothing. The streaming path also gains the tree walk's empty-element rule. With this the spec suite's instance level is fully clean: valid 289/289 accepted and invalid 291/291 rejected; only the asserted schema-correctness class remains.
- RELAX NG grammar scoping and include depth, third burn-down (#131): each `grammar` element opens its own define scope (names are scope-mangled in the flat table, so nested grammars do not collide) and `parentRef` reaches the enclosing scope; `include` is transitive (an included grammar's own includes merge first) and a define carried by the include element replaces the included grammar's same-name define (the 4.7 override); `div` wrappers flatten into their grammar while still contributing ns/datatypeLibrary inheritance (4.11); `value` text is kept untrimmed so whitespace-preserving datatypes compare exactly (6.2.9); and `dataExcept` recurses through choice patterns. The spec suite is now clean except one class: valid 286/289 and invalid 287/291, with all six remaining instance cases the QName datatype (its value space needs prefix resolution in both the schema and instance contexts), plus the asserted schema-correctness item.
- RELAX NG namespaces and semantics, second burn-down (#131): the compiler now implements `ns` inheritance and the QName form of names (4.9-4.10: `ns` walks ancestors and crosses include/externalRef document boundaries; `name="eg:foo"` and `<name>eg:foo</name>` resolve their prefix against the schema's in-scope xmlns declarations; the `name=` shorthand on an attribute stays namespace-less rather than inheriting), an `include` brings in the included grammar's `start` with an include-level override honored (4.7), multiple `<start>` elements combine like defines and a single define or start may omit `combine` while still merging by the named method (4.17), `<data>` supports `<except>` via a new `dataExcept` pattern in the derivative engine (4.12), an empty element matches a data/value pattern whose lexical space admits the empty string (Clark's empty-text rule), and a whitespace-only attribute value matches a pattern that matches empty. Spec-suite baselines tighten from valid 242/289 to 274/289 and invalid 278/291 to 279/291; the name-class and tree-helper sections split to their own files for the length caps.
- RELAX NG simplification, first burn-down (#131): foreign-namespace elements are stripped before pattern interpretation (4.1: only RELAX NG-namespace elements are schema content; a namespace-less element counts only inside a fully unqualified schema), name/type/combine attribute values are whitespace-trimmed (4.2), and include/externalRef hrefs resolve against the xml:base attributes in scope and the loaded document's own URI (4.5, reusing the RFC 3986 resolver), with the document base threaded through nested loads and visited-tracking keyed on resolved hrefs. Spec-suite baselines tighten: valid 242/289 accepted (from 233), invalid 278/291 rejected.

View File

@ -125,7 +125,7 @@ flowchart TB
PureXML is a working, dependency-free XML library today: parse, emit, validate,
query, and stream documents on macOS, Linux, Windows, and WASM. The test suite
currently runs **933 tests in 160 suites** (`swift test`).
currently runs **935 tests in 161 suites** (`swift test`).
### Shipped (libxml2-aligned surface)

View File

@ -14,7 +14,34 @@ extension PureXML.Schema {
/// Whether `root` matches `start`.
func matches(start: PureXML.Schema.Pattern, root: PureXML.Model.Node) -> Bool {
nullable(childDeriv(start, root))
nullable(childDeriv(start, root, env: Self.rootEnvironment))
}
/// The namespace bindings every instance starts with.
static let rootEnvironment: [String: String] = ["xml": "http://www.w3.org/XML/1998/namespace"]
/// `env` extended with the element's own xmlns declarations.
static func extendedEnvironment(_ env: [String: String], with element: PureXML.Model.Element) -> [String: String] {
var extended = env
for attribute in element.attributes where isNamespaceDeclaration(attribute) {
let prefix = attribute.name.prefix == "xmlns" ? attribute.name.localName : ""
extended[prefix] = attribute.value
}
return extended
}
/// Resolves an instance-side QName text against the in-scope bindings:
/// a prefix must be bound; an unprefixed name takes the default
/// namespace (the xs:QName rules).
static func resolveQNameValue(_ raw: String, env: [String: String]) -> (namespace: String, localName: String)? {
let trimmed = raw.trimmingXMLWhitespace()
guard let colon = trimmed.firstIndex(of: ":") else {
return (env[""] ?? "", trimmed)
}
let prefix = String(trimmed[..<colon])
let local = String(trimmed[trimmed.index(after: colon)...])
guard let uri = env[prefix], !uri.isEmpty else { return nil }
return (uri, local)
}
// MARK: Simplifying constructors
@ -70,36 +97,38 @@ extension PureXML.Schema {
// MARK: Text and attribute derivatives
func textDeriv(_ pattern: PureXML.Schema.Pattern, _ string: String) -> PureXML.Schema.Pattern {
func textDeriv(_ pattern: PureXML.Schema.Pattern, _ string: String, env: [String: String] = rootEnvironment) -> PureXML.Schema.Pattern {
switch pattern {
case let .choice(lhs, rhs): return choice(textDeriv(lhs, string), textDeriv(rhs, string))
case let .choice(lhs, rhs): return choice(textDeriv(lhs, string, env: env), textDeriv(rhs, string, env: env))
case let .interleave(lhs, rhs):
return choice(interleave(textDeriv(lhs, string), rhs), interleave(lhs, textDeriv(rhs, string)))
return choice(interleave(textDeriv(lhs, string, env: env), rhs), interleave(lhs, textDeriv(rhs, string, env: env)))
case let .group(lhs, rhs):
let derived = group(textDeriv(lhs, string), rhs)
return nullable(lhs) ? choice(derived, textDeriv(rhs, string)) : derived
case let .after(lhs, rhs): return after(textDeriv(lhs, string), rhs)
let derived = group(textDeriv(lhs, string, env: env), rhs)
return nullable(lhs) ? choice(derived, textDeriv(rhs, string, env: env)) : derived
case let .after(lhs, rhs): return after(textDeriv(lhs, string, env: env), rhs)
case let .oneOrMore(inner):
return group(textDeriv(inner, string), choice(.oneOrMore(inner), .empty))
return group(textDeriv(inner, string, env: env), choice(.oneOrMore(inner), .empty))
case .text: return .text
case let .ref(name): return textDeriv(resolve(name), string)
default: return valueDeriv(pattern, string)
case let .ref(name): return textDeriv(resolve(name), string, env: env)
default: return valueDeriv(pattern, string, env: env)
}
}
private func valueDeriv(_ pattern: PureXML.Schema.Pattern, _ string: String) -> PureXML.Schema.Pattern {
private func valueDeriv(_ pattern: PureXML.Schema.Pattern, _ string: String, env: [String: String]) -> PureXML.Schema.Pattern {
switch pattern {
case let .value(type, literal):
type.valueMatches(string, literal: literal) ? .empty : .notAllowed
case let .data(type): type.validate(string) == nil ? .empty : .notAllowed
case let .dataExcept(type, except):
type.validate(string) == nil && !nullable(textDeriv(except, string)) ? .empty : .notAllowed
case let .list(inner): listMatches(inner, string) ? .empty : .notAllowed
type.validate(string) == nil && !nullable(textDeriv(except, string, env: env)) ? .empty : .notAllowed
case let .valueQName(namespace, localName):
Self.resolveQNameValue(string, env: env).map { $0 == (namespace, localName) } == true ? .empty : .notAllowed
case let .list(inner): listMatches(inner, string, env: env) ? .empty : .notAllowed
default: .notAllowed
}
}
private func listMatches(_ pattern: PureXML.Schema.Pattern, _ string: String) -> Bool {
private func listMatches(_ pattern: PureXML.Schema.Pattern, _ string: String, env _: [String: String]) -> Bool {
var residual = pattern
for token in string.split(whereSeparator: \.isWhitespace) {
residual = textDeriv(residual, String(token))
@ -107,26 +136,26 @@ extension PureXML.Schema {
return nullable(residual)
}
func attributeDeriv(_ pattern: PureXML.Schema.Pattern, _ attribute: PureXML.Model.Attribute) -> PureXML.Schema.Pattern {
func attributeDeriv(_ pattern: PureXML.Schema.Pattern, _ attribute: PureXML.Model.Attribute, env: [String: String] = rootEnvironment) -> PureXML.Schema.Pattern {
switch pattern {
case let .choice(lhs, rhs): return choice(attributeDeriv(lhs, attribute), attributeDeriv(rhs, attribute))
case let .choice(lhs, rhs): return choice(attributeDeriv(lhs, attribute, env: env), attributeDeriv(rhs, attribute, env: env))
case let .group(lhs, rhs):
return choice(group(attributeDeriv(lhs, attribute), rhs), group(lhs, attributeDeriv(rhs, attribute)))
return choice(group(attributeDeriv(lhs, attribute, env: env), rhs), group(lhs, attributeDeriv(rhs, attribute, env: env)))
case let .interleave(lhs, rhs):
return choice(interleave(attributeDeriv(lhs, attribute), rhs), interleave(lhs, attributeDeriv(rhs, attribute)))
return choice(interleave(attributeDeriv(lhs, attribute, env: env), rhs), interleave(lhs, attributeDeriv(rhs, attribute, env: env)))
case let .oneOrMore(inner):
return group(attributeDeriv(inner, attribute), choice(.oneOrMore(inner), .empty))
case let .after(lhs, rhs): return after(attributeDeriv(lhs, attribute), rhs)
return group(attributeDeriv(inner, attribute, env: env), choice(.oneOrMore(inner), .empty))
case let .after(lhs, rhs): return after(attributeDeriv(lhs, attribute, env: env), rhs)
case let .attribute(nameClass, content):
guard nameClass.contains(attribute.name), valueMatches(content, attribute.value) else { return .notAllowed }
guard nameClass.contains(attribute.name), valueMatches(content, attribute.value, env: env) else { return .notAllowed }
return .empty
case let .ref(name): return attributeDeriv(resolve(name), attribute)
case let .ref(name): return attributeDeriv(resolve(name), attribute, env: env)
default: return .notAllowed
}
}
private func valueMatches(_ content: PureXML.Schema.Pattern, _ value: String) -> Bool {
if nullable(textDeriv(content, value)) { return true }
private func valueMatches(_ content: PureXML.Schema.Pattern, _ value: String, env: [String: String]) -> Bool {
if nullable(textDeriv(content, value, env: env)) { return true }
// A whitespace-only attribute value also matches a pattern that
// matches the empty sequence, the same leniency element content
// gets for ignorable whitespace.
@ -143,24 +172,25 @@ extension PureXML.Schema {
extension PureXML.Schema.RelaxNGEngine {
// MARK: Element derivatives
func childDeriv(_ pattern: PureXML.Schema.Pattern, _ node: PureXML.Model.Node) -> PureXML.Schema.Pattern {
func childDeriv(_ pattern: PureXML.Schema.Pattern, _ node: PureXML.Model.Node, env: [String: String] = rootEnvironment) -> PureXML.Schema.Pattern {
switch node {
case let .text(string), let .cdata(string):
textDeriv(pattern, string)
textDeriv(pattern, string, env: env)
case let .element(element):
processElement(pattern, element)
processElement(pattern, element, env: env)
default:
pattern
}
}
private func processElement(_ pattern: PureXML.Schema.Pattern, _ element: PureXML.Model.Element) -> PureXML.Schema.Pattern {
private func processElement(_ pattern: PureXML.Schema.Pattern, _ element: PureXML.Model.Element, env: [String: String]) -> PureXML.Schema.Pattern {
let extended = Self.extendedEnvironment(env, with: element)
let opened = startTagOpenDeriv(pattern, element.name)
let attributed = element.attributes
.filter { !Self.isNamespaceDeclaration($0) }
.reduce(opened) { attributeDeriv($0, $1) }
.reduce(opened) { attributeDeriv($0, $1, env: extended) }
let closed = startTagCloseDeriv(attributed)
let withChildren = childrenDeriv(closed, element.children)
let withChildren = childrenDeriv(closed, element.children, env: extended)
return endTagDeriv(withChildren)
}
@ -210,17 +240,17 @@ extension PureXML.Schema.RelaxNGEngine {
}
}
private func childrenDeriv(_ pattern: PureXML.Schema.Pattern, _ children: [PureXML.Model.Node]) -> PureXML.Schema.Pattern {
private func childrenDeriv(_ pattern: PureXML.Schema.Pattern, _ children: [PureXML.Model.Node], env: [String: String]) -> PureXML.Schema.Pattern {
let coalesced = Self.coalesceText(children)
// Clark's algorithm treats an empty element as optional empty text, so
// a data/value pattern whose lexical space admits "" matches <e/>.
if coalesced.isEmpty { return choice(pattern, textDeriv(pattern, "")) }
if coalesced.isEmpty { return choice(pattern, textDeriv(pattern, "", env: env)) }
if coalesced.count == 1, case let .text(string) = coalesced[0] {
let derived = textDeriv(pattern, string)
let derived = textDeriv(pattern, string, env: env)
return Self.isWhitespace(string) ? choice(pattern, derived) : derived
}
return coalesced.reduce(pattern) { accumulated, node in
Self.isWhitespaceText(node) ? accumulated : childDeriv(accumulated, node)
Self.isWhitespaceText(node) ? accumulated : childDeriv(accumulated, node, env: env)
}
}
@ -285,6 +315,8 @@ extension PureXML.Schema.RelaxNGEngine {
private struct RelaxNGStreamFrame {
var hadElementChild = false
var pendingText = ""
/// The namespace bindings in scope at this element, for QName values.
var env: [String: String] = [:]
}
public extension PureXML.Schema {
@ -299,7 +331,7 @@ public extension PureXML.Schema {
extension PureXML.Schema.RelaxNGEngine {
/// The initial streaming state for a start pattern, with a document-level frame.
func streamingStart(_ start: PureXML.Schema.Pattern) -> PureXML.Schema.RelaxNGStreamingState {
PureXML.Schema.RelaxNGStreamingState(current: start, frames: [RelaxNGStreamFrame()])
PureXML.Schema.RelaxNGStreamingState(current: start, frames: [RelaxNGStreamFrame(env: Self.rootEnvironment)])
}
/// Derives the state by one parse event, the streaming form of `childDeriv`.
@ -310,12 +342,16 @@ extension PureXML.Schema.RelaxNGEngine {
case let .startElement(name, attributes):
flushText(&state, closing: false)
if !state.frames.isEmpty { state.frames[state.frames.count - 1].hadElementChild = true }
var env = state.frames.last?.env ?? Self.rootEnvironment
for attribute in attributes where Self.isNamespaceDeclaration(attribute) {
env[attribute.name.prefix == "xmlns" ? attribute.name.localName : ""] = attribute.value
}
let opened = startTagOpenDeriv(state.current, name)
let attributed = attributes
.filter { !Self.isNamespaceDeclaration($0) }
.reduce(opened) { attributeDeriv($0, $1) }
.reduce(opened) { attributeDeriv($0, $1, env: env) }
state.current = startTagCloseDeriv(attributed)
state.frames.append(RelaxNGStreamFrame())
state.frames.append(RelaxNGStreamFrame(env: env))
case let .characters(text), let .cdata(text):
if !state.frames.isEmpty { state.frames[state.frames.count - 1].pendingText += text }
case .endElement:
@ -335,13 +371,21 @@ extension PureXML.Schema.RelaxNGEngine {
private func flushText(_ state: inout PureXML.Schema.RelaxNGStreamingState, closing: Bool) {
guard let index = state.frames.indices.last else { return }
let text = state.frames[index].pendingText
guard !text.isEmpty else { return }
let env = state.frames[index].env
guard !text.isEmpty else {
// An empty element is optional empty text (the tree walk's rule),
// so a data/value pattern admitting "" matches <e/>.
if closing, !state.frames[index].hadElementChild {
state.current = choice(state.current, textDeriv(state.current, "", env: env))
}
return
}
state.frames[index].pendingText = ""
if !Self.isWhitespace(text) {
state.current = textDeriv(state.current, text)
state.current = textDeriv(state.current, text, env: env)
} else if closing, !state.frames[index].hadElementChild {
// A lone whitespace text child is optional, the `childrenDeriv` rule.
state.current = choice(state.current, textDeriv(state.current, text))
state.current = choice(state.current, textDeriv(state.current, text, env: env))
}
// Whitespace among element siblings is ignorable and skipped.
}

View File

@ -216,6 +216,16 @@ final class RNGCompiler {
/// an unknown datatype, so the value matches nothing.
private func valuePattern(_ node: Tree) -> Pattern {
guard let type = valueType(node) else { return .notAllowed }
if type.base == .qName {
// The QName value space: resolve the literal's prefix against the
// schema's xmlns scope; an unprefixed literal takes the in-scope
// ns attribute (6.2.9).
let raw = RNGNode.text(node)
if let qualified = RNGNode.resolveQName(raw, at: node) {
return .valueQName(namespace: qualified.namespace, localName: qualified.localName)
}
return .valueQName(namespace: effectiveNS(node), localName: raw)
}
return .value(type, RNGNode.rawText(node))
}

View File

@ -40,6 +40,10 @@ public extension PureXML.Schema {
/// pattern must not (4.12).
case dataExcept(SimpleType, Pattern)
case value(SimpleType, String)
/// A `<value type="QName">`, compiled to its (namespace, local) value
/// pair via the schema's xmlns scope; the instance side resolves its
/// text against the instance element's in-scope namespaces.
case valueQName(namespace: String, localName: String)
case list(Pattern)
case ref(String)
case after(Pattern, Pattern)

View File

@ -0,0 +1,43 @@
@testable import PureXML
import Testing
/// The QName datatype's dual-context value space (#131): the schema side
/// resolves the literal against the schema's xmlns/ns scope at compile, and
/// the instance side resolves the text against the instance element's
/// in-scope namespaces at validation, in both tree and streaming modes.
@Suite("RELAX NG QName values")
struct RelaxNGQNameTests {
private let schema = """
<element name="e:foo" xmlns:e="http://www.example.com/1" xmlns="http://relaxng.org/ns/structure/1.0">
<value type="QName" ns="http://www.example.com/2" datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">xyzzy</value>
</element>
"""
@Test("Prefixed and default-namespace instance QNames resolve; unbound ones fail")
func test_dualContextResolution() throws {
let compiled = try PureXML.Schema.RelaxNG(schema)
let prefixed = "<e1:foo xmlns:e1=\"http://www.example.com/1\" xmlns:e2=\"http://www.example.com/2\">e2:xyzzy</e1:foo>"
#expect(try compiled.validate(prefixed))
let viaDefault = "<e1:foo xmlns:e1=\"http://www.example.com/1\" xmlns=\"http://www.example.com/2\">xyzzy</e1:foo>"
#expect(try compiled.validate(viaDefault))
let unbound = "<e1:foo xmlns:e1=\"http://www.example.com/1\">xyzzy</e1:foo>"
#expect(try !compiled.validate(unbound))
let wrongNamespace = "<e1:foo xmlns:e1=\"http://www.example.com/1\" xmlns=\"http://www.example.com/other\">xyzzy</e1:foo>"
#expect(try !compiled.validate(wrongNamespace))
// Streaming agrees with the tree walk.
#expect(try compiled.validate(streaming: prefixed))
#expect(try !compiled.validate(streaming: unbound))
}
@Test("A schema-side prefixed literal resolves against schema declarations")
func test_schemaSidePrefix() throws {
let prefixedLiteral = """
<element name="foo" xmlns:e3="http://www.example.com/2" xmlns="http://relaxng.org/ns/structure/1.0">
<value type="QName" datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">e3:xyzzy</value>
</element>
"""
let compiled = try PureXML.Schema.RelaxNG(prefixedLiteral)
#expect(try compiled.validate("<foo xmlns:q=\"http://www.example.com/2\">q:xyzzy</foo>"))
#expect(try !compiled.validate("<foo>xyzzy</foo>"))
}
}

View File

@ -18,20 +18,17 @@ struct RelaxNGSpecSuiteTests {
ProcessInfo.processInfo.environment["RNG_TS_ROOT"]
}
/// The burn-down frontier, by case index in document order (exact, so
/// progress and regressions both show). Two classes remain:
/// 1. Schema correctness is not validated: every `incorrect` schema
/// compiles (the count is asserted exactly below). Closing it means
/// validating schemas against the RELAX NG grammar and the section
/// 4.16-4.18 restrictions before pattern interpretation.
/// 2. The QName datatype (cases 378-380): its value space needs prefix
/// resolution in BOTH contexts, the schema's xmlns/ns scope at compile
/// and the instance element's in-scope namespaces at validation, which
/// the derivative engine does not thread yet.
/// The burn-down frontier (exact, so progress and regressions both show).
/// The instance level is fully clean: every valid instance is accepted
/// and every invalid one rejected. One class remains: schema correctness
/// is not validated, so every `incorrect` schema compiles (the count is
/// asserted exactly below); closing it means validating schemas against
/// the RELAX NG grammar and the section 4.16-4.18 restrictions before
/// pattern interpretation.
private let knownIncorrectCompiledCount = 213
private let knownCorrectRejected: Set<Int> = []
private let knownValidRejected: Set<Int> = [379, 380]
private let knownInvalidAccepted: Set<Int> = [378, 379]
private let knownValidRejected: Set<Int> = []
private let knownInvalidAccepted: Set<Int> = []
private struct SpecCase {
let index: Int