From 3415ff3d9a930b70c19cdc2be18c19167908d537 Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Wed, 10 Jun 2026 11:34:54 +0200 Subject: [PATCH] feat(relaxng): schema-correctness validation, spec suite complete A RELAX NG schema document is validated before pattern interpretation in three layers: the section-3 grammar check (vocabulary, attributes, arity, text, QNames, datatypeLibrary URIs, hrefs, 4.16 excepts, xmlns prohibitions), compiler checks (4.17 combine rules, unresolved refs, parentRef scope, startless grammars, include/externalRef cycles and overrides, grammar-checked loaded documents, unknown datatypes, built-in-library params), and the 4.19/7.1-7.4 restrictions over the compiled algebra after 4.20/4.21 normalization (prohibited paths, content types, duplicate attributes via representative-name overlap, infinite-class attributes outside oneOrMore, interleave constraints), with recursion detection deliberately before normalization. Incorrect schemas: 213 compiling down to 7, all the pre-Fifth-Edition name-class class (count asserted). Two leniency tests updated to spec behavior. Closes #131 --- CHANGELOG.md | 1 + README.md | 4 +- Sources/Schema/RelaxNGContentType.swift | 18 ++ Sources/Schema/RelaxNGGrammarCheck.swift | 191 ++++++++++++++++++ .../Schema/RelaxNGGrammarCheckSections.swift | 174 ++++++++++++++++ Sources/Schema/RelaxNGParser.swift | 141 ++++++------- Sources/Schema/RelaxNGParserIncludes.swift | 117 +++++++++++ .../Schema/RelaxNGRestrictionChecker.swift | 177 ++++++++++++++++ .../Schema/RelaxNGRestrictionOverlap.swift | 131 ++++++++++++ Sources/Schema/RelaxNGRestrictionPaths.swift | 142 +++++++++++++ Sources/Schema/RelaxNGRestrictions.swift | 22 ++ Sources/Schema/SchemaDocument.swift | 5 + Tests/RelaxNGCombineTests.swift | 11 +- Tests/RelaxNGDatatypeLibraryTests.swift | 19 +- Tests/RelaxNGSchemaCorrectnessTests.swift | 70 +++++++ Tests/RelaxNGSpecSuiteTests.swift | 20 +- 16 files changed, 1147 insertions(+), 96 deletions(-) create mode 100644 Sources/Schema/RelaxNGContentType.swift create mode 100644 Sources/Schema/RelaxNGGrammarCheck.swift create mode 100644 Sources/Schema/RelaxNGGrammarCheckSections.swift create mode 100644 Sources/Schema/RelaxNGParserIncludes.swift create mode 100644 Sources/Schema/RelaxNGRestrictionChecker.swift create mode 100644 Sources/Schema/RelaxNGRestrictionOverlap.swift create mode 100644 Sources/Schema/RelaxNGRestrictionPaths.swift create mode 100644 Sources/Schema/RelaxNGRestrictions.swift create mode 100644 Tests/RelaxNGSchemaCorrectnessTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 0dc121d..592e7d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- RELAX NG schema-correctness validation (#131): a schema document is now validated before any pattern interpretation, in three layers. The grammar check (section 3) rejects unknown or misplaced elements, illegal unqualified and RELAX NG-namespace attributes, wrong child arity, stray text, malformed or undeclared-prefix QNames, non-URI datatypeLibrary values (absolute, fragment-free, valid percent escapes), fragment-carrying hrefs, the 4.16 except restrictions (anyName/nsName inside excepts, propagated through choice), and the xmlns name prohibitions including the ns-attribute and except-mention forms. The compiler checks reject combine conflicts and double combine-less defines (4.17), unresolved refs and parentRef without a parent, startless grammars, include/externalRef self-reference, include overrides with nothing to override, non-grammar include targets (loaded documents are grammar-checked too), unknown datatypes, and params against the built-in library. The restrictions checker (4.19, 7.1-7.4) runs over the compiled algebra after 4.20/4.21 normalization (notAllowed/empty operands simplify away first; recursion detection deliberately precedes it): prohibited paths under attribute, oneOrMore-group, list, and in start; computable content types; duplicate attributes by representative-name overlap; infinite-name-class attributes outside oneOrMore; and the interleave text/element constraints. The spec suite's incorrect class drops from 213 compiling to 7, all pre-Fifth-Edition name-class cases (the xmltest-141 class, count asserted), with zero correct-schema or instance regressions. Two leniency tests updated to the spec behavior (double combine-less defines and unknown datatypes are schema errors). - RELAX NG QName values, instance level clean (#131): `` compiles to its (namespace, local-name) value pair via the schema's xmlns/ns scope, and the derivative engine threads the instance element's in-scope namespace bindings down to value comparison (in both the tree walk and the streaming state, whose frames now carry their bindings), so an instance QName resolves its prefix, or the default namespace when unprefixed, against the instance document itself; an unbound prefix matches nothing. The streaming path also gains the tree walk's empty-element rule. With this the spec suite's instance level is fully clean: valid 289/289 accepted and invalid 291/291 rejected; only the asserted schema-correctness class remains. - RELAX NG grammar scoping and include depth, third burn-down (#131): each `grammar` element opens its own define scope (names are scope-mangled in the flat table, so nested grammars do not collide) and `parentRef` reaches the enclosing scope; `include` is transitive (an included grammar's own includes merge first) and a define carried by the include element replaces the included grammar's same-name define (the 4.7 override); `div` wrappers flatten into their grammar while still contributing ns/datatypeLibrary inheritance (4.11); `value` text is kept untrimmed so whitespace-preserving datatypes compare exactly (6.2.9); and `dataExcept` recurses through choice patterns. The spec suite is now clean except one class: valid 286/289 and invalid 287/291, with all six remaining instance cases the QName datatype (its value space needs prefix resolution in both the schema and instance contexts), plus the asserted schema-correctness item. - RELAX NG namespaces and semantics, second burn-down (#131): the compiler now implements `ns` inheritance and the QName form of names (4.9-4.10: `ns` walks ancestors and crosses include/externalRef document boundaries; `name="eg:foo"` and `eg:foo` resolve their prefix against the schema's in-scope xmlns declarations; the `name=` shorthand on an attribute stays namespace-less rather than inheriting), an `include` brings in the included grammar's `start` with an include-level override honored (4.7), multiple `` elements combine like defines and a single define or start may omit `combine` while still merging by the named method (4.17), `` supports `` via a new `dataExcept` pattern in the derivative engine (4.12), an empty element matches a data/value pattern whose lexical space admits the empty string (Clark's empty-text rule), and a whitespace-only attribute value matches a pattern that matches empty. Spec-suite baselines tighten from valid 242/289 to 274/289 and invalid 278/291 to 279/291; the name-class and tree-helper sections split to their own files for the length caps. diff --git a/README.md b/README.md index f7c1511..4a19ea8 100644 --- a/README.md +++ b/README.md @@ -101,7 +101,7 @@ flowchart TB Epic122["#122 official spec suites"]:::epic XSTS["#129 W3C XSD test suite (XSTS)"]:::todo XSLTTS["#130 OASIS XSLT 1.0 suite (+XPath corpus)"]:::todo - RNGTS["#131 RELAX NG spec suite: runner + baselines in, burning down"]:::review + RNGTS["#131 RELAX NG spec suite: complete, one documented class"]:::done C14NTS["#132 C14N spec vectors: complete"]:::done Epic105 --> Epic121 Epic105 --> Epic122 @@ -125,7 +125,7 @@ flowchart TB PureXML is a working, dependency-free XML library today: parse, emit, validate, query, and stream documents on macOS, Linux, Windows, and WASM. The test suite -currently runs **935 tests in 161 suites** (`swift test`). +currently runs **940 tests in 162 suites** (`swift test`). ### Shipped (libxml2-aligned surface) diff --git a/Sources/Schema/RelaxNGContentType.swift b/Sources/Schema/RelaxNGContentType.swift new file mode 100644 index 0000000..c4d20bb --- /dev/null +++ b/Sources/Schema/RelaxNGContentType.swift @@ -0,0 +1,18 @@ +/// The 7.2 content classes, ordered for choice's max rule. +enum RelaxNGContentType: Comparable { + case empty + case complex + case simple + + static func < (lhs: Self, rhs: Self) -> Bool { + rank(lhs) < rank(rhs) + } + + private static func rank(_ type: Self) -> Int { + switch type { + case .empty: 0 + case .complex: 1 + case .simple: 2 + } + } +} diff --git a/Sources/Schema/RelaxNGGrammarCheck.swift b/Sources/Schema/RelaxNGGrammarCheck.swift new file mode 100644 index 0000000..ea65eff --- /dev/null +++ b/Sources/Schema/RelaxNGGrammarCheck.swift @@ -0,0 +1,191 @@ +/// Validates a RELAX NG schema document against the RELAX NG grammar itself +/// (spec section 3) and the name-class/except restrictions of section 4.16, +/// before any pattern interpretation: unknown or misplaced elements, missing +/// or illegal attributes, wrong child arity, stray text, and prohibited +/// except contents are all schema errors. Foreign-namespace elements and +/// qualified attributes are annotations and ignored throughout. +enum RelaxNGGrammarCheck { + /// The context an element is interpreted in. + enum Context: Equatable { + case pattern + case nameClass + case grammarContent + case includeContent + case dataExcept + case exceptInAnyName + case exceptInNsName + } + + /// The first violation in the schema, or nil when it is grammatical. + static func violation(_ top: RNGTree) -> String? { + check(top, in: .pattern) + } + + static func check(_ node: RNGTree, in context: Context) -> String? { + guard let name = RNGNode.localName(node) else { return nil } + if let problem = attributeProblem(node, name: name) { return problem } + if let problem = textProblem(node, name: name) { return problem } + switch context { + case .pattern: return checkPattern(node, name: name) + case .nameClass: return checkNameClass(node, name: name) + case .grammarContent: return checkGrammarComponent(node, name: name, allowIncludes: true) + case .includeContent: return checkGrammarComponent(node, name: name, allowIncludes: false) + case .dataExcept: return checkDataExceptMember(node, name: name) + case .exceptInAnyName, .exceptInNsName: return checkExceptMember(node, name: name, in: context) + } + } + + /// A member of a name-class except: the 4.16 prohibitions, propagated + /// through choice wrappers. + private static func checkExceptMember(_ node: RNGTree, name: String, in context: Context) -> String? { + if context == .exceptInAnyName, name == "anyName" { + return "anyName is not allowed inside an anyName except (4.16)" + } + if context == .exceptInNsName, name == "anyName" || name == "nsName" { + return "\(name) is not allowed inside an nsName except (4.16)" + } + if name == "choice" { + let members = RNGNode.elementChildren(node) + guard !members.isEmpty else { return " needs at least one name class" } + return firstViolation(members, in: context) + } + return checkNameClass(node, name: name) + } + + // MARK: Patterns + + private static func checkPattern(_ node: RNGTree, name: String) -> String? { + let children = RNGNode.elementChildren(node) + switch name { + case "element", "attribute": + return checkNamed(node, name: name, children: children) + case "group", "interleave", "choice", "optional", "zeroOrMore", "oneOrMore", "mixed", "list": + guard !children.isEmpty else { return "<\(name)> needs at least one pattern" } + return firstViolation(children, in: .pattern) + case "data": + return checkData(node, children: children) + case "grammar": + return firstViolation(children, in: .grammarContent) + default: + return checkLeafPattern(node, name: name, children: children) + } + } + + /// The childless leaves: refs, the empty family, value, externalRef. + private static func checkLeafPattern(_ node: RNGTree, name: String, children: [RNGTree]) -> String? { + switch name { + case "ref", "parentRef": + guard hasNCName(node, "name") else { return "<\(name)> needs an NCName name attribute" } + return children.isEmpty ? nil : "<\(name)> has no content" + case "empty", "text", "notAllowed": + return children.isEmpty ? nil : "<\(name)> has no content" + case "value": + return children.isEmpty ? nil : " holds text only" + case "externalRef": + guard RNGNode.attribute(node, "href") != nil else { return " needs an href attribute" } + return children.isEmpty ? nil : " has no content" + default: + return "<\(name)> is not a RELAX NG pattern" + } + } + + /// `element`/`attribute`: a name attribute or a leading name class, then + /// content (at least one pattern for element, at most one for attribute). + private static func checkNamed(_ node: RNGTree, name: String, children: [RNGTree]) -> String? { + var content = children[...] + if let qualified = RNGNode.attribute(node, "name") { + guard isQName(qualified) else { + return "<\(name)> has a malformed name attribute" + } + if qualified.contains(":"), RNGNode.resolveQName(qualified, at: node) == nil { + return "the prefix of '\(qualified)' is not declared" + } + } else { + guard let nameClass = content.first else { return "<\(name)> needs a name class" } + if let problem = check(nameClass, in: .nameClass) { return problem } + content = content.dropFirst() + } + if name == "element", content.isEmpty { + return " needs a content pattern" + } + if name == "attribute", content.count > 1 { + return " takes at most one pattern" + } + if name == "attribute", let problem = xmlnsAttributeProblem(node) { + return problem + } + return firstViolation(Array(content), in: .pattern) + } + + /// An attribute must not be named `xmlns` (with no namespace) nor live in + /// the xmlns namespace name (4.16). + private static func xmlnsAttributeProblem(_ node: RNGTree) -> String? { + if let name = RNGNode.attribute(node, "name") { + let namespaceName = node.attributes.first { $0.name.prefix == nil && $0.name.localName == "ns" }?.value ?? "" + return xmlnsMentionProblem(localName: name, namespaceName: namespaceName) + } + var stack = RNGNode.elementChildren(node).prefix(1).map(\.self) + while let nameClass = stack.popLast() { + let local = RNGNode.localName(nameClass) + if local == "name" || local == "nsName" { + let namespaceName = RNGNode.attribute(nameClass, "ns") ?? RNGNode.inheritedNS(nameClass) ?? "" + let mentioned = local == "name" ? RNGNode.text(nameClass) : nil + if let problem = xmlnsMentionProblem(localName: mentioned, namespaceName: namespaceName) { + return problem + } + } + if local != "name" { + stack.append(contentsOf: RNGNode.elementChildren(nameClass)) + } + } + return nil + } + + /// The prohibited mentions: the unqualified name `xmlns`, or any name in + /// the xmlns namespace name. + private static func xmlnsMentionProblem(localName: String?, namespaceName: String) -> String? { + let xmlnsURI = "http://www.w3.org/2000/xmlns" + if localName == "xmlns", namespaceName.isEmpty { + return "an attribute must not be named 'xmlns' (4.16)" + } + if namespaceName == xmlnsURI || namespaceName == xmlnsURI + "/" { + return "an attribute must not be in the xmlns namespace (4.16)" + } + return nil + } + + private static func checkData(_ node: RNGTree, children: [RNGTree]) -> String? { + guard hasNCName(node, "type") else { return " needs an NCName type attribute" } + var sawExcept = false + for child in children { + switch RNGNode.localName(child) { + case "param": + if sawExcept { return " may not follow in " } + guard hasNCName(child, "name") else { return " needs a name attribute" } + case "except": + if sawExcept { return " takes at most one " } + sawExcept = true + let members = RNGNode.elementChildren(child) + guard !members.isEmpty else { return " in needs content" } + if let problem = firstViolation(members, in: .dataExcept) { return problem } + default: + return "<\(RNGNode.localName(child) ?? "?")> is not allowed in " + } + } + return nil + } + + /// `except` in `data` admits only data, value, and choices of them (4.16). + private static func checkDataExceptMember(_ node: RNGTree, name: String) -> String? { + switch name { + case "data", "value": + return checkPattern(node, name: name) + case "choice": + let children = RNGNode.elementChildren(node) + guard !children.isEmpty else { return " needs at least one pattern" } + return firstViolation(children, in: .dataExcept) + default: + return "<\(name)> is not allowed inside a data except (4.16)" + } + } +} diff --git a/Sources/Schema/RelaxNGGrammarCheckSections.swift b/Sources/Schema/RelaxNGGrammarCheckSections.swift new file mode 100644 index 0000000..53ebf9a --- /dev/null +++ b/Sources/Schema/RelaxNGGrammarCheckSections.swift @@ -0,0 +1,174 @@ +/// The name-class, grammar-component, and shared checks of the RELAX NG +/// grammar validator, split from the dispatch for the length caps. +extension RelaxNGGrammarCheck { + // MARK: Name classes + + static func checkNameClass(_ node: RNGTree, name: String) -> String? { + let children = RNGNode.elementChildren(node) + switch name { + case "name": + return checkNameElement(node, children: children) + case "anyName", "nsName": + return checkInfiniteNameClass(node, name: name, children: children) + case "choice": + guard !children.isEmpty else { return " needs at least one name class" } + return firstViolation(children, in: .nameClass) + default: + return "<\(name)> is not a name class" + } + } + + private static func checkNameElement(_ node: RNGTree, children: [RNGTree]) -> String? { + guard children.isEmpty else { return " holds text only" } + let value = RNGNode.text(node) + guard isQName(value) else { return " must hold a QName" } + if value.contains(":"), RNGNode.resolveQName(value, at: node) == nil { + return "the prefix of '\(value)' is not declared" + } + return nil + } + + private static func checkInfiniteNameClass(_: RNGTree, name: String, children: [RNGTree]) -> String? { + for child in children { + guard RNGNode.localName(child) == "except" else { + return "<\(name)> admits only an child" + } + } + if children.count > 1 { return "<\(name)> takes at most one " } + guard let except = children.first else { return nil } + let members = RNGNode.elementChildren(except) + guard !members.isEmpty else { return " needs content" } + return firstViolation(members, in: name == "anyName" ? .exceptInAnyName : .exceptInNsName) + } + + // MARK: Grammar components + + static func checkGrammarComponent(_ node: RNGTree, name: String, allowIncludes: Bool) -> String? { + let children = RNGNode.elementChildren(node) + switch name { + case "start", "define": + return checkCombiningComponent(node, name: name, children: children) + case "div": + return firstViolation(children, in: allowIncludes ? .grammarContent : .includeContent) + case "include" where allowIncludes: + guard RNGNode.attribute(node, "href") != nil else { return " needs an href attribute" } + return firstViolation(children, in: .includeContent) + default: + return "<\(name)> is not allowed in this grammar context" + } + } + + /// `start` takes exactly one pattern; `define` needs a name and at least + /// one pattern; both may carry a combine method. + private static func checkCombiningComponent(_ node: RNGTree, name: String, children: [RNGTree]) -> String? { + if let problem = combineProblem(node) { return problem } + if name == "start" { + guard children.count == 1 else { return " takes exactly one pattern" } + return check(children[0], in: .pattern) + } + guard hasNCName(node, "name") else { return " needs an NCName name attribute" } + guard !children.isEmpty else { return " needs at least one pattern" } + return firstViolation(children, in: .pattern) + } + + private static func combineProblem(_ node: RNGTree) -> String? { + guard let combine = RNGNode.attribute(node, "combine"), combine != "choice", combine != "interleave" else { + return nil + } + return "combine must be 'choice' or 'interleave'" + } + + // MARK: Shared checks + + static func firstViolation(_ nodes: [RNGTree], in context: Context) -> String? { + for node in nodes { + if let problem = check(node, in: context) { return problem } + } + return nil + } + + /// Unqualified attributes must be the ones the vocabulary allows; + /// qualified (foreign or xml:*) attributes are annotations. The + /// datatypeLibrary value must be an absolute, fragment-free URI, and an + /// href must carry no fragment. + static func attributeProblem(_ node: RNGTree, name: String) -> String? { + let allowed = allowedAttributes[name] ?? [] + for attribute in node.attributes { + guard attribute.name.prefix == nil else { + if attribute.name.namespaceURI == RNGNode.relaxNGNamespace { + return "RELAX NG-namespace attribute '\(attribute.name.description)' is not allowed" + } + continue + } + let local = attribute.name.localName + if local == "datatypeLibrary" { + guard isDatatypeLibraryURI(attribute.value) else { + return "datatypeLibrary '\(attribute.value)' is not an absolute, fragment-free URI" + } + continue + } + if local == "href", attribute.value.contains("#") { + return "href must not carry a fragment identifier" + } + guard local != "ns", local != "xmlns", !allowed.contains(local) else { continue } + return "attribute '\(local)' is not allowed on <\(name)>" + } + return nil + } + + /// RFC 3986 absolute URI without fragment: `scheme ":" ...` with a scheme + /// of ALPHA *(ALPHA / DIGIT / "+" / "-" / "."), or the empty string for + /// the built-in library. + static func isDatatypeLibraryURI(_ value: String) -> Bool { + if value.isEmpty { return true } + guard !value.contains("#") else { return false } + // A percent must introduce a two-digit hex escape (RFC 2396). + var cursor = value.startIndex + while let percent = value[cursor...].firstIndex(of: "%") { + guard let first = value.index(percent, offsetBy: 1, limitedBy: value.endIndex), + let second = value.index(percent, offsetBy: 2, limitedBy: value.endIndex), + second < value.endIndex, + value[first].isHexDigit, value[second].isHexDigit + else { return false } + cursor = value.index(after: percent) + } + guard let colon = value.firstIndex(of: ":"), colon != value.startIndex, + value.index(after: colon) < value.endIndex + else { return false } + let scheme = value[value.startIndex ..< colon] + guard let first = scheme.first, first.isASCII, first.isLetter else { return false } + return scheme.dropFirst().allSatisfy { character in + character.isASCII && (character.isLetter || character.isNumber || character == "+" || character == "-" || character == ".") + } + } + + static let allowedAttributes: [String: Set] = [ + "element": ["name"], "attribute": ["name"], + "ref": ["name"], "parentRef": ["name"], + "define": ["name", "combine"], "start": ["combine"], + "data": ["type"], "value": ["type"], "param": ["name"], + "externalRef": ["href"], "include": ["href"], + ] + + /// Non-whitespace text is content only inside value, name, and param. + static func textProblem(_ node: RNGTree, name: String) -> String? { + guard !["value", "name", "param"].contains(name) else { return nil } + for child in node.children where child.kind == .text { + if child.stringValue.contains(where: { !$0.isWhitespace }) { + return "text is not allowed inside <\(name)>" + } + } + return nil + } + + static func hasNCName(_ node: RNGTree, _ attribute: String) -> Bool { + guard let value = RNGNode.attribute(node, attribute), !value.isEmpty else { return false } + return !value.contains(":") && PureXML.Parsing.XMLCharacter.isValidName(value) + } + + static func isQName(_ value: String) -> Bool { + let parts = value.split(separator: ":", omittingEmptySubsequences: false) + guard parts.count <= 2, !value.isEmpty else { return false } + return parts.allSatisfy { !$0.isEmpty && PureXML.Parsing.XMLCharacter.isValidName(String($0)) } + } +} diff --git a/Sources/Schema/RelaxNGParser.swift b/Sources/Schema/RelaxNGParser.swift index 5dfd2f9..c76deae 100644 --- a/Sources/Schema/RelaxNGParser.swift +++ b/Sources/Schema/RelaxNGParser.swift @@ -13,18 +13,24 @@ final class RNGCompiler { /// sharing a name agree on one method, and the single define that omits /// `combine` still merges by it. private var defineCombine: [String: String] = [:] - private let loader: (String) -> String? - private var visited: Set = [] + /// How many defines of each name omitted `combine` (at most one may). + private var combinelessDefines: [String: Int] = [:] + /// Every ref/parentRef target, checked against the defines at the end. + private var referenced: Set = [] + /// Schema-correctness problems discovered during compilation. + var violations: [String] = [] + let loader: (String) -> String? + var visited: Set = [] /// The URI of the document currently being compiled (nil for the top /// level): hrefs inside a loaded document resolve against it. - private var documentBase: String? + var documentBase: String? /// The `ns` value carried across a document boundary: an externalRef's /// in-scope ns applies to the loaded pattern when it has none of its own /// (simplification 4.6); the tree-ancestor walk cannot cross documents. - private var fallbackNamespace = "" + var fallbackNamespace = "" /// The start pattern contributed by an `include`d grammar (4.7), used when /// the including grammar declares no start of its own. - private var includedStart: Pattern? + var includedStart: Pattern? /// Each `grammar` element opens its own define scope; names are mangled /// with the scope identifier so nested grammars do not collide and /// `parentRef` reaches the enclosing grammar's scope. @@ -35,6 +41,18 @@ final class RNGCompiler { "\(scopeStack.last ?? 0)\u{1}\(name)" } + /// Builds a ref, recording the target for the resolution check; a + /// parentRef outside any nested grammar is a schema error. + private func reference(_ name: String, parent: Bool) -> Pattern { + if parent, scopeStack.count < 2 { + violations.append("parentRef '\(name)' has no enclosing grammar") + return .notAllowed + } + let target = parent ? parentMangled(name) : mangled(name) + referenced.insert(target) + return .ref(target) + } + private func parentMangled(_ name: String) -> String { let parent = scopeStack.count >= 2 ? scopeStack[scopeStack.count - 2] : (scopeStack.last ?? 0) return "\(parent)\u{1}\(name)" @@ -50,6 +68,14 @@ final class RNGCompiler { self.loader = loader } + /// Refs whose targets no define supplies, as schema errors. + func unresolvedReferences() -> [String] { + referenced.subtracting(defines.keys).sorted().map { target in + let name = target.split(separator: "\u{1}").last.map(String.init) ?? target + return "ref '\(name)' has no matching define" + } + } + func topLevel(_ node: Tree) -> Pattern { RNGNode.localName(node) == "grammar" ? grammar(node) : pattern(node) } @@ -71,6 +97,9 @@ final class RNGCompiler { } let starts = RNGNode.components(node, named: "start") guard let first = starts.first else { + if includedStart == nil { + violations.append("grammar has no start") + } return includedStart ?? .notAllowed } // Multiple starts combine like defines (4.17): the named method wins, @@ -94,12 +123,20 @@ final class RNGCompiler { /// Adds a `define`, honoring `combine`: a second definition of the same name /// with `combine="choice"`/`"interleave"` merges with the existing one rather /// than replacing it (a plain redefinition still replaces). - private func addDefine(_ define: Tree) { + func addDefine(_ define: Tree) { guard let rawName = RNGNode.attribute(define, "name") else { return } let name = mangled(rawName) let pattern = combined(RNGNode.elementChildren(define), .sequence) if let method = RNGNode.attribute(define, "combine") { + if let established = defineCombine[name], established != method { + violations.append("define '\(rawName)' uses conflicting combine methods (4.17)") + } defineCombine[name] = method + } else { + combinelessDefines[name, default: 0] += 1 + if combinelessDefines[name] == 2 { + violations.append("more than one define of '\(rawName)' omits combine (4.17)") + } } guard let existing = defines[name] else { defines[name] = pattern @@ -112,47 +149,6 @@ final class RNGCompiler { } } - /// Merges an `include`d grammar: its `define`s first, then the `include`'s own - /// nested `define`s, which override or (with `combine`) merge with them. - private func mergeInclude(_ node: Tree) { - guard let href = RNGNode.resolvedHref(node, documentBase: documentBase), !visited.contains(href), - let text = loader(href), let root = try? PureXML.parseTree(text), - let grammar = RNGNode.elementChildren(root).first - else { - return - } - visited.insert(href) - let outerBase = documentBase - let outerNamespace = fallbackNamespace - documentBase = href - if RNGNode.inheritedNS(grammar) == nil { - fallbackNamespace = effectiveNS(node) - } - // The included grammar's own includes merge first (4.7 is transitive). - for nested in RNGNode.components(grammar, named: "include") { - mergeInclude(nested) - } - // A define carried by the include element REPLACES the included - // grammar's define of the same name (4.7 override), so those names - // are skipped here and added from the include's children below. - let overrides = Set(RNGNode.children(node, named: "define").compactMap { RNGNode.attribute($0, "name") }) - for define in RNGNode.components(grammar, named: "define") { - guard let name = RNGNode.attribute(define, "name"), !overrides.contains(name) else { continue } - addDefine(define) - } - if includedStart == nil, let start = RNGNode.components(grammar, named: "start").first { - includedStart = combined(RNGNode.elementChildren(start), .sequence) - } - documentBase = outerBase - fallbackNamespace = outerNamespace - if let override = RNGNode.children(node, named: "start").first { - includedStart = combined(RNGNode.elementChildren(override), .sequence) - } - for define in RNGNode.children(node, named: "define") { - addDefine(define) - } - } - // MARK: Patterns func pattern(_ node: Tree) -> Pattern { @@ -176,8 +172,8 @@ final class RNGCompiler { /// nested grammars, and external references. private func referencePattern(_ node: Tree) -> Pattern? { switch RNGNode.localName(node) { - case "ref": .ref(mangled(RNGNode.attribute(node, "name") ?? "")) - case "parentRef": .ref(parentMangled(RNGNode.attribute(node, "name") ?? "")) + case "ref": reference(RNGNode.attribute(node, "name") ?? "", parent: false) + case "parentRef": reference(RNGNode.attribute(node, "name") ?? "", parent: true) case "grammar": grammar(node) case "externalRef": externalRef(node) default: nil @@ -194,13 +190,19 @@ final class RNGCompiler { /// does not define is an unknown datatype, so the pattern matches nothing. private func dataPattern(_ node: Tree) -> Pattern { let typeName = RNGNode.strip(RNGNode.attribute(node, "type") ?? "string") - guard let base = resolvedBuiltin(typeName, library: datatypeLibrary(of: node)) else { return .notAllowed } + guard let base = resolvedBuiltin(typeName, library: datatypeLibrary(of: node)) else { + violations.append("datatype '\(typeName)' is not defined by the datatype library in scope") + return .notAllowed + } var facets = PureXML.Schema.Facets() for param in RNGNode.children(node, named: "param") { if let name = RNGNode.attribute(param, "name") { PureXML.Schema.RelaxNGFacets.apply(name, RNGNode.text(param), into: &facets) } } + if datatypeLibrary(of: node) == "", !RNGNode.children(node, named: "param").isEmpty { + violations.append("the built-in datatype library defines no params for '\(typeName)'") + } let type = PureXML.Schema.SimpleType(base: base, facets: facets) // 4.12: an except child excludes its (implicitly choice-combined) // patterns from the data type's lexical space. @@ -215,7 +217,10 @@ final class RNGCompiler { /// type; an explicit `type` the in-scope `datatypeLibrary` does not define is /// an unknown datatype, so the value matches nothing. private func valuePattern(_ node: Tree) -> Pattern { - guard let type = valueType(node) else { return .notAllowed } + guard let type = valueType(node) else { + violations.append("datatype '\(RNGNode.attribute(node, "type") ?? "")' is not defined by the datatype library in scope") + return .notAllowed + } if type.base == .qName { // The QName value space: resolve the literal's prefix against the // schema's xmlns scope; an unprefixed literal takes the in-scope @@ -263,29 +268,6 @@ final class RNGCompiler { return "" } - /// Resolves an `externalRef` to the referenced schema's pattern, merging any - /// grammar `define`s it carries into this compiler's table. - private func externalRef(_ node: Tree) -> Pattern { - guard let href = RNGNode.resolvedHref(node, documentBase: documentBase), !visited.contains(href), - let text = loader(href), let root = try? PureXML.parseTree(text), - let top = RNGNode.elementChildren(root).first - else { - return .notAllowed - } - visited.insert(href) - let outerBase = documentBase - let outerNamespace = fallbackNamespace - documentBase = href - if RNGNode.inheritedNS(top) == nil { - fallbackNamespace = effectiveNS(node) - } - defer { - documentBase = outerBase - fallbackNamespace = outerNamespace - } - return topLevel(top) - } - private func compositePattern(_ node: Tree) -> Pattern { switch RNGNode.localName(node) { case "group": combined(RNGNode.elementChildren(node), .sequence) @@ -332,8 +314,19 @@ extension PureXML.Schema { guard let top = RNGNode.elementChildren(root).first else { throw PureXML.Schema.SchemaError.notASchema } + // The schema document must itself match the RELAX NG grammar. + if let problem = RelaxNGGrammarCheck.violation(top) { + throw PureXML.Schema.SchemaError.invalidRelaxNG(reason: problem) + } let compiler = RNGCompiler(loader: loader) let start = compiler.topLevel(top) + let unresolved = compiler.unresolvedReferences() + if let problem = compiler.violations.first ?? unresolved.first { + throw PureXML.Schema.SchemaError.invalidRelaxNG(reason: problem) + } + if let problem = RelaxNGRestrictions.violation(start: start, defines: compiler.defines) { + throw PureXML.Schema.SchemaError.invalidRelaxNG(reason: problem) + } return (start, compiler.defines) } } diff --git a/Sources/Schema/RelaxNGParserIncludes.swift b/Sources/Schema/RelaxNGParserIncludes.swift new file mode 100644 index 0000000..5bd92f6 --- /dev/null +++ b/Sources/Schema/RelaxNGParserIncludes.swift @@ -0,0 +1,117 @@ +/// The include and externalRef resolution of the RELAX NG compiler, split +/// from the compiler body for the length caps: transitive merging with 4.7 +/// overrides, per-document bases, cross-document ns, and cycle detection. +extension RNGCompiler { + /// Merges an `include`d grammar: its `define`s first, then the `include`'s own + /// nested `define`s, which override or (with `combine`) merge with them. + func mergeInclude(_ node: Tree) { + guard let href = RNGNode.resolvedHref(node, documentBase: documentBase) else { return } + guard !visited.contains(href) else { + violations.append("include '\(href)' includes itself") + return + } + guard let grammar = loadedGrammar(href) else { return } + visited.insert(href) + defer { visited.remove(href) } + mergeLoadedGrammar(grammar, from: node, href: href) + if let override = RNGNode.children(node, named: "start").first { + includedStart = combined(RNGNode.elementChildren(override), .sequence) + } + for define in RNGNode.children(node, named: "define") { + addDefine(define) + } + } + + /// Merges the loaded grammar's components under its own base and + /// namespace scope: transitive includes first, then its defines (minus + /// the include's overrides) and its start. + private func mergeLoadedGrammar(_ grammar: Tree, from node: Tree, href: String) { + let outerBase = documentBase + let outerNamespace = fallbackNamespace + documentBase = href + if RNGNode.inheritedNS(grammar) == nil { + fallbackNamespace = effectiveNS(node) + } + defer { + documentBase = outerBase + fallbackNamespace = outerNamespace + } + // The included grammar's own includes merge first (4.7 is transitive). + for nested in RNGNode.components(grammar, named: "include") { + mergeInclude(nested) + } + let overrides = includeOverrides(node, grammar: grammar) + for define in RNGNode.components(grammar, named: "define") { + guard let name = RNGNode.attribute(define, "name"), !overrides.contains(name) else { continue } + addDefine(define) + } + if includedStart == nil, let start = RNGNode.components(grammar, named: "start").first { + includedStart = combined(RNGNode.elementChildren(start), .sequence) + } + } + + /// A define carried by the include element REPLACES the included + /// grammar's define of the same name (4.7 override); an override with + /// nothing to override is a schema error. + private func includeOverrides(_ node: Tree, grammar: Tree) -> Set { + let overrides = Set(RNGNode.children(node, named: "define").compactMap { RNGNode.attribute($0, "name") }) + let provided = Set(RNGNode.components(grammar, named: "define").compactMap { RNGNode.attribute($0, "name") }) + for name in overrides.sorted() where !provided.contains(name) { + violations.append("include overrides define '\(name)', which the included grammar does not define") + } + let overridesStart = RNGNode.children(node, named: "start").first != nil + if overridesStart, RNGNode.components(grammar, named: "start").first == nil { + violations.append("include overrides the start, which the included grammar does not define") + } + return overrides + } + + /// Loads and validates an include's target, which must be a grammatical + /// RELAX NG grammar element. + private func loadedGrammar(_ href: String) -> Tree? { + guard let text = loader(href), let root = try? PureXML.parseTree(text), + let grammar = RNGNode.elementChildren(root).first + else { + return nil + } + if RNGNode.localName(grammar) != "grammar" { + violations.append("include '\(href)' must reference a grammar") + return nil + } + if let problem = RelaxNGGrammarCheck.violation(grammar) { + violations.append("included schema '\(href)': \(problem)") + return nil + } + return grammar + } + + func externalRef(_ node: Tree) -> Pattern { + guard let href = RNGNode.resolvedHref(node, documentBase: documentBase) else { return .notAllowed } + guard !visited.contains(href) else { + violations.append("externalRef '\(href)' references itself") + return .notAllowed + } + guard let text = loader(href), let root = try? PureXML.parseTree(text), + let top = RNGNode.elementChildren(root).first + else { + return .notAllowed + } + if let problem = RelaxNGGrammarCheck.violation(top) { + violations.append("referenced schema '\(href)': \(problem)") + return .notAllowed + } + visited.insert(href) + defer { visited.remove(href) } + let outerBase = documentBase + let outerNamespace = fallbackNamespace + documentBase = href + if RNGNode.inheritedNS(top) == nil { + fallbackNamespace = effectiveNS(node) + } + defer { + documentBase = outerBase + fallbackNamespace = outerNamespace + } + return topLevel(top) + } +} diff --git a/Sources/Schema/RelaxNGRestrictionChecker.swift b/Sources/Schema/RelaxNGRestrictionChecker.swift new file mode 100644 index 0000000..f0be616 --- /dev/null +++ b/Sources/Schema/RelaxNGRestrictionChecker.swift @@ -0,0 +1,177 @@ +/// The working state of one restrictions pass. Internal to +/// ``RelaxNGRestrictions``; split across files for the length caps. +final class RelaxNGRestrictionChecker { + typealias Pattern = PureXML.Schema.Pattern + typealias NameClass = PureXML.Schema.NameClass + + let defines: [String: Pattern] + private var checkedElements = 0 + + init(defines: [String: Pattern]) { + self.defines = defines + } + + func resolve(_ name: String) -> Pattern { + defines[name] ?? .notAllowed + } + + /// The 4.20/4.21 normalization: notAllowed and empty operands + /// simplify away before the restrictions are judged, so a branch the + /// schema can never take is not checked. + static func normalized(_ pattern: Pattern) -> Pattern { + switch pattern { + case let .choice(lhs, rhs): + let left = normalized(lhs) + let right = normalized(rhs) + if case .notAllowed = left { return right } + if case .notAllowed = right { return left } + return .choice(left, right) + case let .group(lhs, rhs): + return combineNormalized(normalized(lhs), normalized(rhs), make: { .group($0, $1) }) + case let .interleave(lhs, rhs): + return combineNormalized(normalized(lhs), normalized(rhs), make: { .interleave($0, $1) }) + default: + return normalizedUnary(pattern) + } + } + + /// The unary cases of 4.20: a wrapper around notAllowed is notAllowed, + /// while elements keep their (normalized) content. + private static func normalizedUnary(_ pattern: Pattern) -> Pattern { + switch pattern { + case let .oneOrMore(inner): + let normalizedInner = normalized(inner) + if case .notAllowed = normalizedInner { return .notAllowed } + return .oneOrMore(normalizedInner) + case let .list(inner): + let normalizedInner = normalized(inner) + if case .notAllowed = normalizedInner { return .notAllowed } + return .list(normalizedInner) + case let .attribute(nameClass, content): + let normalizedContent = normalized(content) + if case .notAllowed = normalizedContent { return .notAllowed } + return .attribute(nameClass, normalizedContent) + case let .element(nameClass, content): + return .element(nameClass, normalized(content)) + default: + return pattern + } + } + + private static func combineNormalized(_ lhs: Pattern, _ rhs: Pattern, make: (Pattern, Pattern) -> Pattern) -> Pattern { + if case .notAllowed = lhs { return .notAllowed } + if case .notAllowed = rhs { return .notAllowed } + if case .empty = lhs { return rhs } + if case .empty = rhs { return lhs } + return make(lhs, rhs) + } + + // MARK: 7.1.5 start + + /// `start` admits only element, ref, choice, and notAllowed. + func startProblem(_ pattern: Pattern, visiting: Set = []) -> String? { + switch pattern { + case .element, .notAllowed: + return nil + case let .choice(lhs, rhs): + return startProblem(lhs, visiting: visiting) ?? startProblem(rhs, visiting: visiting) + case let .ref(name): + guard !visiting.contains(name) else { return nil } + return startProblem(resolve(name), visiting: visiting.union([name])) + default: + return "start admits only element, ref, choice, and notAllowed (7.1.5)" + } + } + + // MARK: 4.19 recursion + + /// No define reachable from the start may reach itself through refs + /// without an intervening element (unreachable defines are removed by + /// the 4.19 simplification, so they are exempt). + func recursionProblem(start: Pattern) -> String? { + var reachable: Set = [] + collectReachable(start, into: &reachable) + for name in reachable.sorted() where reaches(resolve(name), target: name, visiting: [name]) { + let plain = name.split(separator: "\u{1}").last.map(String.init) ?? name + return "define '\(plain)' recurses without an intervening element (4.19)" + } + return nil + } + + private func collectReachable(_ pattern: Pattern, into reachable: inout Set) { + switch pattern { + case let .ref(name): + guard reachable.insert(name).inserted else { return } + collectReachable(resolve(name), into: &reachable) + case let .choice(lhs, rhs), let .group(lhs, rhs), let .interleave(lhs, rhs): + collectReachable(lhs, into: &reachable) + collectReachable(rhs, into: &reachable) + case let .oneOrMore(inner), let .list(inner), let .attribute(_, inner), + let .element(_, inner), let .dataExcept(_, inner): + collectReachable(inner, into: &reachable) + default: + break + } + } + + private func reaches(_ pattern: Pattern, target: String, visiting: Set) -> Bool { + switch pattern { + case let .ref(name): + if name == target { return true } + guard !visiting.contains(name) else { return false } + return reaches(resolve(name), target: target, visiting: visiting.union([name])) + case let .choice(lhs, rhs), let .group(lhs, rhs), let .interleave(lhs, rhs): + return reaches(lhs, target: target, visiting: visiting) || reaches(rhs, target: target, visiting: visiting) + case let .oneOrMore(inner), let .list(inner), let .attribute(_, inner): + return reaches(inner, target: target, visiting: visiting) + case .element: + return false // An element guards the recursion. + default: + return false + } + } + + // MARK: 7.2 content types + + func contentType(_ pattern: Pattern, visiting: Set) -> RelaxNGContentType? { + switch pattern { + case .empty, .notAllowed, .attribute: + return .empty + case .text, .element: + return .complex + case .data, .value, .valueQName, .list, .dataExcept: + return .simple + case .choice, .group, .interleave, .oneOrMore: + return combinedContentType(pattern, visiting: visiting) + case let .ref(name): + guard !visiting.contains(name) else { return .complex } + return contentType(resolve(name), visiting: visiting.union([name])) + default: + return .empty + } + } + + /// Combinator content types: choice takes the max; group, interleave, and + /// oneOrMore additionally require groupable operands (7.2). + private func combinedContentType(_ pattern: Pattern, visiting: Set) -> RelaxNGContentType? { + switch pattern { + case let .choice(lhs, rhs): + guard let left = contentType(lhs, visiting: visiting), let right = contentType(rhs, visiting: visiting) else { return nil } + return max(left, right) + case let .group(lhs, rhs), let .interleave(lhs, rhs): + guard let left = contentType(lhs, visiting: visiting), let right = contentType(rhs, visiting: visiting), + groupable(left, right) + else { return nil } + return max(left, right) + case let .oneOrMore(inner): + guard let type = contentType(inner, visiting: visiting), groupable(type, type) else { return nil } + return type + default: + return nil + } + } + + private func groupable(_ lhs: RelaxNGContentType, _ rhs: RelaxNGContentType) -> Bool { + lhs == .empty || rhs == .empty || (lhs == .complex && rhs == .complex) + } +} diff --git a/Sources/Schema/RelaxNGRestrictionOverlap.swift b/Sources/Schema/RelaxNGRestrictionOverlap.swift new file mode 100644 index 0000000..b4ea39a --- /dev/null +++ b/Sources/Schema/RelaxNGRestrictionOverlap.swift @@ -0,0 +1,131 @@ +/// The attribute and interleave restrictions (7.3, 7.4) with the +/// representative-name overlap test for name classes. +extension RelaxNGRestrictionChecker { + // MARK: 7.3 duplicate attributes + + func attributeSets(_ pattern: Pattern, visiting: Set) -> (classes: [NameClass], problem: String?) { + switch pattern { + case let .attribute(nameClass, _): + ([nameClass], nil) + case let .choice(lhs, rhs): + combinedAttributeSets(lhs, rhs, crossCheck: false, visiting: visiting) + case let .group(lhs, rhs), let .interleave(lhs, rhs): + combinedAttributeSets(lhs, rhs, crossCheck: true, visiting: visiting) + case let .oneOrMore(inner), let .list(inner): + attributeSets(inner, visiting: visiting) + case let .ref(name): + visiting.contains(name) + ? ([], nil) + : attributeSets(resolve(name), visiting: visiting.union([name])) + default: + ([], nil) + } + } + + /// Merges two operands' attribute classes; group and interleave require + /// the operands' classes to be pairwise disjoint (7.3). + private func combinedAttributeSets(_ lhs: Pattern, _ rhs: Pattern, crossCheck: Bool, visiting: Set) -> (classes: [NameClass], problem: String?) { + let left = attributeSets(lhs, visiting: visiting) + if left.problem != nil { return left } + let right = attributeSets(rhs, visiting: visiting) + if right.problem != nil { return right } + if crossCheck { + for first in left.classes { + for second in right.classes where nameClassesOverlap(first, second) { + return ([], "the same attribute may occur twice in one element (7.3)") + } + } + } + return (left.classes + right.classes, nil) + } + + // MARK: 7.4 interleave + + func interleaveProblem(_ pattern: Pattern, visiting: Set) -> String? { + switch pattern { + case let .interleave(lhs, rhs): + if hasText(lhs, visiting: visiting), hasText(rhs, visiting: visiting) { + return "text may appear in only one operand of an interleave (7.4)" + } + for first in elementClasses(lhs, visiting: visiting) { + for second in elementClasses(rhs, visiting: visiting) where nameClassesOverlap(first, second) { + return "interleave operands share an element name (7.4)" + } + } + return interleaveProblem(lhs, visiting: visiting) ?? interleaveProblem(rhs, visiting: visiting) + case let .choice(lhs, rhs), let .group(lhs, rhs): + return interleaveProblem(lhs, visiting: visiting) ?? interleaveProblem(rhs, visiting: visiting) + case let .oneOrMore(inner), let .list(inner), let .attribute(_, inner), let .dataExcept(_, inner): + return interleaveProblem(inner, visiting: visiting) + case let .ref(name): + guard !visiting.contains(name) else { return nil } + return interleaveProblem(resolve(name), visiting: visiting.union([name])) + default: + return nil + } + } + + private func hasText(_ pattern: Pattern, visiting: Set) -> Bool { + switch pattern { + case .text: + return true + case let .choice(lhs, rhs), let .group(lhs, rhs), let .interleave(lhs, rhs): + return hasText(lhs, visiting: visiting) || hasText(rhs, visiting: visiting) + case let .oneOrMore(inner): + return hasText(inner, visiting: visiting) + case let .ref(name): + guard !visiting.contains(name) else { return false } + return hasText(resolve(name), visiting: visiting.union([name])) + default: + return false + } + } + + private func elementClasses(_ pattern: Pattern, visiting: Set) -> [NameClass] { + switch pattern { + case let .element(nameClass, _): + return [nameClass] + case let .choice(lhs, rhs), let .group(lhs, rhs), let .interleave(lhs, rhs): + return elementClasses(lhs, visiting: visiting) + elementClasses(rhs, visiting: visiting) + case let .oneOrMore(inner): + return elementClasses(inner, visiting: visiting) + case let .ref(name): + guard !visiting.contains(name) else { return [] } + return elementClasses(resolve(name), visiting: visiting.union([name])) + default: + return [] + } + } + + // MARK: Name-class overlap + + /// Overlap by representative names: every concrete name either class + /// mentions, plus an illustrative name per nsName namespace and one in + /// a namespace neither could mention, tested against both classes. + private func namesClassRepresentatives(_ nameClass: NameClass, into set: inout [PureXML.Model.QualifiedName]) { + switch nameClass { + case let .name(namespace, localName): + set.append(PureXML.Model.QualifiedName(prefix: nil, localName: localName, namespaceURI: namespace.isEmpty ? nil : namespace)) + case let .nsName(namespace): + set.append(PureXML.Model.QualifiedName(prefix: nil, localName: "\u{1}illustrative", namespaceURI: namespace.isEmpty ? nil : namespace)) + case let .nsNameExcept(namespace, except): + set.append(PureXML.Model.QualifiedName(prefix: nil, localName: "\u{1}illustrative", namespaceURI: namespace.isEmpty ? nil : namespace)) + namesClassRepresentatives(except, into: &set) + case .anyName: + set.append(PureXML.Model.QualifiedName(prefix: nil, localName: "\u{1}illustrative", namespaceURI: "\u{1}nowhere")) + case let .anyNameExcept(except): + set.append(PureXML.Model.QualifiedName(prefix: nil, localName: "\u{1}illustrative", namespaceURI: "\u{1}nowhere")) + namesClassRepresentatives(except, into: &set) + case let .choice(lhs, rhs): + namesClassRepresentatives(lhs, into: &set) + namesClassRepresentatives(rhs, into: &set) + } + } + + func nameClassesOverlap(_ first: NameClass, _ second: NameClass) -> Bool { + var candidates: [PureXML.Model.QualifiedName] = [] + namesClassRepresentatives(first, into: &candidates) + namesClassRepresentatives(second, into: &candidates) + return candidates.contains { first.contains($0) && second.contains($0) } + } +} diff --git a/Sources/Schema/RelaxNGRestrictionPaths.swift b/Sources/Schema/RelaxNGRestrictionPaths.swift new file mode 100644 index 0000000..3daa762 --- /dev/null +++ b/Sources/Schema/RelaxNGRestrictionPaths.swift @@ -0,0 +1,142 @@ +/// The prohibition flags one path walk carries (7.1). +struct RelaxNGPathFlags { + var inAttribute = false + var inOneOrMoreGroupOrInterleave = false + var underOneOrMore = false + var inList = false +} + +/// The per-element walks: prohibited paths (7.1), with refs expanding +/// transparently and elements as the simplified grammar's ref targets. +extension RelaxNGRestrictionChecker { + // MARK: Per-element checks (7.1-7.4, 4.16) + + /// Walks every element pattern reachable from `start`, checking its + /// content against the path, content-type, attribute, and interleave + /// restrictions. + func elementProblems(_ start: Pattern) -> String? { + var queue: [Pattern] = [start] + var enqueuedDefines: Set = [] + var seenElements = 0 + while let pattern = queue.popLast() { + switch pattern { + case let .element(nameClass, content): + seenElements += 1 + if seenElements > 10000 { return nil } + _ = nameClass + if let problem = contentProblem(content) { return problem } + queue.append(content) + case let .attribute(_, content): + queue.append(content) + case let .choice(lhs, rhs), let .group(lhs, rhs), let .interleave(lhs, rhs): + queue.append(lhs) + queue.append(rhs) + case let .oneOrMore(inner), let .list(inner), let .dataExcept(_, inner): + queue.append(inner) + case let .ref(name): + if enqueuedDefines.insert(name).inserted { + queue.append(resolve(name)) + } + default: + break + } + } + return nil + } + + /// One element's content: prohibited paths under attribute, oneOrMore + /// group/interleave, and list (7.1), a computable content type (7.2), + /// distinct attributes (7.3), and interleave constraints (7.4). + func contentProblem(_ content: Pattern) -> String? { + if let problem = pathProblem(content, flags: RelaxNGPathFlags(), visiting: []) { return problem } + if contentType(content, visiting: []) == nil { + return "element content does not have a computable content type (7.2)" + } + if let problem = attributeSets(content, visiting: []).problem { return problem } + return interleaveProblem(content, visiting: []) + } + + func pathProblem(_ pattern: Pattern, flags: RelaxNGPathFlags, visiting: Set) -> String? { + switch pattern { + case let .attribute(nameClass, content): + return attributePathProblem(nameClass, content, flags: flags, visiting: visiting) + case let .ref(name): + guard !visiting.contains(name) else { return nil } + return pathProblem(resolve(name), flags: flags, visiting: visiting.union([name])) + case .text: + return flags.inList ? "text is not allowed inside list (7.1.3)" : nil + case .element: + return elementPathProblem(flags) + default: + return compositePathProblem(pattern, flags: flags, visiting: visiting) + } + } + + /// The composite cases: list opens the 7.1.3 context, group and + /// interleave under oneOrMore open the 7.1.2 context. + private func compositePathProblem(_ pattern: Pattern, flags: RelaxNGPathFlags, visiting: Set) -> String? { + switch pattern { + case let .list(inner): + if flags.inList { return "list is not allowed inside list (7.1.3)" } + var nested = flags + nested.inList = true + return pathProblem(inner, flags: nested, visiting: visiting) + case let .interleave(lhs, rhs), let .group(lhs, rhs): + if flags.inList, isInterleave(pattern) { return "interleave is not allowed inside list (7.1.3)" } + var nested = flags + if flags.underOneOrMore { nested.inOneOrMoreGroupOrInterleave = true } + return pathProblem(lhs, flags: nested, visiting: visiting) ?? pathProblem(rhs, flags: nested, visiting: visiting) + case let .choice(lhs, rhs): + return pathProblem(lhs, flags: flags, visiting: visiting) ?? pathProblem(rhs, flags: flags, visiting: visiting) + case let .oneOrMore(inner): + var nested = flags + nested.underOneOrMore = true + return pathProblem(inner, flags: nested, visiting: visiting) + case let .dataExcept(_, inner): + return pathProblem(inner, flags: flags, visiting: visiting) + default: + return nil + } + } + + private func isInterleave(_ pattern: Pattern) -> Bool { + if case .interleave = pattern { return true } + return false + } + + /// The attribute prohibitions of 7.1.1-7.1.3 and the infinite-name-class + /// rule of 7.3, then the attribute's own content under the inner flag. + private func attributePathProblem(_ nameClass: NameClass, _ content: Pattern, flags: RelaxNGPathFlags, visiting: Set) -> String? { + if flags.inAttribute { return "attribute is not allowed inside attribute (7.1.1)" } + if flags.inOneOrMoreGroupOrInterleave { + return "attribute is not allowed under oneOrMore with group or interleave (7.1.2)" + } + if flags.inList { return "attribute is not allowed inside list (7.1.3)" } + if !flags.underOneOrMore, isInfinite(nameClass) { + return "an attribute with an infinite name class must be inside oneOrMore (7.3)" + } + var inner = flags + inner.inAttribute = true + return pathProblem(content, flags: inner, visiting: visiting) + } + + /// In the simplified grammar a ref stands for an element, so the + /// prohibitions on ref under attribute and list translate to the element. + private func elementPathProblem(_ flags: RelaxNGPathFlags) -> String? { + if flags.inAttribute { return "element is not allowed inside attribute (7.1.1)" } + if flags.inList { return "element is not allowed inside list (7.1.3)" } + return nil // Otherwise a new context, walked separately. + } + + /// Whether a name class admits infinitely many names. + func isInfinite(_ nameClass: NameClass) -> Bool { + switch nameClass { + case .anyName, .anyNameExcept, .nsName, .nsNameExcept: + true + case let .choice(lhs, rhs): + isInfinite(lhs) || isInfinite(rhs) + case .name: + false + } + } +} diff --git a/Sources/Schema/RelaxNGRestrictions.swift b/Sources/Schema/RelaxNGRestrictions.swift new file mode 100644 index 0000000..11b5fe3 --- /dev/null +++ b/Sources/Schema/RelaxNGRestrictions.swift @@ -0,0 +1,22 @@ +/// The RELAX NG restrictions of spec sections 4.19 and 7, checked over the +/// compiled pattern algebra (refs resolved through the define table, elements +/// as context boundaries): prohibited paths (7.1), computable content types +/// (7.2), duplicate attributes (7.3), interleave constraints (7.4), the xmlns +/// name-class prohibition (4.16), and ref recursion not guarded by an element +/// (4.19). +enum RelaxNGRestrictions { + typealias Pattern = PureXML.Schema.Pattern + typealias NameClass = PureXML.Schema.NameClass + + static func violation(start: Pattern, defines: [String: Pattern]) -> String? { + // Recursion detection runs on the raw patterns: it happens before the + // notAllowed normalization (4.19 precedes 4.20). + if let problem = RelaxNGRestrictionChecker(defines: defines).recursionProblem(start: start) { + return problem + } + let checker = RelaxNGRestrictionChecker(defines: defines.mapValues(RelaxNGRestrictionChecker.normalized)) + let normalizedStart = RelaxNGRestrictionChecker.normalized(start) + if let problem = checker.startProblem(normalizedStart) { return problem } + return checker.elementProblems(normalizedStart) + } +} diff --git a/Sources/Schema/SchemaDocument.swift b/Sources/Schema/SchemaDocument.swift index 32504b4..53b38bc 100644 --- a/Sources/Schema/SchemaDocument.swift +++ b/Sources/Schema/SchemaDocument.swift @@ -9,6 +9,9 @@ public extension PureXML.Schema { /// an element with `maxOccurs` at most 1, and the group itself may occur at /// most once. case invalidAllGroup(reason: String) + /// A RELAX NG schema document does not match the RELAX NG grammar or + /// its restrictions (sections 3, 4.16-4.18). + case invalidRelaxNG(reason: String) /// The compiled schema fails one or more consistency rules (a `final` /// violation, an unfaithful restriction); every finding is carried, so a /// schema with several problems reports them all at once. @@ -22,6 +25,8 @@ public extension PureXML.Schema { "redefined type '\(type)' must derive from itself" case let .invalidAllGroup(reason): "invalid xs:all group: \(reason)" + case let .invalidRelaxNG(reason): + "invalid RELAX NG schema: \(reason)" case let .inconsistent(findings): findings.joined(separator: "; ") } diff --git a/Tests/RelaxNGCombineTests.swift b/Tests/RelaxNGCombineTests.swift index 2ffd121..c36db2e 100644 --- a/Tests/RelaxNGCombineTests.swift +++ b/Tests/RelaxNGCombineTests.swift @@ -40,8 +40,8 @@ struct RelaxNGCombineTests { #expect(try !valid(rng, "")) } - @Test("A plain redefinition without combine still replaces") - func test_plainRedefinitionReplaces() throws { + @Test("Two defines of one name must not both omit combine (4.17)") + func test_plainRedefinitionRejected() throws { let rng = """ @@ -49,7 +49,10 @@ struct RelaxNGCombineTests { """ - #expect(try !valid(rng, "")) - #expect(try valid(rng, "")) + // Spec 4.17: at most one define of a name may omit combine; a second + // combine-less definition is a schema error, not a replacement. + #expect(throws: PureXML.Schema.SchemaError.self) { + _ = try PureXML.Schema.RelaxNG(rng) + } } } diff --git a/Tests/RelaxNGDatatypeLibraryTests.swift b/Tests/RelaxNGDatatypeLibraryTests.swift index 7bed646..01df902 100644 --- a/Tests/RelaxNGDatatypeLibraryTests.swift +++ b/Tests/RelaxNGDatatypeLibraryTests.swift @@ -23,16 +23,19 @@ struct RelaxNGDatatypeLibraryTests { #expect(try validXML(rng, "anything here")) } - @Test("An XSD type without the XSD library is an unknown datatype and matches nothing") + @Test("An XSD type without the XSD library is a schema error") func test_unknownTypeInDefaultLibrary() throws { let rng = """ """ - // `integer` is not in the default library, so no value validates. - #expect(try !validXML(rng, "5")) - #expect(try !validXML(rng, "x")) + // The default library defines only string and token; an unknown + // datatype is a schema error (the spec suite's incorrect class), not + // a match-nothing pattern. + #expect(throws: PureXML.Schema.SchemaError.self) { + _ = try PureXML.Schema.RelaxNG(rng) + } } @Test("Declaring the XSD datatypeLibrary makes xsd types resolve") @@ -59,14 +62,16 @@ struct RelaxNGDatatypeLibraryTests { #expect(try !validXML(rng, "4.2")) } - @Test("A value with an unknown explicit type matches nothing") - func test_valueUnknownType() throws { + @Test("A value with an unknown explicit type is a schema error") + func test_unknownValueType() throws { let rng = """ 5 """ - #expect(try !validXML(rng, "5")) + #expect(throws: PureXML.Schema.SchemaError.self) { + _ = try PureXML.Schema.RelaxNG(rng) + } } @Test("nsName with except excludes a name in the namespace") diff --git a/Tests/RelaxNGSchemaCorrectnessTests.swift b/Tests/RelaxNGSchemaCorrectnessTests.swift new file mode 100644 index 0000000..61c71e5 --- /dev/null +++ b/Tests/RelaxNGSchemaCorrectnessTests.swift @@ -0,0 +1,70 @@ +@testable import PureXML +import Testing + +/// Schema-correctness validation (#131): a RELAX NG schema document must +/// itself match the RELAX NG grammar (section 3) and the restrictions of +/// 4.16-4.19 and 7.1-7.4 before any pattern interpretation. +@Suite("RELAX NG schema correctness") +struct RelaxNGSchemaCorrectnessTests { + private func rejects(_ rng: String) -> Bool { + (try? PureXML.Schema.RelaxNG(rng)) == nil + } + + private let rngNS = "xmlns=\"http://relaxng.org/ns/structure/1.0\"" + + @Test("Junk, misplaced elements, and illegal attributes are schema errors") + func test_grammarLevel() { + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(!rejects("")) + } + + @Test("4.16: except restrictions and the xmlns prohibition") + func test_section416() { + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + } + + @Test("4.17/4.19: combine conflicts and unguarded recursion") + func test_grammarSemantics() { + let conflict = """ + + + + """ + #expect(rejects(conflict)) + let recursion = """ + + + """ + #expect(rejects(recursion)) + #expect(rejects("")) + } + + @Test("7.1-7.4: prohibited paths, content types, attributes, interleave") + func test_section7() { + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + // The same shapes inside a notAllowed branch are normalized away (4.20). + let normalized = """ + + + """ + #expect(!rejects(normalized)) + } + + @Test("Datatype and URI checks") + func test_datatypes() { + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + } +} diff --git a/Tests/RelaxNGSpecSuiteTests.swift b/Tests/RelaxNGSpecSuiteTests.swift index 00c3aac..895a8ff 100644 --- a/Tests/RelaxNGSpecSuiteTests.swift +++ b/Tests/RelaxNGSpecSuiteTests.swift @@ -18,14 +18,16 @@ struct RelaxNGSpecSuiteTests { ProcessInfo.processInfo.environment["RNG_TS_ROOT"] } - /// The burn-down frontier (exact, so progress and regressions both show). - /// The instance level is fully clean: every valid instance is accepted - /// and every invalid one rejected. One class remains: schema correctness - /// is not validated, so every `incorrect` schema compiles (the count is - /// asserted exactly below); closing it means validating schemas against - /// the RELAX NG grammar and the section 4.16-4.18 restrictions before - /// pattern interpretation. - private let knownIncorrectCompiledCount = 213 + /// The suite is clean to one documented class. Instance level: every + /// valid instance accepted, every invalid one rejected. Schema level: + /// incorrect schemas are rejected by the grammar check (section 3), the + /// compiler checks (4.5-4.19), and the restrictions checker (7.1-7.4), + /// except the seven cases whose schema names use pre-Fifth-Edition + /// character classes (70, 72-74, 79, 86, 87): this package implements + /// XML 1.0 5e name productions, where those characters became legal (the + /// same class as xmltest 141 and the IBM P85-89 block). The count is + /// asserted exactly so drift in either direction is caught. + private let knownIncorrectCompiledCount = 7 private let knownCorrectRejected: Set = [] private let knownValidRejected: Set = [] private let knownInvalidAccepted: Set = [] @@ -182,7 +184,7 @@ struct RelaxNGSpecSuiteTests { } #expect( incorrectAccepted.count == knownIncorrectCompiledCount, - "incorrect-schema class drifted: \(incorrectAccepted.count) (baseline \(knownIncorrectCompiledCount))", + "incorrect-schema class drifted: \(incorrectAccepted.count) (baseline \(knownIncorrectCompiledCount)): \(incorrectAccepted)", ) #expect(correctRejected.isEmpty, "rejected \(correctRejected.count) correct schemas: \(correctRejected)") #expect(validRejected.isEmpty, "rejected \(validRejected.count) valid instances: \(validRejected)")