diff --git a/CHANGELOG.md b/CHANGELOG.md index 5d18ff4..ea38a46 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- An identity constraint's `selector`/`field` `xpath` is now validated against the XSD 1.0 restricted-XPath subset at compile time (#158). That subset (Part 1, 3.11.6) is far narrower than full XPath: a path is an optional leading `.//` then `/`-separated steps, a step is `.`, a name test (`QName`, `*`, or `NCName:*`), or a `child::` axis, and a `field` path may end in an attribute step (`@x` or `attribute::x`); there are no predicates (`[...]`), no other axes (`self::`, `descendant::`), no absolute or interior `//`. The xpath was stored raw and compiled lazily with `try?`, so a malformed one (a predicate, a bare `/`, an empty union branch, a spaced name-test colon) was silently accepted. It is now lexed into a whitespace-tolerant token stream and parsed against the grammar, so whitespace between tokens is allowed (`child :: a`, `. //.`) while `//` must be adjacent and a name test's own `:` may not be spaced; the name-test lexer uses the XML `NameChar` set, so a non-ASCII NCName is accepted. XSTS: invalid schemas accepted 785 to 736, no other bucket raised (valid schemas rejected held at 72). - A `complexContent` derivation's content is now checked for order and cardinality, not just child membership (#158). A `restriction` or `extension` inside `complexContent` has the content model `(annotation?, (group | all | choice | sequence)?, (attribute | attributeGroup)*, anyAttribute?)`: an optional model group, then any number of `attribute`/`attributeGroup`, then an optional `anyAttribute`. The `allowedChildren` membership table admitted the right child names but not their sequence or count, so a second model group (`sequence` then `sequence`), a model group after an attribute, an attribute after `anyAttribute`, or a second `anyAttribute` was accepted. The children are matched against an ordered slot model (one model group, unbounded attributes, one `anyAttribute`) by a forward-only walk; the check runs only in the `complexContent` context, where a model group is part of the model, so a `simpleContent` derivation (facets, no model group) keeps its own rules and no valid schema is rejected. XSTS: invalid schemas accepted 864 to 785, no other bucket raised (valid schemas rejected held at 72). - Unique Particle Attribution (`cos-nonambig`, the content-model determinism constraint) is now enforced at compile time (#156, #157), the keystone of the remaining schema-validity tail. A content model must be deterministic so each element in an instance is attributable to a single particle without lookahead; an ambiguous model (`` of two same-name elements, an optional or repeating element followed by a distinct same-name element, two same-name members of an `all`, an element competing with a wildcard that admits it) was accepted. The check runs a Glushkov position automaton over a literal, QName-resolved view of the raw schema tree built specifically for this rule, independent of the instance-validation content model: a `ref` is one position carrying its prefix-resolved QName (not the blanket target namespace), and substitution groups are not expanded, which removes the two false-overlap sources (cross-namespace refs, expanded substitution members) that a check over the compiled model produced. Determinism is per particle, the W3C rule's defining property: a particle competing with a repetition of itself is never a violation, so a fixed count `{n,n}` unrolls into copies in sequence (`b{2,2}, b` stays the deterministic `b, b, b`) while a true range or unbounded repeat uses a self-loop, and only two distinct source particles with overlapping labels conflict. The overlap test is QName-only (not substitution-group-aware), an accepted under-rejection that can never over-reject. XSTS: invalid schemas accepted 884 to 864, no other bucket raised (valid schemas rejected held at 72). - A `keyref`'s `refer` must name a `key` or `unique` constraint (#147). A `refer` pointing at a non-existent (or non-key/unique) constraint was accepted; it is now resolved against the document's `key`/`unique` names, with a forward `refer` (textually before the key) handled by collecting all names first. The check is skipped when the document pulls in external definitions through `import`/`include`/`redefine`, which the default compile does not load. XSTS: invalid schemas accepted 891 to 884, no other bucket raised. diff --git a/Sources/Schema/SchemaIdentityXPath.swift b/Sources/Schema/SchemaIdentityXPath.swift new file mode 100644 index 0000000..740eb62 --- /dev/null +++ b/Sources/Schema/SchemaIdentityXPath.swift @@ -0,0 +1,191 @@ +extension PureXML.Schema.XSDParser { + /// Validity of an identity constraint's `selector`/`field` `xpath` against the + /// XSD 1.0 restricted-XPath subset (Part 1, 3.11.6), a small grammar far + /// narrower than full XPath: + /// + /// Selector ::= Path ( '|' Path )* + /// Path ::= ('.//')? Step ( '/' Step )* + /// Step ::= '.' | NameTest | 'child' '::' NameTest + /// NameTest ::= QName | '*' | NCName ':' '*' + /// Field ::= Path ( '|' Path )* (a field's last step may be an attribute: + /// '@' NameTest or 'attribute' '::' NameTest) + /// + /// So no predicates (`[...]`), no axes beyond `child::`/`attribute::`, no absolute + /// or interior `//` (only a leading `.//`), and `@`/`attribute::` only as a + /// field's trailing step. Being a subset of XPath syntax, whitespace is allowed + /// between tokens (so `child :: a` and `. //.` are valid), but `//` must be two + /// adjacent slashes and a name test's own `:` admits no whitespace (so `tid : *` + /// is not). The xpath was stored raw and compiled lazily with `try?`, so a + /// malformed one was silently accepted; it is now rejected at compile time. + static func identityXPathErrors(_ node: XSDTree, local: String) -> [String] { + guard local == "selector" || local == "field", + let xpath = PureXML.Schema.XSDNode.attribute(node, "xpath") + else { return [] } + return validXPath(xpath, isField: local == "field") + ? [] + : ["the \(local) xpath '\(xpath)' is not a valid identity-constraint path"] + } + + private enum XPathToken: Equatable { + case slash, doubleSlash, axis, atSign, pipe, dot, star, name(String) + } + + private static func validXPath(_ xpath: String, isField: Bool) -> Bool { + guard let tokens = tokenize(xpath) else { return false } + let paths = tokens.split(separator: .pipe, omittingEmptySubsequences: false) + guard !paths.isEmpty else { return false } + return paths.allSatisfy { validPath(Array($0), isField: isField) } + } + + /// Lexes the restricted-XPath token stream, skipping inter-token whitespace, or + /// nil on an unlexable character (a predicate bracket, a parenthesis, a `:` that + /// is neither part of a name test nor a `::` axis). + private static func tokenize(_ xpath: String) -> [XPathToken]? { + var tokens: [XPathToken] = [] + let characters = Array(xpath) + var index = 0 + while index < characters.count { + let character = characters[index] + if character.isWhitespace { + index += 1 + } else if let simple = simpleToken(character) { + tokens.append(simple) + index += 1 + } else if character == "/" { + let double = index + 1 < characters.count && characters[index + 1] == "/" + tokens.append(double ? .doubleSlash : .slash) + index += double ? 2 : 1 + } else if character == ":" { + guard index + 1 < characters.count, characters[index + 1] == ":" else { return nil } + tokens.append(.axis) + index += 2 + } else if isNameStart(character) { + tokens.append(.name(readName(characters, &index))) + } else { + return nil + } + } + return tokens + } + + /// The single-character tokens that need no lookahead. + private static func simpleToken(_ character: Character) -> XPathToken? { + switch character { + case "|": .pipe + case "@": .atSign + case ".": .dot + case "*": .star + default: nil + } + } + + /// Reads a name test starting at `index`: a name run, optionally glued (no + /// whitespace) to `:` and a second name run or `*`, leaving a following `::` for + /// the lexer to read as an axis. + private static func readName(_ characters: [Character], _ index: inout Int) -> String { + var name = takeNameRun(characters, &index) + if index + 1 < characters.count, characters[index] == ":", characters[index + 1] != ":" { + index += 1 + if characters[index] == "*" { + name += ":*" + index += 1 + } else { + name += ":" + takeNameRun(characters, &index) + } + } + return name + } + + private static func takeNameRun(_ characters: [Character], _ index: inout Int) -> String { + var run = "" + while index < characters.count, isNameContinuation(characters[index]) { + run.append(characters[index]) + index += 1 + } + return run + } + + /// A name-test character set matching XML 1.0 `NameChar` (minus `:`, which is + /// glue or an axis here), so the lexer admits exactly the names `Lexical.isNCName` + /// will validate, including non-ASCII NCName characters Swift's `isLetter` misses. + private static func isNameContinuation(_ character: Character) -> Bool { + character != ":" && character.unicodeScalars.allSatisfy(PureXML.Parsing.XMLCharacter.isNameChar) + } + + /// A name-test start character: XML 1.0 `NameStartChar` (minus `:`); the first + /// scalar must start a name and any later scalars (a combining grapheme) be name + /// characters. + private static func isNameStart(_ character: Character) -> Bool { + guard character != ":", let first = character.unicodeScalars.first, + PureXML.Parsing.XMLCharacter.isNameStart(first) + else { return false } + return character.unicodeScalars.dropFirst().allSatisfy(PureXML.Parsing.XMLCharacter.isNameChar) + } + + /// An optional leading `.//`, then `/`-separated steps; `//` is rejected except + /// as that leading prefix, and an empty step (a leading/trailing/doubled `/`) + /// fails. + private static func validPath(_ tokens: [XPathToken], isField: Bool) -> Bool { + guard !tokens.isEmpty else { return false } + var body = tokens[...] + if body.first == .dot, body.dropFirst().first == .doubleSlash { + body = body.dropFirst(2) + guard !body.isEmpty else { return false } + } + var steps: [[XPathToken]] = [[]] + for token in body { + switch token { + case .slash: steps.append([]) + case .doubleSlash: return false + default: steps[steps.count - 1].append(token) + } + } + return steps.enumerated().allSatisfy { index, step in + validStep(step, isField: isField, isLast: index == steps.count - 1) + } + } + + /// A single step: `.`, a name test, a `child::` axis step, or (only as a field's + /// last step) an `@`/`attribute::` attribute step. + private static func validStep(_ step: [XPathToken], isField: Bool, isLast: Bool) -> Bool { + switch step.first { + case .dot where step.count == 1: + true + case .atSign: + isField && isLast && nameTest(Array(step.dropFirst())) + case .name("child") where step.count > 1 && step[1] == .axis: + nameTest(Array(step.dropFirst(2))) + case .name("attribute") where step.count > 1 && step[1] == .axis: + isField && isLast && nameTest(Array(step.dropFirst(2))) + default: + nameTest(step) + } + } + + /// A single name-test token: `*`, a QName, or `NCName ':' '*'`. + private static func nameTest(_ tokens: [XPathToken]) -> Bool { + guard tokens.count == 1 else { return false } + switch tokens[0] { + case .star: + return true + case let .name(value): + return validNameTestName(value) + default: + return false + } + } + + private static func validNameTestName(_ value: String) -> Bool { + let parts = value.split(separator: ":", omittingEmptySubsequences: false) + switch parts.count { + case 1: + return PureXML.Schema.Lexical.isNCName(value) + case 2: + let localPart = String(parts[1]) + return PureXML.Schema.Lexical.isNCName(String(parts[0])) + && (localPart == "*" || PureXML.Schema.Lexical.isNCName(localPart)) + default: + return false + } + } +} diff --git a/Sources/Schema/SchemaStructure.swift b/Sources/Schema/SchemaStructure.swift index 5307fa6..1b53acb 100644 --- a/Sources/Schema/SchemaStructure.swift +++ b/Sources/Schema/SchemaStructure.swift @@ -151,6 +151,9 @@ extension PureXML.Schema.XSDParser { if local == "complexContent" { errors += complexContentOrderErrors(node) } + if let local, local == "selector" || local == "field" { + errors += identityXPathErrors(node, local: local) + } } for child in children { collectStructure(child, into: &errors) diff --git a/Tests/SchemaIdentityXPathTests.swift b/Tests/SchemaIdentityXPathTests.swift new file mode 100644 index 0000000..f701f6e --- /dev/null +++ b/Tests/SchemaIdentityXPathTests.swift @@ -0,0 +1,71 @@ +@testable import PureXML +import Testing + +/// The XSD 1.0 restricted-XPath subset (Part 1, 3.11.6) for an identity +/// constraint's `selector` and `field`. A `selector` is a path of abbreviated +/// steps (`.`, `.//`, name tests, the `child::` axis); a `field` may additionally +/// end in an attribute step (`@x` or `attribute::x`). Whitespace between tokens is +/// allowed, but `//` must be adjacent and a name test's `:` may not be spaced; no +/// predicates, no absolute paths, no other axes. These pin both the rejections and +/// the valid shapes the W3C suite expects accepted (idI/idJ). +@Suite("Identity-constraint XPath subset") +struct SchemaIdentityXPathTests { + private func compile(_ selector: String, fields: [String] = ["@v"]) throws { + let fieldElements = fields.map { #""# }.joined() + _ = try PureXML.Schema.Document(#""" + + + + + + + \#(fieldElements) + + + + """#) + } + + private func rejectsSelector(_ selector: String) -> Bool { + do { try compile(selector) + return false + } catch { return true } + } + + private func rejectsField(_ field: String) -> Bool { + do { try compile("root", fields: [field]) + return false + } catch { return true } + } + + @Test("valid selector paths compile") + func test_validSelectors() throws { + for selector in [".", ".//a", "a/b/c", "a|b", ".//a | .//b", "*", "ns:a", "ns:*", "child::a", "child::ns:*", ". //."] { + try compile(selector) + } + // A name test built from non-ASCII NCName characters (U+203F UNDERTIE) is a + // valid name and must not be rejected by the path lexer. + try compile("a\u{203F}b") + } + + @Test("valid field paths compile, including attribute steps") + func test_validFields() throws { + for field in ["@v", "a/@v", "attribute::v", ".//a/@v", "@ns:v", "attribute::ns:v", "a | @v", "attribute :: v"] { + try compile("root", fields: [field]) + } + } + + @Test("malformed selector paths are rejected") + func test_invalidSelectors() { + for selector in ["", "|", "| a", "a|", "/", "//", "/a", ".//", ".//.//a", "a//b", "self::a", "descendant::a", "a[1]", "child::", "tid : *", "@v", "attribute::v"] { + #expect(rejectsSelector(selector), "expected rejection: '\(selector)'") + } + } + + @Test("malformed field paths are rejected") + func test_invalidFields() { + for field in ["", "a[1]", "@ ", "@*[1]", "/@v", "self::v", "a/@v/b", "tid : *"] { + #expect(rejectsField(field), "expected rejection: '\(field)'") + } + } +} diff --git a/Tests/XSDIdentityTests.swift b/Tests/XSDIdentityTests.swift index 9aa65b4..4248692 100644 --- a/Tests/XSDIdentityTests.swift +++ b/Tests/XSDIdentityTests.swift @@ -160,8 +160,8 @@ struct XSDIdentityTests { #expect(try validate(xsd, dangling).contains { $0.reason.contains("no matching key 'pk'") }) } - @Test("A malformed field XPath is a located error, not a silently disabled constraint") - func test_malformedFieldReported() throws { + @Test("A malformed field XPath is rejected at compile time, not silently disabled") + func test_malformedFieldReported() { let xsd = """ \(head) @@ -172,12 +172,11 @@ struct XSDIdentityTests { """ - let errors = try validate(xsd, "") - #expect(errors.contains { $0.reason.contains("invalid field XPath '@id['") }, "\(errors.map(\.reason))") + #expect(rejectsIdentityXPath(xsd), "a field xpath with a predicate bracket is not in the XSD subset") } - @Test("A malformed selector XPath is a located error") - func test_malformedSelectorReported() throws { + @Test("A malformed selector XPath is rejected at compile time") + func test_malformedSelectorReported() { let xsd = """ \(head) @@ -188,8 +187,18 @@ struct XSDIdentityTests { """ - let errors = try validate(xsd, "") - #expect(errors.contains { $0.reason.contains("invalid selector XPath 'item['") }, "\(errors.map(\.reason))") + #expect(rejectsIdentityXPath(xsd), "a selector xpath with a predicate bracket is not in the XSD subset") + } + + /// Whether compiling `xsd` is rejected for an out-of-subset identity-constraint + /// XPath (the malformed path is caught at compile time, not silently disabled). + private func rejectsIdentityXPath(_ xsd: String) -> Bool { + do { + _ = try PureXML.Schema.Document(xsd) + return false + } catch { + return "\(error)".contains("is not a valid identity-constraint path") + } } @Test("A child-element field locates the error at that child") diff --git a/Tests/XSTSSuiteTests.swift b/Tests/XSTSSuiteTests.swift index 447d569..7519e83 100644 --- a/Tests/XSTSSuiteTests.swift +++ b/Tests/XSTSSuiteTests.swift @@ -29,7 +29,7 @@ struct XSTSSuiteTests { /// XSTS` (debug is far slower and the corpus is large). Per-case deviations /// are written to /tmp/xsts-failures.txt for the burn-down. private let knownSchemaValidRejected = 72 - private let knownSchemaInvalidAccepted = 785 + private let knownSchemaInvalidAccepted = 736 private let knownInstanceValidRejected = 214 private let knownInstanceInvalidAccepted = 165 diff --git a/docs/schema-validity-burndown.md b/docs/schema-validity-burndown.md index 732d2ac..5b821b1 100644 --- a/docs/schema-validity-burndown.md +++ b/docs/schema-validity-burndown.md @@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open. | Bucket | Count | Reading | |---|---|---| | valid-schemas-rejected | 72 | we rarely reject a good schema | -| **invalid-schemas-accepted** | **2461 -> 785** | we rarely catch a bad one (facet, id, structural content-model, UPA determinism, and complexContent-derivation order validity landed) | +| **invalid-schemas-accepted** | **2461 -> 736** | we rarely catch a bad one (facet, id, structural content-model, UPA determinism, complexContent-derivation order, and identity-constraint XPath-subset validity landed) | | valid-instances-rejected | 233 | (instance side, separately tracked) | | invalid-instances-accepted | 165 | |