feat(schema): validate identity-constraint selector/field XPath subset
An identity constraint's selector and field xpath must be in the XSD 1.0 restricted-XPath subset (Part 1, 3.11.6), far narrower than full XPath: a leading optional .//, then /-separated steps, a step being '.', a name test (QName, '*', or NCName:*), or a child:: axis; a field path may end in an attribute step (@x or attribute::x). No predicates, no other axes, no absolute or interior //. The xpath was stored raw and compiled lazily with try?, so a malformed one was silently accepted. It is now lexed into a whitespace-tolerant token stream and parsed against the grammar: whitespace between tokens is allowed (child :: a, . //.) while // must be adjacent and a name test's own ':' may not be spaced. The name lexer uses the XML NameChar set so a non-ASCII NCName is accepted. The two identity tests that fed a malformed xpath and expected a validation-time located error now assert the stronger compile-time rejection. XSTS: invalid schemas accepted 785 to 736, valid schemas rejected held at 72, instance buckets unchanged.
This commit is contained in:
parent
8a7445c5bb
commit
cfaf9b7dd1
@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- An identity constraint's `selector`/`field` `xpath` is now validated against the XSD 1.0 restricted-XPath subset at compile time (#158). That subset (Part 1, 3.11.6) is far narrower than full XPath: a path is an optional leading `.//` then `/`-separated steps, a step is `.`, a name test (`QName`, `*`, or `NCName:*`), or a `child::` axis, and a `field` path may end in an attribute step (`@x` or `attribute::x`); there are no predicates (`[...]`), no other axes (`self::`, `descendant::`), no absolute or interior `//`. The xpath was stored raw and compiled lazily with `try?`, so a malformed one (a predicate, a bare `/`, an empty union branch, a spaced name-test colon) was silently accepted. It is now lexed into a whitespace-tolerant token stream and parsed against the grammar, so whitespace between tokens is allowed (`child :: a`, `. //.`) while `//` must be adjacent and a name test's own `:` may not be spaced; the name-test lexer uses the XML `NameChar` set, so a non-ASCII NCName is accepted. XSTS: invalid schemas accepted 785 to 736, no other bucket raised (valid schemas rejected held at 72).
|
||||
- A `complexContent` derivation's content is now checked for order and cardinality, not just child membership (#158). A `restriction` or `extension` inside `complexContent` has the content model `(annotation?, (group | all | choice | sequence)?, (attribute | attributeGroup)*, anyAttribute?)`: an optional model group, then any number of `attribute`/`attributeGroup`, then an optional `anyAttribute`. The `allowedChildren` membership table admitted the right child names but not their sequence or count, so a second model group (`sequence` then `sequence`), a model group after an attribute, an attribute after `anyAttribute`, or a second `anyAttribute` was accepted. The children are matched against an ordered slot model (one model group, unbounded attributes, one `anyAttribute`) by a forward-only walk; the check runs only in the `complexContent` context, where a model group is part of the model, so a `simpleContent` derivation (facets, no model group) keeps its own rules and no valid schema is rejected. XSTS: invalid schemas accepted 864 to 785, no other bucket raised (valid schemas rejected held at 72).
|
||||
- Unique Particle Attribution (`cos-nonambig`, the content-model determinism constraint) is now enforced at compile time (#156, #157), the keystone of the remaining schema-validity tail. A content model must be deterministic so each element in an instance is attributable to a single particle without lookahead; an ambiguous model (`<choice>` of two same-name elements, an optional or repeating element followed by a distinct same-name element, two same-name members of an `all`, an element competing with a wildcard that admits it) was accepted. The check runs a Glushkov position automaton over a literal, QName-resolved view of the raw schema tree built specifically for this rule, independent of the instance-validation content model: a `ref` is one position carrying its prefix-resolved QName (not the blanket target namespace), and substitution groups are not expanded, which removes the two false-overlap sources (cross-namespace refs, expanded substitution members) that a check over the compiled model produced. Determinism is per particle, the W3C rule's defining property: a particle competing with a repetition of itself is never a violation, so a fixed count `{n,n}` unrolls into copies in sequence (`b{2,2}, b` stays the deterministic `b, b, b`) while a true range or unbounded repeat uses a self-loop, and only two distinct source particles with overlapping labels conflict. The overlap test is QName-only (not substitution-group-aware), an accepted under-rejection that can never over-reject. XSTS: invalid schemas accepted 884 to 864, no other bucket raised (valid schemas rejected held at 72).
|
||||
- A `keyref`'s `refer` must name a `key` or `unique` constraint (#147). A `refer` pointing at a non-existent (or non-key/unique) constraint was accepted; it is now resolved against the document's `key`/`unique` names, with a forward `refer` (textually before the key) handled by collecting all names first. The check is skipped when the document pulls in external definitions through `import`/`include`/`redefine`, which the default compile does not load. XSTS: invalid schemas accepted 891 to 884, no other bucket raised.
|
||||
|
||||
191
Sources/Schema/SchemaIdentityXPath.swift
Normal file
191
Sources/Schema/SchemaIdentityXPath.swift
Normal file
@ -0,0 +1,191 @@
|
||||
extension PureXML.Schema.XSDParser {
|
||||
/// Validity of an identity constraint's `selector`/`field` `xpath` against the
|
||||
/// XSD 1.0 restricted-XPath subset (Part 1, 3.11.6), a small grammar far
|
||||
/// narrower than full XPath:
|
||||
///
|
||||
/// Selector ::= Path ( '|' Path )*
|
||||
/// Path ::= ('.//')? Step ( '/' Step )*
|
||||
/// Step ::= '.' | NameTest | 'child' '::' NameTest
|
||||
/// NameTest ::= QName | '*' | NCName ':' '*'
|
||||
/// Field ::= Path ( '|' Path )* (a field's last step may be an attribute:
|
||||
/// '@' NameTest or 'attribute' '::' NameTest)
|
||||
///
|
||||
/// So no predicates (`[...]`), no axes beyond `child::`/`attribute::`, no absolute
|
||||
/// or interior `//` (only a leading `.//`), and `@`/`attribute::` only as a
|
||||
/// field's trailing step. Being a subset of XPath syntax, whitespace is allowed
|
||||
/// between tokens (so `child :: a` and `. //.` are valid), but `//` must be two
|
||||
/// adjacent slashes and a name test's own `:` admits no whitespace (so `tid : *`
|
||||
/// is not). The xpath was stored raw and compiled lazily with `try?`, so a
|
||||
/// malformed one was silently accepted; it is now rejected at compile time.
|
||||
static func identityXPathErrors(_ node: XSDTree, local: String) -> [String] {
|
||||
guard local == "selector" || local == "field",
|
||||
let xpath = PureXML.Schema.XSDNode.attribute(node, "xpath")
|
||||
else { return [] }
|
||||
return validXPath(xpath, isField: local == "field")
|
||||
? []
|
||||
: ["the \(local) xpath '\(xpath)' is not a valid identity-constraint path"]
|
||||
}
|
||||
|
||||
private enum XPathToken: Equatable {
|
||||
case slash, doubleSlash, axis, atSign, pipe, dot, star, name(String)
|
||||
}
|
||||
|
||||
private static func validXPath(_ xpath: String, isField: Bool) -> Bool {
|
||||
guard let tokens = tokenize(xpath) else { return false }
|
||||
let paths = tokens.split(separator: .pipe, omittingEmptySubsequences: false)
|
||||
guard !paths.isEmpty else { return false }
|
||||
return paths.allSatisfy { validPath(Array($0), isField: isField) }
|
||||
}
|
||||
|
||||
/// Lexes the restricted-XPath token stream, skipping inter-token whitespace, or
|
||||
/// nil on an unlexable character (a predicate bracket, a parenthesis, a `:` that
|
||||
/// is neither part of a name test nor a `::` axis).
|
||||
private static func tokenize(_ xpath: String) -> [XPathToken]? {
|
||||
var tokens: [XPathToken] = []
|
||||
let characters = Array(xpath)
|
||||
var index = 0
|
||||
while index < characters.count {
|
||||
let character = characters[index]
|
||||
if character.isWhitespace {
|
||||
index += 1
|
||||
} else if let simple = simpleToken(character) {
|
||||
tokens.append(simple)
|
||||
index += 1
|
||||
} else if character == "/" {
|
||||
let double = index + 1 < characters.count && characters[index + 1] == "/"
|
||||
tokens.append(double ? .doubleSlash : .slash)
|
||||
index += double ? 2 : 1
|
||||
} else if character == ":" {
|
||||
guard index + 1 < characters.count, characters[index + 1] == ":" else { return nil }
|
||||
tokens.append(.axis)
|
||||
index += 2
|
||||
} else if isNameStart(character) {
|
||||
tokens.append(.name(readName(characters, &index)))
|
||||
} else {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return tokens
|
||||
}
|
||||
|
||||
/// The single-character tokens that need no lookahead.
|
||||
private static func simpleToken(_ character: Character) -> XPathToken? {
|
||||
switch character {
|
||||
case "|": .pipe
|
||||
case "@": .atSign
|
||||
case ".": .dot
|
||||
case "*": .star
|
||||
default: nil
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads a name test starting at `index`: a name run, optionally glued (no
|
||||
/// whitespace) to `:` and a second name run or `*`, leaving a following `::` for
|
||||
/// the lexer to read as an axis.
|
||||
private static func readName(_ characters: [Character], _ index: inout Int) -> String {
|
||||
var name = takeNameRun(characters, &index)
|
||||
if index + 1 < characters.count, characters[index] == ":", characters[index + 1] != ":" {
|
||||
index += 1
|
||||
if characters[index] == "*" {
|
||||
name += ":*"
|
||||
index += 1
|
||||
} else {
|
||||
name += ":" + takeNameRun(characters, &index)
|
||||
}
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
private static func takeNameRun(_ characters: [Character], _ index: inout Int) -> String {
|
||||
var run = ""
|
||||
while index < characters.count, isNameContinuation(characters[index]) {
|
||||
run.append(characters[index])
|
||||
index += 1
|
||||
}
|
||||
return run
|
||||
}
|
||||
|
||||
/// A name-test character set matching XML 1.0 `NameChar` (minus `:`, which is
|
||||
/// glue or an axis here), so the lexer admits exactly the names `Lexical.isNCName`
|
||||
/// will validate, including non-ASCII NCName characters Swift's `isLetter` misses.
|
||||
private static func isNameContinuation(_ character: Character) -> Bool {
|
||||
character != ":" && character.unicodeScalars.allSatisfy(PureXML.Parsing.XMLCharacter.isNameChar)
|
||||
}
|
||||
|
||||
/// A name-test start character: XML 1.0 `NameStartChar` (minus `:`); the first
|
||||
/// scalar must start a name and any later scalars (a combining grapheme) be name
|
||||
/// characters.
|
||||
private static func isNameStart(_ character: Character) -> Bool {
|
||||
guard character != ":", let first = character.unicodeScalars.first,
|
||||
PureXML.Parsing.XMLCharacter.isNameStart(first)
|
||||
else { return false }
|
||||
return character.unicodeScalars.dropFirst().allSatisfy(PureXML.Parsing.XMLCharacter.isNameChar)
|
||||
}
|
||||
|
||||
/// An optional leading `.//`, then `/`-separated steps; `//` is rejected except
|
||||
/// as that leading prefix, and an empty step (a leading/trailing/doubled `/`)
|
||||
/// fails.
|
||||
private static func validPath(_ tokens: [XPathToken], isField: Bool) -> Bool {
|
||||
guard !tokens.isEmpty else { return false }
|
||||
var body = tokens[...]
|
||||
if body.first == .dot, body.dropFirst().first == .doubleSlash {
|
||||
body = body.dropFirst(2)
|
||||
guard !body.isEmpty else { return false }
|
||||
}
|
||||
var steps: [[XPathToken]] = [[]]
|
||||
for token in body {
|
||||
switch token {
|
||||
case .slash: steps.append([])
|
||||
case .doubleSlash: return false
|
||||
default: steps[steps.count - 1].append(token)
|
||||
}
|
||||
}
|
||||
return steps.enumerated().allSatisfy { index, step in
|
||||
validStep(step, isField: isField, isLast: index == steps.count - 1)
|
||||
}
|
||||
}
|
||||
|
||||
/// A single step: `.`, a name test, a `child::` axis step, or (only as a field's
|
||||
/// last step) an `@`/`attribute::` attribute step.
|
||||
private static func validStep(_ step: [XPathToken], isField: Bool, isLast: Bool) -> Bool {
|
||||
switch step.first {
|
||||
case .dot where step.count == 1:
|
||||
true
|
||||
case .atSign:
|
||||
isField && isLast && nameTest(Array(step.dropFirst()))
|
||||
case .name("child") where step.count > 1 && step[1] == .axis:
|
||||
nameTest(Array(step.dropFirst(2)))
|
||||
case .name("attribute") where step.count > 1 && step[1] == .axis:
|
||||
isField && isLast && nameTest(Array(step.dropFirst(2)))
|
||||
default:
|
||||
nameTest(step)
|
||||
}
|
||||
}
|
||||
|
||||
/// A single name-test token: `*`, a QName, or `NCName ':' '*'`.
|
||||
private static func nameTest(_ tokens: [XPathToken]) -> Bool {
|
||||
guard tokens.count == 1 else { return false }
|
||||
switch tokens[0] {
|
||||
case .star:
|
||||
return true
|
||||
case let .name(value):
|
||||
return validNameTestName(value)
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
private static func validNameTestName(_ value: String) -> Bool {
|
||||
let parts = value.split(separator: ":", omittingEmptySubsequences: false)
|
||||
switch parts.count {
|
||||
case 1:
|
||||
return PureXML.Schema.Lexical.isNCName(value)
|
||||
case 2:
|
||||
let localPart = String(parts[1])
|
||||
return PureXML.Schema.Lexical.isNCName(String(parts[0]))
|
||||
&& (localPart == "*" || PureXML.Schema.Lexical.isNCName(localPart))
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -151,6 +151,9 @@ extension PureXML.Schema.XSDParser {
|
||||
if local == "complexContent" {
|
||||
errors += complexContentOrderErrors(node)
|
||||
}
|
||||
if let local, local == "selector" || local == "field" {
|
||||
errors += identityXPathErrors(node, local: local)
|
||||
}
|
||||
}
|
||||
for child in children {
|
||||
collectStructure(child, into: &errors)
|
||||
|
||||
71
Tests/SchemaIdentityXPathTests.swift
Normal file
71
Tests/SchemaIdentityXPathTests.swift
Normal file
@ -0,0 +1,71 @@
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
/// The XSD 1.0 restricted-XPath subset (Part 1, 3.11.6) for an identity
|
||||
/// constraint's `selector` and `field`. A `selector` is a path of abbreviated
|
||||
/// steps (`.`, `.//`, name tests, the `child::` axis); a `field` may additionally
|
||||
/// end in an attribute step (`@x` or `attribute::x`). Whitespace between tokens is
|
||||
/// allowed, but `//` must be adjacent and a name test's `:` may not be spaced; no
|
||||
/// predicates, no absolute paths, no other axes. These pin both the rejections and
|
||||
/// the valid shapes the W3C suite expects accepted (idI/idJ).
|
||||
@Suite("Identity-constraint XPath subset")
|
||||
struct SchemaIdentityXPathTests {
|
||||
private func compile(_ selector: String, fields: [String] = ["@v"]) throws {
|
||||
let fieldElements = fields.map { #"<xs:field xpath="\#($0)"/>"# }.joined()
|
||||
_ = try PureXML.Schema.Document(#"""
|
||||
<xs:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">
|
||||
<xs:element name="root" type="xs:string"/>
|
||||
<xs:element name="r">
|
||||
<xs:complexType><xs:sequence><xs:element ref="root"/></xs:sequence></xs:complexType>
|
||||
<xs:unique name="u">
|
||||
<xs:selector xpath="\#(selector)"/>
|
||||
\#(fieldElements)
|
||||
</xs:unique>
|
||||
</xs:element>
|
||||
</xs:schema>
|
||||
"""#)
|
||||
}
|
||||
|
||||
private func rejectsSelector(_ selector: String) -> Bool {
|
||||
do { try compile(selector)
|
||||
return false
|
||||
} catch { return true }
|
||||
}
|
||||
|
||||
private func rejectsField(_ field: String) -> Bool {
|
||||
do { try compile("root", fields: [field])
|
||||
return false
|
||||
} catch { return true }
|
||||
}
|
||||
|
||||
@Test("valid selector paths compile")
|
||||
func test_validSelectors() throws {
|
||||
for selector in [".", ".//a", "a/b/c", "a|b", ".//a | .//b", "*", "ns:a", "ns:*", "child::a", "child::ns:*", ". //."] {
|
||||
try compile(selector)
|
||||
}
|
||||
// A name test built from non-ASCII NCName characters (U+203F UNDERTIE) is a
|
||||
// valid name and must not be rejected by the path lexer.
|
||||
try compile("a\u{203F}b")
|
||||
}
|
||||
|
||||
@Test("valid field paths compile, including attribute steps")
|
||||
func test_validFields() throws {
|
||||
for field in ["@v", "a/@v", "attribute::v", ".//a/@v", "@ns:v", "attribute::ns:v", "a | @v", "attribute :: v"] {
|
||||
try compile("root", fields: [field])
|
||||
}
|
||||
}
|
||||
|
||||
@Test("malformed selector paths are rejected")
|
||||
func test_invalidSelectors() {
|
||||
for selector in ["", "|", "| a", "a|", "/", "//", "/a", ".//", ".//.//a", "a//b", "self::a", "descendant::a", "a[1]", "child::", "tid : *", "@v", "attribute::v"] {
|
||||
#expect(rejectsSelector(selector), "expected rejection: '\(selector)'")
|
||||
}
|
||||
}
|
||||
|
||||
@Test("malformed field paths are rejected")
|
||||
func test_invalidFields() {
|
||||
for field in ["", "a[1]", "@ ", "@*[1]", "/@v", "self::v", "a/@v/b", "tid : *"] {
|
||||
#expect(rejectsField(field), "expected rejection: '\(field)'")
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -160,8 +160,8 @@ struct XSDIdentityTests {
|
||||
#expect(try validate(xsd, dangling).contains { $0.reason.contains("no matching key 'pk'") })
|
||||
}
|
||||
|
||||
@Test("A malformed field XPath is a located error, not a silently disabled constraint")
|
||||
func test_malformedFieldReported() throws {
|
||||
@Test("A malformed field XPath is rejected at compile time, not silently disabled")
|
||||
func test_malformedFieldReported() {
|
||||
let xsd = """
|
||||
\(head)
|
||||
<xs:element name="list">
|
||||
@ -172,12 +172,11 @@ struct XSDIdentityTests {
|
||||
</xs:element>
|
||||
</xs:schema>
|
||||
"""
|
||||
let errors = try validate(xsd, "<list><item id=\"1\"/><item id=\"1\"/></list>")
|
||||
#expect(errors.contains { $0.reason.contains("invalid field XPath '@id['") }, "\(errors.map(\.reason))")
|
||||
#expect(rejectsIdentityXPath(xsd), "a field xpath with a predicate bracket is not in the XSD subset")
|
||||
}
|
||||
|
||||
@Test("A malformed selector XPath is a located error")
|
||||
func test_malformedSelectorReported() throws {
|
||||
@Test("A malformed selector XPath is rejected at compile time")
|
||||
func test_malformedSelectorReported() {
|
||||
let xsd = """
|
||||
\(head)
|
||||
<xs:element name="list">
|
||||
@ -188,8 +187,18 @@ struct XSDIdentityTests {
|
||||
</xs:element>
|
||||
</xs:schema>
|
||||
"""
|
||||
let errors = try validate(xsd, "<list><item id=\"1\"/></list>")
|
||||
#expect(errors.contains { $0.reason.contains("invalid selector XPath 'item['") }, "\(errors.map(\.reason))")
|
||||
#expect(rejectsIdentityXPath(xsd), "a selector xpath with a predicate bracket is not in the XSD subset")
|
||||
}
|
||||
|
||||
/// Whether compiling `xsd` is rejected for an out-of-subset identity-constraint
|
||||
/// XPath (the malformed path is caught at compile time, not silently disabled).
|
||||
private func rejectsIdentityXPath(_ xsd: String) -> Bool {
|
||||
do {
|
||||
_ = try PureXML.Schema.Document(xsd)
|
||||
return false
|
||||
} catch {
|
||||
return "\(error)".contains("is not a valid identity-constraint path")
|
||||
}
|
||||
}
|
||||
|
||||
@Test("A child-element field locates the error at that child")
|
||||
|
||||
@ -29,7 +29,7 @@ struct XSTSSuiteTests {
|
||||
/// XSTS` (debug is far slower and the corpus is large). Per-case deviations
|
||||
/// are written to /tmp/xsts-failures.txt for the burn-down.
|
||||
private let knownSchemaValidRejected = 72
|
||||
private let knownSchemaInvalidAccepted = 785
|
||||
private let knownSchemaInvalidAccepted = 736
|
||||
private let knownInstanceValidRejected = 214
|
||||
private let knownInstanceInvalidAccepted = 165
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open.
|
||||
| Bucket | Count | Reading |
|
||||
|---|---|---|
|
||||
| valid-schemas-rejected | 72 | we rarely reject a good schema |
|
||||
| **invalid-schemas-accepted** | **2461 -> 785** | we rarely catch a bad one (facet, id, structural content-model, UPA determinism, and complexContent-derivation order validity landed) |
|
||||
| **invalid-schemas-accepted** | **2461 -> 736** | we rarely catch a bad one (facet, id, structural content-model, UPA determinism, complexContent-derivation order, and identity-constraint XPath-subset validity landed) |
|
||||
| valid-instances-rejected | 233 | (instance side, separately tracked) |
|
||||
| invalid-instances-accepted | 165 | |
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user