feat(schema): validate identity-constraint selector/field XPath subset

An identity constraint's selector and field xpath must be in the XSD 1.0
restricted-XPath subset (Part 1, 3.11.6), far narrower than full XPath: a
leading optional .//, then /-separated steps, a step being '.', a name test
(QName, '*', or NCName:*), or a child:: axis; a field path may end in an
attribute step (@x or attribute::x). No predicates, no other axes, no absolute
or interior //. The xpath was stored raw and compiled lazily with try?, so a
malformed one was silently accepted.

It is now lexed into a whitespace-tolerant token stream and parsed against the
grammar: whitespace between tokens is allowed (child :: a, . //.) while // must
be adjacent and a name test's own ':' may not be spaced. The name lexer uses
the XML NameChar set so a non-ASCII NCName is accepted.

The two identity tests that fed a malformed xpath and expected a
validation-time located error now assert the stronger compile-time rejection.

XSTS: invalid schemas accepted 785 to 736, valid schemas rejected held at 72,
instance buckets unchanged.
This commit is contained in:
Mihaela Mihaljevic 2026-06-14 10:30:06 +02:00
parent 8a7445c5bb
commit cfaf9b7dd1
7 changed files with 285 additions and 10 deletions

View File

@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- An identity constraint's `selector`/`field` `xpath` is now validated against the XSD 1.0 restricted-XPath subset at compile time (#158). That subset (Part 1, 3.11.6) is far narrower than full XPath: a path is an optional leading `.//` then `/`-separated steps, a step is `.`, a name test (`QName`, `*`, or `NCName:*`), or a `child::` axis, and a `field` path may end in an attribute step (`@x` or `attribute::x`); there are no predicates (`[...]`), no other axes (`self::`, `descendant::`), no absolute or interior `//`. The xpath was stored raw and compiled lazily with `try?`, so a malformed one (a predicate, a bare `/`, an empty union branch, a spaced name-test colon) was silently accepted. It is now lexed into a whitespace-tolerant token stream and parsed against the grammar, so whitespace between tokens is allowed (`child :: a`, `. //.`) while `//` must be adjacent and a name test's own `:` may not be spaced; the name-test lexer uses the XML `NameChar` set, so a non-ASCII NCName is accepted. XSTS: invalid schemas accepted 785 to 736, no other bucket raised (valid schemas rejected held at 72).
- A `complexContent` derivation's content is now checked for order and cardinality, not just child membership (#158). A `restriction` or `extension` inside `complexContent` has the content model `(annotation?, (group | all | choice | sequence)?, (attribute | attributeGroup)*, anyAttribute?)`: an optional model group, then any number of `attribute`/`attributeGroup`, then an optional `anyAttribute`. The `allowedChildren` membership table admitted the right child names but not their sequence or count, so a second model group (`sequence` then `sequence`), a model group after an attribute, an attribute after `anyAttribute`, or a second `anyAttribute` was accepted. The children are matched against an ordered slot model (one model group, unbounded attributes, one `anyAttribute`) by a forward-only walk; the check runs only in the `complexContent` context, where a model group is part of the model, so a `simpleContent` derivation (facets, no model group) keeps its own rules and no valid schema is rejected. XSTS: invalid schemas accepted 864 to 785, no other bucket raised (valid schemas rejected held at 72).
- Unique Particle Attribution (`cos-nonambig`, the content-model determinism constraint) is now enforced at compile time (#156, #157), the keystone of the remaining schema-validity tail. A content model must be deterministic so each element in an instance is attributable to a single particle without lookahead; an ambiguous model (`<choice>` of two same-name elements, an optional or repeating element followed by a distinct same-name element, two same-name members of an `all`, an element competing with a wildcard that admits it) was accepted. The check runs a Glushkov position automaton over a literal, QName-resolved view of the raw schema tree built specifically for this rule, independent of the instance-validation content model: a `ref` is one position carrying its prefix-resolved QName (not the blanket target namespace), and substitution groups are not expanded, which removes the two false-overlap sources (cross-namespace refs, expanded substitution members) that a check over the compiled model produced. Determinism is per particle, the W3C rule's defining property: a particle competing with a repetition of itself is never a violation, so a fixed count `{n,n}` unrolls into copies in sequence (`b{2,2}, b` stays the deterministic `b, b, b`) while a true range or unbounded repeat uses a self-loop, and only two distinct source particles with overlapping labels conflict. The overlap test is QName-only (not substitution-group-aware), an accepted under-rejection that can never over-reject. XSTS: invalid schemas accepted 884 to 864, no other bucket raised (valid schemas rejected held at 72).
- A `keyref`'s `refer` must name a `key` or `unique` constraint (#147). A `refer` pointing at a non-existent (or non-key/unique) constraint was accepted; it is now resolved against the document's `key`/`unique` names, with a forward `refer` (textually before the key) handled by collecting all names first. The check is skipped when the document pulls in external definitions through `import`/`include`/`redefine`, which the default compile does not load. XSTS: invalid schemas accepted 891 to 884, no other bucket raised.

View File

@ -0,0 +1,191 @@
extension PureXML.Schema.XSDParser {
/// Validity of an identity constraint's `selector`/`field` `xpath` against the
/// XSD 1.0 restricted-XPath subset (Part 1, 3.11.6), a small grammar far
/// narrower than full XPath:
///
/// Selector ::= Path ( '|' Path )*
/// Path ::= ('.//')? Step ( '/' Step )*
/// Step ::= '.' | NameTest | 'child' '::' NameTest
/// NameTest ::= QName | '*' | NCName ':' '*'
/// Field ::= Path ( '|' Path )* (a field's last step may be an attribute:
/// '@' NameTest or 'attribute' '::' NameTest)
///
/// So no predicates (`[...]`), no axes beyond `child::`/`attribute::`, no absolute
/// or interior `//` (only a leading `.//`), and `@`/`attribute::` only as a
/// field's trailing step. Being a subset of XPath syntax, whitespace is allowed
/// between tokens (so `child :: a` and `. //.` are valid), but `//` must be two
/// adjacent slashes and a name test's own `:` admits no whitespace (so `tid : *`
/// is not). The xpath was stored raw and compiled lazily with `try?`, so a
/// malformed one was silently accepted; it is now rejected at compile time.
static func identityXPathErrors(_ node: XSDTree, local: String) -> [String] {
guard local == "selector" || local == "field",
let xpath = PureXML.Schema.XSDNode.attribute(node, "xpath")
else { return [] }
return validXPath(xpath, isField: local == "field")
? []
: ["the \(local) xpath '\(xpath)' is not a valid identity-constraint path"]
}
private enum XPathToken: Equatable {
case slash, doubleSlash, axis, atSign, pipe, dot, star, name(String)
}
private static func validXPath(_ xpath: String, isField: Bool) -> Bool {
guard let tokens = tokenize(xpath) else { return false }
let paths = tokens.split(separator: .pipe, omittingEmptySubsequences: false)
guard !paths.isEmpty else { return false }
return paths.allSatisfy { validPath(Array($0), isField: isField) }
}
/// Lexes the restricted-XPath token stream, skipping inter-token whitespace, or
/// nil on an unlexable character (a predicate bracket, a parenthesis, a `:` that
/// is neither part of a name test nor a `::` axis).
private static func tokenize(_ xpath: String) -> [XPathToken]? {
var tokens: [XPathToken] = []
let characters = Array(xpath)
var index = 0
while index < characters.count {
let character = characters[index]
if character.isWhitespace {
index += 1
} else if let simple = simpleToken(character) {
tokens.append(simple)
index += 1
} else if character == "/" {
let double = index + 1 < characters.count && characters[index + 1] == "/"
tokens.append(double ? .doubleSlash : .slash)
index += double ? 2 : 1
} else if character == ":" {
guard index + 1 < characters.count, characters[index + 1] == ":" else { return nil }
tokens.append(.axis)
index += 2
} else if isNameStart(character) {
tokens.append(.name(readName(characters, &index)))
} else {
return nil
}
}
return tokens
}
/// The single-character tokens that need no lookahead.
private static func simpleToken(_ character: Character) -> XPathToken? {
switch character {
case "|": .pipe
case "@": .atSign
case ".": .dot
case "*": .star
default: nil
}
}
/// Reads a name test starting at `index`: a name run, optionally glued (no
/// whitespace) to `:` and a second name run or `*`, leaving a following `::` for
/// the lexer to read as an axis.
private static func readName(_ characters: [Character], _ index: inout Int) -> String {
var name = takeNameRun(characters, &index)
if index + 1 < characters.count, characters[index] == ":", characters[index + 1] != ":" {
index += 1
if characters[index] == "*" {
name += ":*"
index += 1
} else {
name += ":" + takeNameRun(characters, &index)
}
}
return name
}
private static func takeNameRun(_ characters: [Character], _ index: inout Int) -> String {
var run = ""
while index < characters.count, isNameContinuation(characters[index]) {
run.append(characters[index])
index += 1
}
return run
}
/// A name-test character set matching XML 1.0 `NameChar` (minus `:`, which is
/// glue or an axis here), so the lexer admits exactly the names `Lexical.isNCName`
/// will validate, including non-ASCII NCName characters Swift's `isLetter` misses.
private static func isNameContinuation(_ character: Character) -> Bool {
character != ":" && character.unicodeScalars.allSatisfy(PureXML.Parsing.XMLCharacter.isNameChar)
}
/// A name-test start character: XML 1.0 `NameStartChar` (minus `:`); the first
/// scalar must start a name and any later scalars (a combining grapheme) be name
/// characters.
private static func isNameStart(_ character: Character) -> Bool {
guard character != ":", let first = character.unicodeScalars.first,
PureXML.Parsing.XMLCharacter.isNameStart(first)
else { return false }
return character.unicodeScalars.dropFirst().allSatisfy(PureXML.Parsing.XMLCharacter.isNameChar)
}
/// An optional leading `.//`, then `/`-separated steps; `//` is rejected except
/// as that leading prefix, and an empty step (a leading/trailing/doubled `/`)
/// fails.
private static func validPath(_ tokens: [XPathToken], isField: Bool) -> Bool {
guard !tokens.isEmpty else { return false }
var body = tokens[...]
if body.first == .dot, body.dropFirst().first == .doubleSlash {
body = body.dropFirst(2)
guard !body.isEmpty else { return false }
}
var steps: [[XPathToken]] = [[]]
for token in body {
switch token {
case .slash: steps.append([])
case .doubleSlash: return false
default: steps[steps.count - 1].append(token)
}
}
return steps.enumerated().allSatisfy { index, step in
validStep(step, isField: isField, isLast: index == steps.count - 1)
}
}
/// A single step: `.`, a name test, a `child::` axis step, or (only as a field's
/// last step) an `@`/`attribute::` attribute step.
private static func validStep(_ step: [XPathToken], isField: Bool, isLast: Bool) -> Bool {
switch step.first {
case .dot where step.count == 1:
true
case .atSign:
isField && isLast && nameTest(Array(step.dropFirst()))
case .name("child") where step.count > 1 && step[1] == .axis:
nameTest(Array(step.dropFirst(2)))
case .name("attribute") where step.count > 1 && step[1] == .axis:
isField && isLast && nameTest(Array(step.dropFirst(2)))
default:
nameTest(step)
}
}
/// A single name-test token: `*`, a QName, or `NCName ':' '*'`.
private static func nameTest(_ tokens: [XPathToken]) -> Bool {
guard tokens.count == 1 else { return false }
switch tokens[0] {
case .star:
return true
case let .name(value):
return validNameTestName(value)
default:
return false
}
}
private static func validNameTestName(_ value: String) -> Bool {
let parts = value.split(separator: ":", omittingEmptySubsequences: false)
switch parts.count {
case 1:
return PureXML.Schema.Lexical.isNCName(value)
case 2:
let localPart = String(parts[1])
return PureXML.Schema.Lexical.isNCName(String(parts[0]))
&& (localPart == "*" || PureXML.Schema.Lexical.isNCName(localPart))
default:
return false
}
}
}

View File

@ -151,6 +151,9 @@ extension PureXML.Schema.XSDParser {
if local == "complexContent" {
errors += complexContentOrderErrors(node)
}
if let local, local == "selector" || local == "field" {
errors += identityXPathErrors(node, local: local)
}
}
for child in children {
collectStructure(child, into: &errors)

View File

@ -0,0 +1,71 @@
@testable import PureXML
import Testing
/// The XSD 1.0 restricted-XPath subset (Part 1, 3.11.6) for an identity
/// constraint's `selector` and `field`. A `selector` is a path of abbreviated
/// steps (`.`, `.//`, name tests, the `child::` axis); a `field` may additionally
/// end in an attribute step (`@x` or `attribute::x`). Whitespace between tokens is
/// allowed, but `//` must be adjacent and a name test's `:` may not be spaced; no
/// predicates, no absolute paths, no other axes. These pin both the rejections and
/// the valid shapes the W3C suite expects accepted (idI/idJ).
@Suite("Identity-constraint XPath subset")
struct SchemaIdentityXPathTests {
private func compile(_ selector: String, fields: [String] = ["@v"]) throws {
let fieldElements = fields.map { #"<xs:field xpath="\#($0)"/>"# }.joined()
_ = try PureXML.Schema.Document(#"""
<xs:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="root" type="xs:string"/>
<xs:element name="r">
<xs:complexType><xs:sequence><xs:element ref="root"/></xs:sequence></xs:complexType>
<xs:unique name="u">
<xs:selector xpath="\#(selector)"/>
\#(fieldElements)
</xs:unique>
</xs:element>
</xs:schema>
"""#)
}
private func rejectsSelector(_ selector: String) -> Bool {
do { try compile(selector)
return false
} catch { return true }
}
private func rejectsField(_ field: String) -> Bool {
do { try compile("root", fields: [field])
return false
} catch { return true }
}
@Test("valid selector paths compile")
func test_validSelectors() throws {
for selector in [".", ".//a", "a/b/c", "a|b", ".//a | .//b", "*", "ns:a", "ns:*", "child::a", "child::ns:*", ". //."] {
try compile(selector)
}
// A name test built from non-ASCII NCName characters (U+203F UNDERTIE) is a
// valid name and must not be rejected by the path lexer.
try compile("a\u{203F}b")
}
@Test("valid field paths compile, including attribute steps")
func test_validFields() throws {
for field in ["@v", "a/@v", "attribute::v", ".//a/@v", "@ns:v", "attribute::ns:v", "a | @v", "attribute :: v"] {
try compile("root", fields: [field])
}
}
@Test("malformed selector paths are rejected")
func test_invalidSelectors() {
for selector in ["", "|", "| a", "a|", "/", "//", "/a", ".//", ".//.//a", "a//b", "self::a", "descendant::a", "a[1]", "child::", "tid : *", "@v", "attribute::v"] {
#expect(rejectsSelector(selector), "expected rejection: '\(selector)'")
}
}
@Test("malformed field paths are rejected")
func test_invalidFields() {
for field in ["", "a[1]", "@ ", "@*[1]", "/@v", "self::v", "a/@v/b", "tid : *"] {
#expect(rejectsField(field), "expected rejection: '\(field)'")
}
}
}

View File

@ -160,8 +160,8 @@ struct XSDIdentityTests {
#expect(try validate(xsd, dangling).contains { $0.reason.contains("no matching key 'pk'") })
}
@Test("A malformed field XPath is a located error, not a silently disabled constraint")
func test_malformedFieldReported() throws {
@Test("A malformed field XPath is rejected at compile time, not silently disabled")
func test_malformedFieldReported() {
let xsd = """
\(head)
<xs:element name="list">
@ -172,12 +172,11 @@ struct XSDIdentityTests {
</xs:element>
</xs:schema>
"""
let errors = try validate(xsd, "<list><item id=\"1\"/><item id=\"1\"/></list>")
#expect(errors.contains { $0.reason.contains("invalid field XPath '@id['") }, "\(errors.map(\.reason))")
#expect(rejectsIdentityXPath(xsd), "a field xpath with a predicate bracket is not in the XSD subset")
}
@Test("A malformed selector XPath is a located error")
func test_malformedSelectorReported() throws {
@Test("A malformed selector XPath is rejected at compile time")
func test_malformedSelectorReported() {
let xsd = """
\(head)
<xs:element name="list">
@ -188,8 +187,18 @@ struct XSDIdentityTests {
</xs:element>
</xs:schema>
"""
let errors = try validate(xsd, "<list><item id=\"1\"/></list>")
#expect(errors.contains { $0.reason.contains("invalid selector XPath 'item['") }, "\(errors.map(\.reason))")
#expect(rejectsIdentityXPath(xsd), "a selector xpath with a predicate bracket is not in the XSD subset")
}
/// Whether compiling `xsd` is rejected for an out-of-subset identity-constraint
/// XPath (the malformed path is caught at compile time, not silently disabled).
private func rejectsIdentityXPath(_ xsd: String) -> Bool {
do {
_ = try PureXML.Schema.Document(xsd)
return false
} catch {
return "\(error)".contains("is not a valid identity-constraint path")
}
}
@Test("A child-element field locates the error at that child")

View File

@ -29,7 +29,7 @@ struct XSTSSuiteTests {
/// XSTS` (debug is far slower and the corpus is large). Per-case deviations
/// are written to /tmp/xsts-failures.txt for the burn-down.
private let knownSchemaValidRejected = 72
private let knownSchemaInvalidAccepted = 785
private let knownSchemaInvalidAccepted = 736
private let knownInstanceValidRejected = 214
private let knownInstanceInvalidAccepted = 165

View File

@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open.
| Bucket | Count | Reading |
|---|---|---|
| valid-schemas-rejected | 72 | we rarely reject a good schema |
| **invalid-schemas-accepted** | **2461 -> 785** | we rarely catch a bad one (facet, id, structural content-model, UPA determinism, and complexContent-derivation order validity landed) |
| **invalid-schemas-accepted** | **2461 -> 736** | we rarely catch a bad one (facet, id, structural content-model, UPA determinism, complexContent-derivation order, and identity-constraint XPath-subset validity landed) |
| valid-instances-rejected | 233 | (instance side, separately tracked) |
| invalid-instances-accepted | 165 | |