feat(schema): enforce Unique Particle Attribution (cos-nonambig)

Reject ambiguous content models at compile time via a Glushkov position
automaton over a literal, QName-resolved view of the raw schema tree, built
just for this rule and independent of the instance-validation content model.

The literal view resolves a ref's QName through the schema's prefix bindings
(not the blanket target namespace) and does not expand substitution groups,
removing the two false-overlap sources a check over the compiled model
produced. Determinism is per particle: a particle competing with a repetition
of itself is never a violation, so a fixed count {n,n} unrolls into copies in
sequence while a true range or unbounded repeat uses a self-loop, and only two
distinct source particles with overlapping labels conflict. The overlap test
is QName-only, an under-rejection that can never over-reject.

A position cap bounds pathological nesting (skipping the check, never
over-rejecting). The element-declarations-consistent test loses its
incidentally-ambiguous (a?, a) case in favor of a deterministic (a, b, a).

XSTS: invalid schemas accepted 884 to 864, valid schemas rejected held at 72,
instance buckets unchanged.
This commit is contained in:
Mihaela Mihaljevic 2026-06-14 09:35:33 +02:00
parent 42a87b132c
commit 90ae16cba1
7 changed files with 453 additions and 5 deletions

View File

@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Unique Particle Attribution (`cos-nonambig`, the content-model determinism constraint) is now enforced at compile time (#156, #157), the keystone of the remaining schema-validity tail. A content model must be deterministic so each element in an instance is attributable to a single particle without lookahead; an ambiguous model (`<choice>` of two same-name elements, an optional or repeating element followed by a distinct same-name element, two same-name members of an `all`, an element competing with a wildcard that admits it) was accepted. The check runs a Glushkov position automaton over a literal, QName-resolved view of the raw schema tree built specifically for this rule, independent of the instance-validation content model: a `ref` is one position carrying its prefix-resolved QName (not the blanket target namespace), and substitution groups are not expanded, which removes the two false-overlap sources (cross-namespace refs, expanded substitution members) that a check over the compiled model produced. Determinism is per particle, the W3C rule's defining property: a particle competing with a repetition of itself is never a violation, so a fixed count `{n,n}` unrolls into copies in sequence (`b{2,2}, b` stays the deterministic `b, b, b`) while a true range or unbounded repeat uses a self-loop, and only two distinct source particles with overlapping labels conflict. The overlap test is QName-only (not substitution-group-aware), an accepted under-rejection that can never over-reject. XSTS: invalid schemas accepted 884 to 864, no other bucket raised (valid schemas rejected held at 72).
- A `keyref`'s `refer` must name a `key` or `unique` constraint (#147). A `refer` pointing at a non-existent (or non-key/unique) constraint was accepted; it is now resolved against the document's `key`/`unique` names, with a forward `refer` (textually before the key) handled by collecting all names first. The check is skipped when the document pulls in external definitions through `import`/`include`/`redefine`, which the default compile does not load. XSTS: invalid schemas accepted 891 to 884, no other bucket raised.
- Component-name uniqueness is now enforced (#147). Within a schema, global type names (`simpleType` and `complexType` share one symbol space), global element names, global attribute names, named model-group names, and named attribute-group names must each be unique, and identity-constraint names must be unique across the document; a duplicate was accepted, with the later definition silently overwriting the earlier. Only the document's own globals are examined (an `xs:redefine`'s same-named type is not a clash), and the symbol spaces are kept separate (a type and an element may share a name). XSTS: invalid schemas accepted 909 to 891, no other bucket raised.
- Element Declarations Consistent (`cos-element-consistent`) is now enforced (#147). Within one complex type's content model, two element declarations with the same name must have the same type; two same-name elements with different types (e.g. `a` as `xs:string` and as `xs:integer`) were accepted. The check walks the content model (through model groups and content-derivation wrappers) but not into an element's own nested type, so a reused name in a nested anonymous type, a separate content model, is not flagged. Modeled on Xerces (libxml2 only partially enforces this); substitution-group expansion is not folded in (an accepted under-rejection). XSTS: invalid schemas accepted 925 to 909, no other bucket raised.

View File

@ -0,0 +1,330 @@
public extension PureXML.Schema {
/// Unique Particle Attribution (cos-nonambig, XSD 1.0 3.8.6): a content model
/// must be deterministic, so each element in an instance is attributed to one
/// particle without lookahead. Checked on the Glushkov position automaton: the
/// model is ambiguous iff some decision set, the start `firstpos` or any
/// `followpos(p)`, holds two distinct positions whose labels can match the same
/// element. No DFA subset construction is needed.
///
/// This works on a LITERAL view of the raw schema tree, distinct from the
/// compiled (instance) content model: a `ref` is one position carrying its
/// resolved QName (prefix bindings, not the blanket target namespace), and
/// substitution groups are NOT expanded. The overlap test is QName-only (not
/// substitution-group-aware), so it can only under-report substitution-group
/// ambiguities, never over-report. Those two choices remove the false overlaps
/// (cross-namespace refs; expanded substitution members) that a check over the
/// compiled model produced.
enum ContentModelDeterminism {
private static let modelGroupNames: Set<String> = ["sequence", "choice", "all"]
private static let particleNames: Set<String> = ["element", "sequence", "choice", "all", "group", "any"]
/// Every UPA-violation message across the schema's complex-type content
/// models, deduplicated in first-seen order.
static func violations(in schema: XSDTree, context: XSDContext) -> [String] {
let bindings = namespaceBindings(schema)
var messages: [String] = []
var seen: Set<String> = []
forEachComplexType(schema) { complexType in
guard let model = contentModelNode(complexType),
let conflict = violation(model, bindings, context)
else { return }
if seen.insert(conflict).inserted { messages.append(conflict) }
}
return messages
}
/// The prefix-to-namespace bindings declared on the schema element (its
/// `xmlns:prefix` attributes; the default namespace under the empty key).
private static func namespaceBindings(_ schema: XSDTree) -> [String: String] {
var bindings: [String: String] = [:]
for attribute in schema.attributes {
if attribute.name.prefix == "xmlns" {
bindings[attribute.name.localName] = attribute.value
} else if attribute.name.prefix == nil, attribute.name.localName == "xmlns" {
bindings[""] = attribute.value
}
}
return bindings
}
/// Visits every `complexType` element in the schema document, skipping
/// foreign annotation content.
private static func forEachComplexType(_ node: XSDTree, _ visit: (XSDTree) -> Void) {
let local = PureXML.Schema.XSDNode.localName(node)
if local == "appinfo" || local == "documentation" { return }
if local == "complexType" { visit(node) }
for child in PureXML.Schema.XSDNode.elementChildren(node) {
forEachComplexType(child, visit)
}
}
/// The model-group node (`sequence`/`choice`/`all`/`group`) that holds a
/// complex type's content model, reached through `complexContent` and its
/// `restriction`/`extension`, or nil for simple or empty content.
private static func contentModelNode(_ complexType: XSDTree) -> XSDTree? {
if let direct = modelGroupChild(complexType) { return direct }
guard let complexContent = PureXML.Schema.XSDNode.firstChild(complexType, named: "complexContent") else { return nil }
for derivation in PureXML.Schema.XSDNode.elementChildren(complexContent) {
if let model = modelGroupChild(derivation) { return model }
}
return nil
}
private static func modelGroupChild(_ node: XSDTree) -> XSDTree? {
PureXML.Schema.XSDNode.elementChildren(node).first {
["sequence", "choice", "all", "group"].contains(PureXML.Schema.XSDNode.localName($0) ?? "")
}
}
/// A UPA-violation message for the content model rooted at `model`, or nil.
private static func violation(_ model: XSDTree, _ bindings: [String: String], _ context: XSDContext) -> String? {
var automaton = PureXML.Schema.PositionAutomaton()
let root = build(model, &automaton, bindings, context, [])
// A model that hit the position cap was only partially built; skip it
// rather than risk a result from an incomplete automaton.
if automaton.labels.count > positionCap { return nil }
var decisionSets = automaton.followpos
decisionSets.append(root.first)
for set in decisionSets {
if let conflict = overlap(in: set, automaton: automaton) {
return "ambiguous content model: '\(conflict)' can be matched by more than one particle"
}
}
return nil
}
/// A ceiling on positions per content model. A pathological schema is bounded
/// here and its UPA check skipped (an under-rejection, never an
/// over-rejection). Real content models are far below this.
private static let positionCap = 4096
private static func build(
_ node: XSDTree,
_ automaton: inout PureXML.Schema.PositionAutomaton,
_ bindings: [String: String],
_ context: XSDContext,
_ visiting: Set<String>,
) -> PureXML.Schema.Positions {
if automaton.labels.count > positionCap { return .empty }
let (minimum, maximum) = PureXML.Schema.XSDNode.occurrence(node)
// A `maxOccurs="0"` particle can never occur, and a negative maxOccurs is
// malformed (flagged by occurrence-order validation): either contributes
// nothing here, and this guards the copy count below against a bad bound.
if let maximum, maximum <= 0 { return .empty }
// A fixed count {n,n} is n copies in sequence with no repetition loop, so
// `b{2,2}, b` stays the deterministic `b, b, b`. The exact count above one
// does not change the determinism question, so it is clamped to two.
if let maximum, minimum == maximum {
let copies = (0 ..< Swift.min(maximum, 2)).map { _ in buildInner(node, &automaton, bindings, context, visiting) }
return sequence(copies, &automaton)
}
// Here min < max. Build one copy. A self-loop (last -> first) is added
// only when the particle can truly repeat (max unbounded or above one);
// unrolling a repetition into separate copies would put one element in two
// positions of a decision set and read as ambiguity, when a repetition of
// a deterministic body is itself deterministic. A {0,1} is merely optional,
// not a repetition, so it gets no loop. The floor sets nullability.
let inner = buildInner(node, &automaton, bindings, context, visiting)
if maximum == nil || (maximum ?? 0) > 1 {
for position in inner.last {
automaton.followpos[position].formUnion(inner.first)
}
}
return PureXML.Schema.Positions(nullable: minimum == 0 || inner.nullable, first: inner.first, last: inner.last)
}
private static func buildInner(
_ node: XSDTree,
_ automaton: inout PureXML.Schema.PositionAutomaton,
_ bindings: [String: String],
_ context: XSDContext,
_ visiting: Set<String>,
) -> PureXML.Schema.Positions {
switch PureXML.Schema.XSDNode.localName(node) {
case "element":
let position = automaton.position(.name(elementQName(node, bindings, context)), node)
return PureXML.Schema.Positions(nullable: false, first: [position], last: [position])
case "any":
let position = automaton.position(.wildcard(PureXML.Schema.XSDParser.wildcard(node, context)), node)
return PureXML.Schema.Positions(nullable: false, first: [position], last: [position])
case "sequence":
return sequence(childSets(node, &automaton, bindings, context, visiting), &automaton)
case "choice":
return choice(childSets(node, &automaton, bindings, context, visiting))
case "all":
return all(childSets(node, &automaton, bindings, context, visiting), &automaton)
case "group":
return groupReference(node, &automaton, bindings, context, visiting)
default:
return .empty
}
}
private static func childSets(
_ node: XSDTree,
_ automaton: inout PureXML.Schema.PositionAutomaton,
_ bindings: [String: String],
_ context: XSDContext,
_ visiting: Set<String>,
) -> [PureXML.Schema.Positions] {
PureXML.Schema.XSDNode.elementChildren(node)
.filter { particleNames.contains(PureXML.Schema.XSDNode.localName($0) ?? "") }
.map { build($0, &automaton, bindings, context, visiting) }
}
/// Inlines a `group` reference's model group (its sole `all`/`choice`/
/// `sequence`), guarding against a cyclic reference.
private static func groupReference(
_ node: XSDTree,
_ automaton: inout PureXML.Schema.PositionAutomaton,
_ bindings: [String: String],
_ context: XSDContext,
_ visiting: Set<String>,
) -> PureXML.Schema.Positions {
guard let ref = PureXML.Schema.XSDNode.attribute(node, "ref") else { return .empty }
let name = PureXML.Schema.XSDNode.stripPrefix(ref)
guard !visiting.contains(name), let definition = context.groups[name],
let model = PureXML.Schema.XSDNode.elementChildren(definition).first(where: { modelGroupNames.contains(PureXML.Schema.XSDNode.localName($0) ?? "") })
else { return .empty }
return build(model, &automaton, bindings, context, visiting.union([name]))
}
private static func elementQName(_ node: XSDTree, _ bindings: [String: String], _ context: XSDContext) -> PureXML.Model.QualifiedName {
if let ref = PureXML.Schema.XSDNode.attribute(node, "ref") {
return resolveQName(ref, bindings, context)
}
let name = PureXML.Schema.XSDNode.attribute(node, "name") ?? ""
let form = PureXML.Schema.XSDNode.attribute(node, "form")
let qualified = form == "qualified" || (form == nil && context.elementFormQualified)
return PureXML.Model.QualifiedName(localName: name, namespaceURI: qualified ? context.targetNamespace : nil)
}
/// Resolves a QName reference to (localName, namespaceURI): a prefix maps
/// through the schema's bindings (or stands for itself if unbound, keeping
/// distinct prefixes distinct); an unprefixed name takes the default
/// namespace binding.
private static func resolveQName(_ qname: String, _ bindings: [String: String], _: XSDContext) -> PureXML.Model.QualifiedName {
let parts = qname.split(separator: ":", maxSplits: 1).map(String.init)
if parts.count == 2 {
return PureXML.Model.QualifiedName(localName: parts[1], namespaceURI: bindings[parts[0]] ?? parts[0])
}
return PureXML.Model.QualifiedName(localName: qname, namespaceURI: bindings[""])
}
private static func sequence(_ children: [PureXML.Schema.Positions], _ automaton: inout PureXML.Schema.PositionAutomaton) -> PureXML.Schema.Positions {
var first: Set<Int> = []
var prefixNullable = true
var preceding: Set<Int> = []
for child in children {
if prefixNullable { first.formUnion(child.first) }
for position in preceding {
automaton.followpos[position].formUnion(child.first)
}
preceding = child.nullable ? preceding.union(child.last) : child.last
if !child.nullable { prefixNullable = false }
}
var last: Set<Int> = []
var suffixNullable = true
for child in children.reversed() {
if suffixNullable { last.formUnion(child.last) }
if !child.nullable { suffixNullable = false }
}
return PureXML.Schema.Positions(nullable: children.allSatisfy(\.nullable), first: first, last: last)
}
private static func choice(_ children: [PureXML.Schema.Positions]) -> PureXML.Schema.Positions {
PureXML.Schema.Positions(
nullable: children.isEmpty || children.contains(where: \.nullable),
first: children.reduce(into: Set<Int>()) { $0.formUnion($1.first) },
last: children.reduce(into: Set<Int>()) { $0.formUnion($1.last) },
)
}
/// `all`: members are order-independent, so each may follow every other and
/// the members form one decision set; two same-named members then overlap.
private static func all(_ children: [PureXML.Schema.Positions], _ automaton: inout PureXML.Schema.PositionAutomaton) -> PureXML.Schema.Positions {
for source in children {
for target in children where target.first != source.first {
for position in source.last {
automaton.followpos[position].formUnion(target.first)
}
}
}
return choice(children)
}
private static func overlap(in set: Set<Int>, automaton: PureXML.Schema.PositionAutomaton) -> String? {
let positions = set.sorted()
for outer in 0 ..< positions.count {
let outerPosition = positions[outer]
for inner in (outer + 1) ..< positions.count {
let innerPosition = positions[inner]
// Two positions conflict only if they are distinct particles whose
// labels can match the same element; one particle's own repetition
// (same source) is deterministic by construction.
guard automaton.particles[outerPosition] != automaton.particles[innerPosition],
labelsOverlap(automaton.labels[outerPosition], automaton.labels[innerPosition])
else { continue }
return describe(automaton.labels[outerPosition])
}
}
return nil
}
private static func labelsOverlap(_ lhs: TermLabel, _ rhs: TermLabel) -> Bool {
switch (lhs, rhs) {
case let (.name(lhsName), .name(rhsName)):
lhsName.localName == rhsName.localName && lhsName.namespaceURI == rhsName.namespaceURI
case let (.name(name), .wildcard(wildcard)), let (.wildcard(wildcard), .name(name)):
wildcard.admits(name)
case let (.wildcard(lhsCard), .wildcard(rhsCard)):
wildcardsOverlap(lhsCard, rhsCard)
}
}
private static func wildcardsOverlap(_ lhs: Wildcard, _ rhs: Wildcard) -> Bool {
switch (lhs.namespace, rhs.namespace) {
case (.any, _), (_, .any):
true
case let (.enumerated(lhsSet), .enumerated(rhsSet)):
!lhsSet.isDisjoint(with: rhsSet)
default:
false
}
}
private static func describe(_ label: TermLabel) -> String {
switch label {
case let .name(name): name.localName
case .wildcard: "any"
}
}
}
/// The firstpos/lastpos/nullable summary of a sub-model, used while building the
/// position automaton.
private struct Positions {
let nullable: Bool
let first: Set<Int>
let last: Set<Int>
static let empty = Positions(nullable: true, first: [], last: [])
}
/// The Glushkov position automaton under construction: one entry per position
/// for its transition label, its `followpos` set, and the source particle it
/// came from (Unique Particle Attribution is per particle, so a particle never
/// conflicts with a repetition of itself).
private struct PositionAutomaton {
var labels: [TermLabel] = []
var followpos: [Set<Int>] = []
var particles: [ObjectIdentifier] = []
mutating func position(_ label: TermLabel, _ source: XSDTree) -> Int {
labels.append(label)
followpos.append([])
particles.append(ObjectIdentifier(source))
return labels.count - 1
}
}
}

View File

@ -33,7 +33,9 @@ extension PureXML.Schema {
context.attributeFormQualified = XSDNode.attribute(schema, "attributeFormDefault") == "qualified"
context.complexTypeNodes = indexByName(allChildren(containers, named: "complexType"))
context.globalAttributes = indexByName(allChildren(containers, named: "attribute"))
for error in idAttributeErrors(schema) + structureErrors(schema) + componentNameErrors(schema) {
let consistencyErrors = idAttributeErrors(schema) + structureErrors(schema)
+ componentNameErrors(schema) + ContentModelDeterminism.violations(in: schema, context: context)
for error in consistencyErrors {
context.diagnostics.report(error)
}
var types = namedTypes(containers, into: &context)

View File

@ -0,0 +1,114 @@
@testable import PureXML
import Testing
/// Unique Particle Attribution (cos-nonambig, XSD 1.0 3.8.6): a content model
/// must be deterministic, so each element in an instance is attributable to a
/// single particle without lookahead. The check runs on a literal, QName-resolved
/// view of the raw schema tree (refs are not substitution-group expanded, the
/// overlap test is QName-only), and is per particle: a particle competing with a
/// repetition of itself is never a violation. These cases pin both the rejections
/// and the deterministic shapes that must keep compiling (XSTS valid mgZ005,
/// ctZ008, particlesHa142, groupH001).
@Suite("Content model determinism (UPA)")
struct ContentModelDeterminismTests {
private func compile(_ body: String) throws {
_ = try PureXML.Schema.Document("""
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
\(body)
</xs:schema>
""")
}
private func rejects(_ body: String) -> Bool {
do { try compile(body)
return false
} catch { return true }
}
private func complexType(_ model: String) -> String {
#"<xs:complexType name="t">\#(model)</xs:complexType>"#
}
// MARK: - Ambiguous models are rejected
@Test("a choice of two same-name elements is ambiguous")
func test_choiceOfSameName() {
#expect(rejects(complexType(#"<xs:choice><xs:element name="a"/><xs:element name="a"/></xs:choice>"#)))
}
@Test("an optional element followed by a same-name element is ambiguous")
func test_optionalThenSameName() {
#expect(rejects(complexType(#"<xs:sequence><xs:element name="a" minOccurs="0"/><xs:element name="a"/></xs:sequence>"#)))
}
@Test("a repeating element followed by a same-name element is ambiguous")
func test_repeatThenSameName() {
#expect(rejects(complexType(#"<xs:sequence><xs:element name="a" maxOccurs="unbounded"/><xs:element name="a"/></xs:sequence>"#)))
}
@Test("two same-name members of an all group are ambiguous")
func test_allWithSameName() {
#expect(rejects(complexType(#"<xs:all><xs:element name="a"/><xs:element name="a"/></xs:all>"#)))
}
@Test("an element competing with a wildcard that admits it is ambiguous")
func test_elementAndWildcard() {
#expect(rejects(complexType(#"<xs:choice><xs:element name="a"/><xs:any processContents="skip"/></xs:choice>"#)))
}
// MARK: - Deterministic models keep compiling
@Test("distinct elements in sequence and choice are deterministic")
func test_distinctNames() throws {
try compile(complexType(#"<xs:sequence><xs:element name="a"/><xs:element name="b" minOccurs="0"/></xs:sequence>"#))
try compile(complexType(#"<xs:choice><xs:element name="a"/><xs:element name="b"/></xs:choice>"#))
try compile(complexType(#"<xs:all><xs:element name="a"/><xs:element name="b"/><xs:element name="c"/></xs:all>"#))
}
@Test("a fixed-count element followed by a distinct same-name element is deterministic (mgZ005)")
func test_fixedCountThenSameName() throws {
try compile(complexType(#"""
<xs:sequence>
<xs:element name="a" minOccurs="0"/>
<xs:element name="b" minOccurs="2" maxOccurs="2"/>
<xs:element name="b"/>
</xs:sequence>
"""#))
}
@Test("a fixed-count group wrapping a repeating element is deterministic (ctZ008)")
func test_fixedCountGroupWithRepeat() throws {
try compile(complexType(#"""
<xs:sequence minOccurs="2" maxOccurs="2">
<xs:element name="a" minOccurs="1" maxOccurs="2"/>
<xs:element name="b" minOccurs="0"/>
</xs:sequence>
"""#))
}
@Test("an element followed by its own optional repeat is deterministic (particlesHa142)")
func test_elementThenOptionalSameName() throws {
try compile(complexType(#"""
<xs:choice>
<xs:sequence minOccurs="0">
<xs:element name="a" type="xs:string"/>
<xs:element name="a" type="xs:string" minOccurs="0"/>
</xs:sequence>
</xs:choice>
"""#))
}
@Test("a bounded-repeat choice of group references is deterministic (groupH001)")
func test_repeatedChoiceOfGroups() throws {
try compile(#"""
<xs:complexType name="A">
<xs:choice minOccurs="0" maxOccurs="4">
<xs:group ref="x"/>
<xs:group ref="y"/>
</xs:choice>
</xs:complexType>
<xs:group name="x"><xs:sequence><xs:element name="x1"/><xs:element name="x2"/></xs:sequence></xs:group>
<xs:group name="y"><xs:choice><xs:element name="y1"/><xs:element name="y2"/></xs:choice></xs:group>
"""#)
}
}

View File

@ -204,10 +204,11 @@ struct SchemaStructureTests {
<xs:element name="a" type="xs:integer"/>
</xs:sequence></xs:complexType>
"""#))
// Same name, same type is fine.
// Same name, same type is consistent (separated so it stays unambiguous).
try compile(#"""
<xs:complexType name="t"><xs:sequence>
<xs:element name="a" type="xs:string" minOccurs="0"/>
<xs:element name="a" type="xs:string"/>
<xs:element name="b" type="xs:string"/>
<xs:element name="a" type="xs:string"/>
</xs:sequence></xs:complexType>
"""#)

View File

@ -29,7 +29,7 @@ struct XSTSSuiteTests {
/// XSTS` (debug is far slower and the corpus is large). Per-case deviations
/// are written to /tmp/xsts-failures.txt for the burn-down.
private let knownSchemaValidRejected = 72
private let knownSchemaInvalidAccepted = 884
private let knownSchemaInvalidAccepted = 864
private let knownInstanceValidRejected = 214
private let knownInstanceInvalidAccepted = 165

View File

@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open.
| Bucket | Count | Reading |
|---|---|---|
| valid-schemas-rejected | 72 | we rarely reject a good schema |
| **invalid-schemas-accepted** | **2461 -> 884** | we rarely catch a bad one (facet, id, and structural content-model validity landed) |
| **invalid-schemas-accepted** | **2461 -> 864** | we rarely catch a bad one (facet, id, structural content-model, and UPA determinism validity landed) |
| valid-instances-rejected | 233 | (instance side, separately tracked) |
| invalid-instances-accepted | 165 | |