feat(schema): reject unresolvable references at compile time

Every QName a schema names must resolve to a declared component or a built-in:
type/base/itemType/memberTypes to a type, element/attribute/group/attributeGroup
ref to the matching global component, and element substitutionGroup to a global
element. An undeclared reference was previously accepted at compile time and only
surfaced lazily during instance validation, if at all.

References are matched by local name, with the value trimmed (a QName attribute
is whiteSpace=collapse). The whole check is skipped when the document declares an
import/include/redefine, since the default compile does not load externals and
the pools would be incomplete; that guard is gated on the XSD namespace so a
foreign element named "import" does not trip it.

XSTS: invalid schemas accepted 1092 to 1051 (-41); valid schemas rejected held
at 72 (an interim version regressed one valid schema on a whitespace-padded
base="  xs:string ", fixed by trimming); instance buckets unchanged.
This commit is contained in:
Mihaela Mihaljevic 2026-06-14 00:17:29 +02:00
parent 62e53b0fc1
commit cbeea0e3c2
8 changed files with 181 additions and 17 deletions

View File

@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Unresolvable references are now rejected at compile time (#147). Every QName a schema names (a `type`/`base`/`itemType`/`memberTypes` type, an `element`/`attribute`/`group`/`attributeGroup` `ref`, an element `substitutionGroup`) must resolve to a declared component or a built-in; an undeclared reference was previously accepted and only surfaced (if at all) lazily during instance validation. The reference is matched by local name with whitespace trimmed, as the rest of the compiler resolves it. The check is skipped when the document pulls in external definitions through `import`/`include`/`redefine`, which the default compile does not load, so a reference into them is never flagged. XSTS: invalid schemas accepted 1092 to 1051, no other bucket raised.
- `pattern` facet regular expressions are now validated at compile time (#147). Each `pattern` value is compiled when the schema is; a structurally malformed one (unbalanced parentheses `((a)`, a quantifier with nothing to repeat `?a`) is rejected, where before it was only compiled lazily at instance time (`try?`) and so silently ignored. The check is conservative: a construct the engine merely does not support on an otherwise-valid pattern (the empty pattern, an untabulated `\p{Is...}` block such as a surrogate or historic block, the lenient `{,m}` quantifier) is tolerated, so no valid schema is rejected. XSTS: invalid schemas accepted 1129 to 1092, no other bucket raised.
- Schema-vocabulary name and reference attribute values are now checked for lexical validity at compile time (#147). A component's `name` must be a valid NCName (`name=""`, `name="123"`, `name="a:b"` are rejected), and the reference attributes `type`/`base`/`ref`/`itemType`/`refer`/`substitutionGroup` must be lexical QNames (`type=":_"`, `type="a:b:c"` rejected), with `memberTypes` a list of QNames; these were unchecked and the schema accepted. Only the lexical form is checked (a QName's prefix is resolved elsewhere), so no valid schema is rejected. XSTS: invalid schemas accepted 1190 to 1129, no other bucket raised.
- Schema-vocabulary attribute values are now validated at compile time (#147), the attribute-model sibling of the structural content-model check. Enumerated attributes (`form`/`elementFormDefault`/`attributeFormDefault` to `qualified`/`unqualified`, `use` to `optional`/`prohibited`/`required`, `processContents` to `skip`/`lax`/`strict`, `mixed`/`abstract`/`nillable` to a boolean) and `minOccurs`/`maxOccurs` (a `nonNegativeInteger`, `maxOccurs` also `unbounded`) are checked against their fixed value space; an out-of-space value (`use="foo"`, `mixed="yes"`, `minOccurs="-1"`) was previously ignored and the schema accepted. Only unprefixed attributes on XSD-namespace elements are checked, so a foreign attribute is never flagged. XSTS: invalid schemas accepted 1313 to 1190, no other bucket raised.

View File

@ -0,0 +1,95 @@
extension PureXML.Schema.XSDParser {
/// The built-in type names a reference may name without a declaration: the
/// XSD Part 2 datatypes, the built-in list types, and the ur-types.
private static let referenceBuiltins: Set<String> = {
var names = Set(PureXML.Schema.BuiltinType.allCases.map(\.rawValue))
names.formUnion(["anyType", "anySimpleType", "anyAtomicType", "NOTATION", "IDREFS", "ENTITIES", "NMTOKENS"])
return names
}()
/// Schema-validity findings for unresolvable references: every QName a schema
/// names (a `type`/`base`/`itemType`/`memberTypes` type, an `element`/
/// `attribute`/`group`/`attributeGroup` `ref`, an element `substitutionGroup`)
/// must resolve to a declared component or a built-in. The reference is matched
/// by local name, as the rest of the compiler resolves it.
///
/// Skipped when the document pulls in external definitions through
/// `import`/`include`/`redefine`: the default compile does not load them, so
/// the pools would be incomplete and a reference into them must not be flagged.
static func referenceErrors(_ schema: XSDTree, in context: PureXML.Schema.XSDContext, elements: [String: PureXML.Schema.ElementType]) -> [String] {
if hasExternalReference(schema) { return [] }
let types = referenceBuiltins.union(context.simpleTypes.keys).union(context.complexTypeNodes.keys)
let pools: [String: Set<String>] = [
"element": Set(elements.keys),
"attribute": Set(context.globalAttributes.keys),
"group": Set(context.groups.keys),
"attributeGroup": Set(context.attributeGroups.keys),
]
var errors: [String] = []
collectReferenceErrors(schema, types: types, pools: pools, into: &errors)
return errors
}
/// Whether the document declares an `import`, `include`, or `redefine`, so its
/// component pools may be completed by an external document the default compile
/// does not load.
private static func hasExternalReference(_ schema: XSDTree) -> Bool {
var found = false
func walk(_ node: XSDTree) {
if node.name?.namespaceURI == xsdNamespace, let local = PureXML.Schema.XSDNode.localName(node),
["import", "include", "redefine"].contains(local) {
found = true
return
}
for child in PureXML.Schema.XSDNode.elementChildren(node) where !found {
walk(child)
}
}
walk(schema)
return found
}
private static func collectReferenceErrors(
_ node: XSDTree,
types: Set<String>,
pools: [String: Set<String>],
into errors: inout [String],
) {
let local = PureXML.Schema.XSDNode.localName(node)
if local == "appinfo" || local == "documentation" { return }
if node.name?.namespaceURI == xsdNamespace, let local {
errors += referenceErrors(at: node, local: local, types: types, pools: pools)
}
for child in PureXML.Schema.XSDNode.elementChildren(node) {
collectReferenceErrors(child, types: types, pools: pools, into: &errors)
}
}
private static func referenceErrors(at node: XSDTree, local: String, types: Set<String>, pools: [String: Set<String>]) -> [String] {
var errors: [String] = []
for attribute in ["type", "base", "itemType"] {
if let qname = PureXML.Schema.XSDNode.attribute(node, attribute), !types.contains(localName(qname)) {
errors.append("\(attribute) references undeclared type '\(qname)'")
}
}
if let members = PureXML.Schema.XSDNode.attribute(node, "memberTypes") {
for token in members.split(whereSeparator: \.isWhitespace) where !types.contains(localName(String(token))) {
errors.append("memberTypes references undeclared type '\(token)'")
}
}
if let head = PureXML.Schema.XSDNode.attribute(node, "substitutionGroup"), pools["element"]?.contains(localName(head)) != true {
errors.append("substitutionGroup references undeclared element '\(head)'")
}
if let reference = PureXML.Schema.XSDNode.attribute(node, "ref"), let pool = pools[local], !pool.contains(localName(reference)) {
errors.append("\(local) ref references undeclared '\(reference)'")
}
return errors
}
/// The local part of a QName reference, after the whitespace normalization a
/// `whiteSpace="collapse"` QName attribute receives (a value may be written
/// with surrounding or, harmlessly, interior whitespace).
private static func localName(_ qname: String) -> String {
PureXML.Schema.XSDNode.stripPrefix(qname.trimmingXMLWhitespace())
}
}

View File

@ -1,7 +1,7 @@
extension PureXML.Schema.XSDParser {
/// The XML Schema namespace; only elements in it are schema vocabulary, so
/// foreign elements (and annotation content) are not structurally checked.
private static let xsdNamespace = "http://www.w3.org/2001/XMLSchema"
static let xsdNamespace = "http://www.w3.org/2001/XMLSchema"
/// The schema-for-schemas child content model (XSD 1.0 Structures): the
/// XSD-namespace element local names each component admits as children. The

View File

@ -36,14 +36,11 @@ extension PureXML.Schema {
for error in idAttributeErrors(schema) + structureErrors(schema) {
context.diagnostics.report(error)
}
resolveSimpleTypes(allChildren(containers, named: "simpleType"), into: &context)
var types: [String: ElementType] = context.simpleTypes.mapValues(ElementType.simple)
for node in allChildren(containers, named: "complexType") {
if let name = XSDNode.attribute(node, "name") {
types[name] = .complex(complexType(node, context))
}
}
var types = namedTypes(containers, into: &context)
let elements = globalElements(containers, context, into: &types)
for error in referenceErrors(schema, in: context, elements: elements) {
context.diagnostics.report(error)
}
let (nillable, elementConstraints) = elementMetadata(containers)
return XSDCompiled(
elements: elements,
@ -62,6 +59,19 @@ extension PureXML.Schema {
)
}
/// The named-type table: the global simple types resolved into `context`,
/// then each global complex type compiled and added by name.
private static func namedTypes(_ containers: [XSDTree], into context: inout XSDContext) -> [String: ElementType] {
resolveSimpleTypes(allChildren(containers, named: "simpleType"), into: &context)
var types: [String: ElementType] = context.simpleTypes.mapValues(ElementType.simple)
for node in allChildren(containers, named: "complexType") {
if let name = XSDNode.attribute(node, "name") {
types[name] = .complex(complexType(node, context))
}
}
return types
}
/// The global element declarations, also registering each under a reserved
/// `element:name` key in `types` so an `xs:element ref="name"` resolves to
/// this declaration's type (a colon keeps it out of the NCName-only type

View File

@ -0,0 +1,56 @@
@testable import PureXML
import Testing
/// Every QName a schema names (a `type`/`base`/`itemType`/`memberTypes` type, an
/// element/attribute/group/attributeGroup `ref`, an element `substitutionGroup`)
/// must resolve to a declared component or a built-in. An undeclared reference
/// was previously accepted at compile time and only surfaced (if at all) lazily
/// during instance validation. The check is skipped when the document pulls in
/// external definitions through `import`/`include`/`redefine`, which the default
/// compile does not load.
@Suite("Schema reference resolution")
struct SchemaReferenceTests {
private func compile(_ body: String) throws {
_ = try PureXML.Schema.Document("""
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
\(body)
</xs:schema>
""")
}
private func rejects(_ body: String) -> Bool {
do { try compile(body)
return false
} catch { return true }
}
@Test("a reference to a declared component or a built-in resolves")
func test_resolves() throws {
try compile(#"<xs:element name="a" type="xs:string"/>"#)
try compile(#"<xs:simpleType name="t"><xs:restriction base="xs:integer"/></xs:simpleType><xs:element name="a" type="t"/>"#)
try compile(#"<xs:element name="g" type="xs:string"/><xs:complexType name="c"><xs:sequence><xs:element ref="g"/></xs:sequence></xs:complexType>"#)
}
@Test("an undeclared reference is rejected")
func test_undeclared() {
#expect(rejects(#"<xs:element name="a" type="DoesNotExist"/>"#))
#expect(rejects(#"<xs:simpleType name="t"><xs:restriction base="Missing"/></xs:simpleType>"#))
#expect(rejects(#"<xs:complexType name="c"><xs:sequence><xs:element ref="ghost"/></xs:sequence></xs:complexType>"#))
#expect(rejects(#"<xs:complexType name="c"><xs:group ref="noGroup"/></xs:complexType>"#))
}
@Test("a reference value tolerates surrounding whitespace (whiteSpace collapse)")
func test_whitespacePadded() throws {
try compile("<xs:simpleType name=\"t\"><xs:restriction base=\" xs:string \"/></xs:simpleType>")
}
@Test("references are not checked when an import may supply them")
func test_importSkipsCheck() throws {
// A dangling reference into an imported namespace is tolerated: the default
// compile does not load the import, so the reference cannot be verified.
try compile(#"""
<xs:import namespace="urn:other"/>
<xs:element name="a" type="other:Foo" xmlns:other="urn:other"/>
"""#)
}
}

View File

@ -29,7 +29,7 @@ struct XSTSSuiteTests {
/// XSTS` (debug is far slower and the corpus is large). Per-case deviations
/// are written to /tmp/xsts-failures.txt for the burn-down.
private let knownSchemaValidRejected = 72
private let knownSchemaInvalidAccepted = 1092
private let knownSchemaInvalidAccepted = 1051
private let knownInstanceValidRejected = 214
private let knownInstanceInvalidAccepted = 165

View File

@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open.
| Bucket | Count | Reading |
|---|---|---|
| valid-schemas-rejected | 72 | we rarely reject a good schema |
| **invalid-schemas-accepted** | **2461 -> 1092** | we rarely catch a bad one (facet, id, and structural content-model validity landed) |
| **invalid-schemas-accepted** | **2461 -> 1051** | we rarely catch a bad one (facet, id, and structural content-model validity landed) |
| valid-instances-rejected | 233 | (instance side, separately tracked) |
| invalid-instances-accepted | 165 | |
@ -61,11 +61,13 @@ Each iteration targets one self-contained rule family with a clean root cause.
type / element / attribute names unique per namespace; no duplicate attribute
uses on a type. Cheap graph checks (~12 identity-constraint-name cases, plus
type/element/attribute name collisions).
4. **Resolvable references.** Every QName a schema names (`type`, element /
attribute / group `ref`, restriction / extension `base`, `itemType`,
`memberTypes`, `substitutionGroup`, keyref `refer`) must resolve to a
declaration at compile time. Machinery exists (`resolveReference`); not run
exhaustively at compile time. The `xsd0NN.e` "undeclared X" family.
4. **Resolvable references.** _Done (1092 -> 1051)._ `type`/`base`/`itemType`/
`memberTypes` types, `element`/`attribute`/`group`/`attributeGroup` `ref`, and
element `substitutionGroup` must resolve to a declared component or a built-in
(`SchemaReferences.swift`), matched by local name, skipped when
`import`/`include`/`redefine` may supply externals. **Still open:** keyref
`refer` resolution; the context-sensitive applicability rules (`form` on a
global decl, `ref` excluding `name`/`type`).
5. **Facet applicability per base** (the open facet-family tail): which facets
a base admits (`fractionDigits` only on decimal-derived, `length` only on
string/binary/list, etc.), and a fixed facet may not be changed.

View File

@ -47,7 +47,7 @@ and to document the remainder as spec-justified, deliberate exclusions.
## Category A: genuine correctness bugs
### Fixed (twenty-one root causes, ~2043 deviations cleared)
### Fixed (twenty-two root causes, ~2084 deviations cleared)
list-datatype length facets (#146) · union pattern/enumeration facets · duration
partial-order facets · QName length non-constraining · the full XSD `\p{Is...}`
@ -65,7 +65,7 @@ validity (`min`/`maxInclusive`, `min`/`maxExclusive`, `enumeration` values valid
in the base space; inclusive/exclusive exclusions; bound ordering) · `gMonth`
`--MM--` lexical form · schema `id` attribute validity (`xs:ID`: NCName and
unique within the document). The schema-validity work brought invalid schemas
accepted 2461 to 1092 (also: schema-document structural validity, content model + attribute values + name/reference + pattern-regex lexical validity, against the schema-for-schemas); see `schema-validity-burndown.md`.
accepted 2461 to 1051 (schema-document structural validity: content model, attribute values, name/reference + pattern-regex lexical validity, and reference resolution, against the schema-for-schemas); see `schema-validity-burndown.md`.
Plus the measurement fix (exclude W3C-disputed `status="queried"` entries).
### Remaining, root-caused (the hard tail)