From 6147666e49b85dbaeb43032c9b367d06c495aa5b Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Sat, 13 Jun 2026 22:48:20 +0200 Subject: [PATCH] feat(schema): structural validity against the schema-for-schemas content model The general mechanism for the structural tail of the schema-validity gap, in place of hand-coding a rule per component. A data-driven table encodes the XSD 1.0 schema-for-schemas child content model; at compile time it rejects any component child the model does not admit, more than one annotation (or an annotation that is not first, except on schema/redefine, which allow several), and an identity constraint missing its selector or field. The walk visits only XSD-namespace elements and skips appinfo/documentation foreign content, so foreign children are never flagged; restriction/extension use the permissive union of their simple/complex/simpleContent contexts, so a child valid in some context is never rejected. One table covers every component. XSTS: invalid schemas accepted 1688 to 1313 (-375); valid schemas rejected held at 72 (no over-rejection); instance buckets unchanged. --- CHANGELOG.md | 1 + Sources/Schema/SchemaStructure.swift | 93 ++++++++++++++++++++++++++++ Sources/Schema/XSDParser.swift | 2 +- Tests/SchemaStructureTests.swift | 87 ++++++++++++++++++++++++++ Tests/XSTSSuiteTests.swift | 2 +- docs/schema-validity-burndown.md | 16 +++-- docs/xsts-deviations.md | 18 +++--- 7 files changed, 203 insertions(+), 16 deletions(-) create mode 100644 Sources/Schema/SchemaStructure.swift create mode 100644 Tests/SchemaStructureTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 73ec800..e67da9f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- Schema documents are now structurally validated against the schema-for-schemas content model (#147), the general mechanism for the structural tail of the schema-validity gap. A data-driven table of the XSD 1.0 child content model rejects, at compile time, any component child the model does not admit, more than one `annotation` (or an `annotation` that is not first, except on `schema`/`redefine`), and an identity constraint missing its selector or field. The check walks only XSD-namespace elements and skips `xs:appinfo`/`xs:documentation` foreign content, so no valid schema is rejected; `restriction`/`extension` use the permissive union of their contexts (never over-rejecting). One table covers every component rather than a rule hand-coded per element. XSTS: invalid schemas accepted 1688 to 1313 (-375), no other bucket raised. - The `id` attribute on XSD components is now validated at compile time (#147). It is of type `xs:ID`, so each value must be a valid NCName and unique within the schema document; the values were never checked, so a malformed (`id=""`, `id="123"`) or duplicated `id` left the schema wrongly accepted. The schema-document tree is walked, skipping `xs:appinfo`/`xs:documentation` foreign content (whose `id` attributes are not `xs:ID`) and treating only the unprefixed `id` (not a foreign `xml:id`/`pre:id`) as the component identifier, so no valid schema is rejected. XSTS: invalid schemas accepted 1789 to 1688, no other bucket raised. - Value-bound facet definitions are now validated at compile time (#147), continuing the facet-validity work. Per XSD Part 2 Datatypes 4.3, each `minInclusive`/`maxInclusive`/`minExclusive`/`maxExclusive` and `enumeration` value must be a valid value of the base type; `minInclusive` excludes `minExclusive` and `maxInclusive` excludes `maxExclusive`; and the lower bound may not exceed the upper (nor equal it when either side is exclusive, an empty value space), compared in the base primitive's value space. XSTS: invalid schemas accepted 2082 to 1789, no other bucket raised. - `xs:gMonth` now accepts the XSD 1.0 lexical form `--MM--` (the trailing `--` is optional, the later `--MM` erratum form still accepted), with an optional timezone (#147). Only `--MM` was recognized, so valid `gMonth` values and the schemas whose facets used them were wrongly rejected. Surfaced by the value-bound facet check above (which validates a bound value against its base type). XSTS: valid instances rejected 233 to 214; it also removed 26 false schema rejections the bound-facet check would otherwise have caused. diff --git a/Sources/Schema/SchemaStructure.swift b/Sources/Schema/SchemaStructure.swift new file mode 100644 index 0000000..546de5a --- /dev/null +++ b/Sources/Schema/SchemaStructure.swift @@ -0,0 +1,93 @@ +extension PureXML.Schema.XSDParser { + /// The XML Schema namespace; only elements in it are schema vocabulary, so + /// foreign elements (and annotation content) are not structurally checked. + private static let xsdNamespace = "http://www.w3.org/2001/XMLSchema" + + /// The schema-for-schemas child content model (XSD 1.0 Structures): the + /// XSD-namespace element local names each component admits as children. The + /// `restriction`/`extension` entries are the permissive union across their + /// simple/complex/simpleContent contexts, so this never rejects a child that + /// is valid in some context; order beyond the leading annotation is not + /// modelled here. Leaf components and facets admit only `annotation`. + private static let allowedChildren: [String: Set] = { + let facets: Set = [ + "minExclusive", "minInclusive", "maxExclusive", "maxInclusive", "totalDigits", + "fractionDigits", "length", "minLength", "maxLength", "enumeration", "whiteSpace", "pattern", + ] + var table: [String: Set] = [ + "schema": ["include", "import", "redefine", "annotation", "simpleType", "complexType", "group", "attributeGroup", "element", "attribute", "notation"], + "complexType": ["annotation", "simpleContent", "complexContent", "group", "all", "choice", "sequence", "attribute", "attributeGroup", "anyAttribute"], + "simpleContent": ["annotation", "restriction", "extension"], + "complexContent": ["annotation", "restriction", "extension"], + "simpleType": ["annotation", "restriction", "list", "union"], + "element": ["annotation", "simpleType", "complexType", "unique", "key", "keyref"], + "attribute": ["annotation", "simpleType"], + "attributeGroup": ["annotation", "attribute", "attributeGroup", "anyAttribute"], + "group": ["annotation", "all", "choice", "sequence"], + "all": ["annotation", "element"], + "choice": ["annotation", "element", "group", "choice", "sequence", "any"], + "sequence": ["annotation", "element", "group", "choice", "sequence", "any"], + "restriction": Set(["annotation", "simpleType", "group", "all", "choice", "sequence", "attribute", "attributeGroup", "anyAttribute"]).union(facets), + "extension": ["annotation", "group", "all", "choice", "sequence", "attribute", "attributeGroup", "anyAttribute"], + "unique": ["annotation", "selector", "field"], + "key": ["annotation", "selector", "field"], + "keyref": ["annotation", "selector", "field"], + "annotation": ["appinfo", "documentation"], + "list": ["annotation", "simpleType"], + "union": ["annotation", "simpleType"], + "redefine": ["annotation", "simpleType", "complexType", "group", "attributeGroup"], + ] + for leaf in ["selector", "field", "any", "anyAttribute", "notation", "import", "include"] + Array(facets) { + table[leaf] = ["annotation"] + } + return table + }() + + /// Components whose annotation may be repeated and interspersed (XSD 1.0: + /// `schema` and `redefine`); everywhere else `annotation` is `annotation?` + /// and must be the first child. + private static let multipleAnnotation: Set = ["schema", "redefine"] + + /// Schema-validity findings for the structural content of a schema document: + /// each XSD-namespace element's XSD-namespace children must be admitted by the + /// schema-for-schemas content model, the single `annotation` (where allowed + /// once) must be first, and an identity constraint needs a selector and field. + /// Walks the one document rooted at `schema`, skipping foreign content. + static func structureErrors(_ schema: XSDTree) -> [String] { + var errors: [String] = [] + collectStructure(schema, into: &errors) + return errors + } + + private static func collectStructure(_ node: XSDTree, into errors: inout [String]) { + let local = PureXML.Schema.XSDNode.localName(node) + if local == "appinfo" || local == "documentation" { return } + let children = PureXML.Schema.XSDNode.elementChildren(node) + if node.name?.namespaceURI == xsdNamespace, let local, let allowed = allowedChildren[local] { + let names = children.filter { $0.name?.namespaceURI == xsdNamespace }.compactMap(PureXML.Schema.XSDNode.localName) + errors += childErrors(local: local, children: names, allowed: allowed) + } + for child in children { + collectStructure(child, into: &errors) + } + } + + private static func childErrors(local: String, children: [String], allowed: Set) -> [String] { + var errors: [String] = [] + for name in children where !allowed.contains(name) { + errors.append("element '\(name)' is not allowed in '\(local)'") + } + if !multipleAnnotation.contains(local) { + let annotations = children.count(where: { $0 == "annotation" }) + if annotations > 1 { + errors.append("'\(local)' may have at most one annotation") + } else if annotations == 1, children.first != "annotation" { + errors.append("the annotation in '\(local)' must be the first child") + } + } + if ["unique", "key", "keyref"].contains(local), !(children.contains("selector") && children.contains("field")) { + errors.append("identity constraint '\(local)' requires a selector and at least one field") + } + return errors + } +} diff --git a/Sources/Schema/XSDParser.swift b/Sources/Schema/XSDParser.swift index 6d81931..271fd62 100644 --- a/Sources/Schema/XSDParser.swift +++ b/Sources/Schema/XSDParser.swift @@ -33,7 +33,7 @@ extension PureXML.Schema { context.attributeFormQualified = XSDNode.attribute(schema, "attributeFormDefault") == "qualified" context.complexTypeNodes = indexByName(allChildren(containers, named: "complexType")) context.globalAttributes = indexByName(allChildren(containers, named: "attribute")) - for error in idAttributeErrors(schema) { + for error in idAttributeErrors(schema) + structureErrors(schema) { context.diagnostics.report(error) } resolveSimpleTypes(allChildren(containers, named: "simpleType"), into: &context) diff --git a/Tests/SchemaStructureTests.swift b/Tests/SchemaStructureTests.swift new file mode 100644 index 0000000..470a18f --- /dev/null +++ b/Tests/SchemaStructureTests.swift @@ -0,0 +1,87 @@ +@testable import PureXML +import Testing + +/// Structural validity of a schema document against the schema-for-schemas +/// content model (XSD 1.0 Structures): each component's children must be +/// admitted by the model, an `annotation` (where allowed once) must be first, +/// and an identity constraint needs a selector and field. Such a schema is +/// invalid and must be rejected at compile time; the children were previously +/// unchecked and the schema accepted (XSTS invalid-schema ctB/ctG/ctH, etc.). +@Suite("Schema structural validity") +struct SchemaStructureTests { + private func compile(_ body: String) throws { + _ = try PureXML.Schema.Document(""" + + \(body) + + """) + } + + private func rejects(_ body: String) -> Bool { + do { try compile(body) + return false + } catch { return true } + } + + @Test("well-formed components compile") + func test_valid() throws { + try compile(#""" + + doc + + + + """#) + try compile(#""" + + + + + + """#) + } + + @Test("a child not admitted by the content model is rejected") + func test_disallowedChild() { + // element is not a direct child of complexType (it must sit in a group). + #expect(rejects(#""#)) + // attribute admits only annotation and simpleType, not sequence. + #expect(rejects(#""#)) + } + + @Test("at most one annotation, and it must be first") + func test_annotationPlacement() { + #expect(rejects(#""" + + one + two + + """#)) + #expect(rejects(#""" + + + late + + """#)) + } + + @Test("schema and redefine may carry several annotations") + func test_multipleAnnotationOnSchema() throws { + try compile(#""" + one + + two + """#) + } + + @Test("an identity constraint requires a selector and field") + func test_identityConstraintRequiresParts() { + #expect(rejects(#""" + + + + + + """#)) + } +} diff --git a/Tests/XSTSSuiteTests.swift b/Tests/XSTSSuiteTests.swift index aae904e..d33c184 100644 --- a/Tests/XSTSSuiteTests.swift +++ b/Tests/XSTSSuiteTests.swift @@ -29,7 +29,7 @@ struct XSTSSuiteTests { /// XSTS` (debug is far slower and the corpus is large). Per-case deviations /// are written to /tmp/xsts-failures.txt for the burn-down. private let knownSchemaValidRejected = 72 - private let knownSchemaInvalidAccepted = 1688 + private let knownSchemaInvalidAccepted = 1313 private let knownInstanceValidRejected = 214 private let knownInstanceInvalidAccepted = 165 diff --git a/docs/schema-validity-burndown.md b/docs/schema-validity-burndown.md index 7da2364..b7b2b4e 100644 --- a/docs/schema-validity-burndown.md +++ b/docs/schema-validity-burndown.md @@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open. | Bucket | Count | Reading | |---|---|---| | valid-schemas-rejected | 72 | we rarely reject a good schema | -| **invalid-schemas-accepted** | **2461 -> 1688** | we rarely catch a bad one (facet-definition validity landed) | +| **invalid-schemas-accepted** | **2461 -> 1313** | we rarely catch a bad one (facet, id, and structural content-model validity landed) | | valid-instances-rejected | 233 | (instance side, separately tracked) | | invalid-instances-accepted | 165 | | @@ -69,11 +69,15 @@ Each iteration targets one self-contained rule family with a clean root cause. 5. **Facet applicability per base** (the open facet-family tail): which facets a base admits (`fractionDigits` only on decimal-derived, `length` only on string/binary/list, etc.), and a fixed facet may not be changed. -6. **Particle / model-group structural rules** and **schema-for-schemas - structural validity** (child order, misplaced annotation, illegal content): - the largest, most tedious tail. Altitude decision: validate the schema - *document* against the normative schema-for-schemas (one general mechanism) - rather than hand-coding each rule. Defer until the above are done; re-measure. +6. **Schema-for-schemas structural validity.** _Content-model pass done + (1688 -> 1313, -375)._ A data-driven `allowedChildren` table (the schema-for- + schemas child content model) rejects disallowed children, multi/misplaced + `annotation`, and identity constraints missing selector/field, via one general + mechanism (`SchemaStructure.swift`). **Still open:** the **attribute model** + (which attributes each component admits, and their value types) is the obvious + next sibling table and the largest remaining lever; child *order* beyond the + leading annotation; and particle / model-group semantic rules (Unique Particle + Attribution, Particle-Valid-Restriction completeness). ## Per-iteration protocol (the PR-critic-loop) diff --git a/docs/xsts-deviations.md b/docs/xsts-deviations.md index 5bc81da..ffa3be3 100644 --- a/docs/xsts-deviations.md +++ b/docs/xsts-deviations.md @@ -47,7 +47,7 @@ and to document the remainder as spec-justified, deliberate exclusions. ## Category A: genuine correctness bugs -### Fixed (seventeen root causes, ~1447 deviations cleared) +### Fixed (eighteen root causes, ~1822 deviations cleared) list-datatype length facets (#146) · union pattern/enumeration facets · duration partial-order facets · QName length non-constraining · the full XSD `\p{Is...}` @@ -65,7 +65,7 @@ validity (`min`/`maxInclusive`, `min`/`maxExclusive`, `enumeration` values valid in the base space; inclusive/exclusive exclusions; bound ordering) · `gMonth` `--MM--` lexical form · schema `id` attribute validity (`xs:ID`: NCName and unique within the document). The schema-validity work brought invalid schemas -accepted 2461 to 1688; see `schema-validity-burndown.md`. +accepted 2461 to 1313 (also: schema-document structural validity against the schema-for-schemas content model); see `schema-validity-burndown.md`. Plus the measurement fix (exclude W3C-disputed `status="queried"` entries). ### Remaining, root-caused (the hard tail) @@ -106,12 +106,14 @@ schema-for-schemas. Examples found: | `annotation/annotB` | ~62 | | `addB`, `stF`, and the long tail | remainder of 2467 | -**Decision:** enforcing this is a large, low-real-world-value effort (people do -not author structurally broken schemas, and a too-lenient *compiler* does not -corrupt valid-document validation). It would require a schema-document -validation pass against the schema-for-schemas. Treated as a **documented, -deliberate exclusion** unless a concrete need arises. If implemented, it is its -own epic, not part of the instance-correctness burn-down. +**Status (revised):** now being implemented as the general schema-for-schemas +mechanism, not a per-rule pile. The content-model pass has landed (`id` validity, +facet-definition validity, and the `allowedChildren` structural table), driving +invalid-schemas-accepted from 2461 to 1313. The remaining structural work (the +component **attribute model**, child order, particle semantics) continues under +`schema-validity-burndown.md`. A too-lenient compiler does not corrupt +valid-document validation, but rubber-stamping invalid schemas is the largest +conformance gap, so it is being closed rather than excluded. ## Category C: arguable / needs investigation