feat(schema): validate constraining-facet definitions at compile time
First cut at the schema-validity gap: the validator ran instance-validity rules but not schema-validity rules, so it silently accepted invalid schemas. Per XSD Part 2 Datatypes 4.3, a length-family facet value (length, minLength, maxLength, totalDigits, fractionDigits) must be a valid nonNegativeInteger (totalDigits a positiveInteger), length may not co-occur with minLength/maxLength, and the integer ranges must be ordered. Malformed values (length="", "-1", "a", "1e2") were silently dropped by numeric facet parsing and the schema accepted. The definition is now read raw from the restriction and validated at compile time. Well-formedness is checked independently of machine-integer range (via a canonical digit string), so a huge but valid value is not misreported as malformed, and ordering compares by length then lexically. Findings travel through a shared reference-type diagnostics collector on the value-type parsing context (surviving its copies) and join the model-level consistency findings thrown from Schema.Document.init. Limited to facets declared on the local restriction, so an inherited facet is never wrongly flagged. The min/maxInclusive value-space rules, facet applicability per base, and inherited co-occurrence remain for later iterations (see docs/schema-validity-burndown.md). XSTS: invalid schemas accepted 2461 to 2082, no change to any other bucket.
This commit is contained in:
parent
dc929774c3
commit
d3daea5b91
@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- Constraining-facet definitions are now validated at compile time (#147), the first cut at the schema-validity gap (invalid schemas the validator was silently accepting). Per XSD Part 2 Datatypes 4.3, each length-family facet value (`length`, `minLength`, `maxLength`, `totalDigits`, `fractionDigits`) must be a valid `nonNegativeInteger` (`totalDigits` a `positiveInteger`), `length` may not co-occur with `minLength`/`maxLength`, and the integer ranges must be ordered. Malformed values (`length=""`, `"-1"`, `"a"`, `"1e2"`) were dropped by numeric parsing and the schema accepted; the definition is now read raw and rejected at compile time, with well-formedness checked independently of machine-integer range so a huge but valid value is not misreported. The findings travel through a shared diagnostics collector on the parsing context and join the model-level consistency findings. XSTS: invalid schemas accepted 2461 to 2082, with no change to any other bucket.
|
||||
- A built-in datatype named by `xsi:type` (`xsd:int`, `xsd:decimal`, `xsd:string`) is now resolved, gated by substitution validity (#147). The override resolved only against schema-declared types, so a built-in `xsi:type` was reported "unknown" and the element wrongly rejected. The built-in is now resolved when it validly substitutes for the declared type per Schema-Validity Assessment: the ur-type (`anyType`/`anySimpleType`) admits any type, and a bare built-in declared type admits any built-in derived from it through the XSD Part 2 datatype lattice (new `BuiltinType.derives(from:)`). An invalid built-in substitution is a located error, not a silent acceptance, closing the gap the resolution exposed. XSTS: valid instances rejected 248 to 233 (with no regression in invalid instances accepted, which the two fixes below preserve).
|
||||
- A simple-typed element now rejects non-schema-instance attributes (#147). A simple type declares no attribute uses, so an element whose type is simple, including one whose simple type comes from an `xsi:type` override and including an `xsi:nil` element, may carry only the `xsi:*` and namespace attributes; any other attribute is a violation. The check was absent, so resolving a built-in `xsi:type` to a simple type would otherwise have accepted stray attributes on it.
|
||||
- Identity-constraint fields compare in their value space when the selected node carries an `xsi:type` (#147). `unique`/`key`/`keyref` tuples compared lexically, so two `xsi:type="xsd:decimal"` fields `3.0` and `3`, the same value, were treated as distinct and a `unique` violation was missed. Each field value now carries the simple type read from the node's `xsi:type`; two values are equal in that type's value space when both share a built-in base (reusing `SimpleType.valueMatches`), otherwise by lexical form.
|
||||
|
||||
@ -120,14 +120,17 @@ public extension PureXML.Schema {
|
||||
targetNamespace = compiled.targetNamespace
|
||||
// Schema consistency through the validation framework: every named
|
||||
// type is checked by the composable rules (final respected, Particle
|
||||
// Valid (Restriction)), and ALL findings are reported together.
|
||||
// Valid (Restriction)), and ALL findings are reported together, joined
|
||||
// with the schema-validity findings gathered while parsing (malformed
|
||||
// facet definitions and the like).
|
||||
let findings = PureXML.Validation.XSDSchema.consistencyErrors(
|
||||
types: compiled.types,
|
||||
typeDerivation: compiled.typeDerivation,
|
||||
typeFinal: compiled.typeFinal,
|
||||
)
|
||||
if !findings.isEmpty {
|
||||
throw SchemaError.inconsistent(findings.map { String(describing: $0) })
|
||||
let allFindings = compiled.schemaErrors + findings.map { String(describing: $0) }
|
||||
if !allFindings.isEmpty {
|
||||
throw SchemaError.inconsistent(allFindings)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -124,6 +124,30 @@ extension PureXML.Schema {
|
||||
/// The schema's target namespace, so the root element's namespace can be
|
||||
/// checked against it.
|
||||
var targetNamespace: String?
|
||||
/// Schema-validity findings gathered while parsing (malformed facet
|
||||
/// definitions and the like), reported together with the model-level
|
||||
/// consistency findings when the schema is compiled.
|
||||
var schemaErrors: [String] = []
|
||||
}
|
||||
|
||||
/// Collects schema-validity findings during parsing. A reference type so the
|
||||
/// findings survive the value-type ``XSDContext``'s copies (`visiting(_:)`):
|
||||
/// every copy shares the one collector, and the parser harvests it at the end.
|
||||
final class SchemaDiagnostics {
|
||||
private(set) var errors: [String] = []
|
||||
|
||||
func report(_ message: String) {
|
||||
errors.append(message)
|
||||
}
|
||||
|
||||
/// The findings with exact-duplicate messages removed, first occurrence
|
||||
/// kept. The same malformed definition is visited more than once (named
|
||||
/// simple types resolve over repeated passes), so dedup keeps one finding
|
||||
/// per distinct problem.
|
||||
var deduplicated: [String] {
|
||||
var seen: Set<String> = []
|
||||
return errors.filter { seen.insert($0).inserted }
|
||||
}
|
||||
}
|
||||
|
||||
/// The parsing context: the named simple types resolved so far, plus the
|
||||
@ -159,6 +183,9 @@ extension PureXML.Schema {
|
||||
/// Named complex types being resolved up the current `complexContent`
|
||||
/// derivation chain, a guard against a cyclic base reference.
|
||||
var visitingTypes: Set<String> = []
|
||||
/// The shared schema-validity finding collector. A reference type, so the
|
||||
/// value-type context's copies all report into the one place.
|
||||
let diagnostics = SchemaDiagnostics()
|
||||
|
||||
func visiting(_ group: String) -> XSDContext {
|
||||
var copy = self
|
||||
|
||||
@ -40,17 +40,7 @@ extension PureXML.Schema {
|
||||
types[name] = .complex(complexType(node, context))
|
||||
}
|
||||
}
|
||||
var elements: [String: ElementType] = [:]
|
||||
for node in allChildren(containers, named: "element") where XSDNode.attribute(node, "name") != nil {
|
||||
let name = XSDNode.attribute(node, "name") ?? ""
|
||||
let type = elementType(node, context)
|
||||
elements[name] = type
|
||||
// An `xs:element ref="name"` resolves to this declaration's type
|
||||
// through a reserved key (a colon keeps it out of the NCName-only
|
||||
// type namespace), letting forward and recursive refs resolve at
|
||||
// validation time.
|
||||
types[elementKey(name)] = type
|
||||
}
|
||||
let elements = globalElements(containers, context, into: &types)
|
||||
let (nillable, elementConstraints) = elementMetadata(containers)
|
||||
return XSDCompiled(
|
||||
elements: elements,
|
||||
@ -65,9 +55,29 @@ extension PureXML.Schema {
|
||||
typeDerivation: derivation.typeDerivation,
|
||||
typeFinal: derivation.typeFinal,
|
||||
targetNamespace: context.targetNamespace,
|
||||
schemaErrors: context.diagnostics.deduplicated,
|
||||
)
|
||||
}
|
||||
|
||||
/// The global element declarations, also registering each under a reserved
|
||||
/// `element:name` key in `types` so an `xs:element ref="name"` resolves to
|
||||
/// this declaration's type (a colon keeps it out of the NCName-only type
|
||||
/// namespace), letting forward and recursive refs resolve at validation time.
|
||||
private static func globalElements(
|
||||
_ containers: [XSDTree],
|
||||
_ context: XSDContext,
|
||||
into types: inout [String: ElementType],
|
||||
) -> [String: ElementType] {
|
||||
var elements: [String: ElementType] = [:]
|
||||
for node in allChildren(containers, named: "element") where XSDNode.attribute(node, "name") != nil {
|
||||
let name = XSDNode.attribute(node, "name") ?? ""
|
||||
let type = elementType(node, context)
|
||||
elements[name] = type
|
||||
types[elementKey(name)] = type
|
||||
}
|
||||
return elements
|
||||
}
|
||||
|
||||
private static func allChildren(_ containers: [XSDTree], named name: String) -> [XSDTree] {
|
||||
containers.flatMap { XSDNode.children($0, named: name) }
|
||||
}
|
||||
|
||||
@ -27,9 +27,70 @@ extension PureXML.Schema {
|
||||
facets.enumeration = nil
|
||||
}
|
||||
applyFacets(restriction, into: &facets)
|
||||
for error in facetDefinitionErrors(restriction) {
|
||||
context.diagnostics.report(error)
|
||||
}
|
||||
return SimpleType(base: baseType.base, facets: facets, variety: baseType.variety)
|
||||
}
|
||||
|
||||
/// Schema-validity findings for the constraining facets declared on one
|
||||
/// `restriction` (XSD Part 2 Datatypes 4.3): each length-family facet value
|
||||
/// must be a valid `nonNegativeInteger` (`totalDigits` a `positiveInteger`),
|
||||
/// `length` may not co-occur with `minLength`/`maxLength`, and the integer
|
||||
/// ranges must be ordered. Reads the raw lexical from the restriction, since
|
||||
/// numeric facet parsing silently drops a malformed value. Limited to the
|
||||
/// facets declared on this step, so an inherited facet is never flagged here.
|
||||
static func facetDefinitionErrors(_ restriction: XSDTree) -> [String] {
|
||||
var errors: [String] = []
|
||||
var values: [String: String] = [:] // facet name -> canonical digit string
|
||||
let countFacets: Set = ["length", "minLength", "maxLength", "totalDigits", "fractionDigits"]
|
||||
for facet in XSDNode.elementChildren(restriction) {
|
||||
guard let name = XSDNode.localName(facet), countFacets.contains(name) else { continue }
|
||||
let raw = XSDNode.attribute(facet, "value")
|
||||
guard let canonical = raw.flatMap(canonicalNonNegativeInteger) else {
|
||||
errors.append("facet '\(name)' value '\(raw ?? "")' is not a valid nonNegativeInteger")
|
||||
continue
|
||||
}
|
||||
if name == "totalDigits", canonical == "0" {
|
||||
errors.append("facet 'totalDigits' value '\(raw ?? "")' must be a positiveInteger")
|
||||
}
|
||||
values[name] = canonical
|
||||
}
|
||||
if values["length"] != nil, values["minLength"] != nil {
|
||||
errors.append("facets 'length' and 'minLength' may not both be specified")
|
||||
}
|
||||
if values["length"] != nil, values["maxLength"] != nil {
|
||||
errors.append("facets 'length' and 'maxLength' may not both be specified")
|
||||
}
|
||||
if let low = values["minLength"], let high = values["maxLength"], greater(low, than: high) {
|
||||
errors.append("facet 'minLength' (\(low)) exceeds 'maxLength' (\(high))")
|
||||
}
|
||||
if let fraction = values["fractionDigits"], let total = values["totalDigits"], greater(fraction, than: total) {
|
||||
errors.append("facet 'fractionDigits' (\(fraction)) exceeds 'totalDigits' (\(total))")
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
/// The canonical digit string of a lexical `nonNegativeInteger` (optional
|
||||
/// leading `+`, ASCII digits, surrounding whitespace tolerated), with the
|
||||
/// sign and leading zeros stripped (`"0"` for zero), or nil when the lexical
|
||||
/// is not one (`-1`, ``, `a`, `1e2`). Independent of machine-integer range,
|
||||
/// so a huge but well-formed value is canonicalized, not misreported as
|
||||
/// malformed; the canonical form orders two values by length then lexically.
|
||||
private static func canonicalNonNegativeInteger(_ lexical: String) -> String? {
|
||||
var digits = Substring(lexical.trimmingXMLWhitespace())
|
||||
if digits.first == "+" { digits = digits.dropFirst() }
|
||||
guard !digits.isEmpty, digits.allSatisfy({ $0.isASCII && $0.isNumber }) else { return nil }
|
||||
let trimmed = digits.drop { $0 == "0" }
|
||||
return trimmed.isEmpty ? "0" : String(trimmed)
|
||||
}
|
||||
|
||||
/// Whether canonical nonNegativeInteger `lhs` is greater than `rhs`: a longer
|
||||
/// digit string is larger, and same-length strings compare lexicographically.
|
||||
private static func greater(_ lhs: String, than rhs: String) -> Bool {
|
||||
lhs.count != rhs.count ? lhs.count > rhs.count : lhs > rhs
|
||||
}
|
||||
|
||||
static func simpleTypeReference(_ typeName: String, _ context: XSDContext) -> SimpleType {
|
||||
let local = XSDNode.stripPrefix(typeName)
|
||||
if let builtin = BuiltinType(rawValue: local) { return SimpleType(base: builtin) }
|
||||
|
||||
83
Tests/SchemaFacetDefinitionTests.swift
Normal file
83
Tests/SchemaFacetDefinitionTests.swift
Normal file
@ -0,0 +1,83 @@
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
/// Schema-validity of constraining-facet definitions (XSD Part 2 Datatypes 4.3):
|
||||
/// a length-family facet value must be a valid `nonNegativeInteger`
|
||||
/// (`totalDigits` a `positiveInteger`), `length` may not co-occur with
|
||||
/// `minLength`/`maxLength`, and the integer ranges must be ordered. Such a schema
|
||||
/// is invalid and must be rejected at compile time; previously the malformed
|
||||
/// value was silently dropped and the schema accepted (XSTS invalid-schema
|
||||
/// MS DataTypes facet cases).
|
||||
@Suite("Facet definition validity")
|
||||
struct SchemaFacetDefinitionTests {
|
||||
private func compile(_ facets: String, base: String = "xs:string") throws {
|
||||
_ = try PureXML.Schema.Document("""
|
||||
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
|
||||
<xs:element name="v">
|
||||
<xs:simpleType>
|
||||
<xs:restriction base="\(base)">\(facets)</xs:restriction>
|
||||
</xs:simpleType>
|
||||
</xs:element>
|
||||
</xs:schema>
|
||||
""")
|
||||
}
|
||||
|
||||
private func rejects(_ facets: String, base: String = "xs:string") -> Bool {
|
||||
do { try compile(facets, base: base)
|
||||
return false
|
||||
} catch { return true }
|
||||
}
|
||||
|
||||
@Test("a well-formed facet definition compiles")
|
||||
func test_valid() throws {
|
||||
try compile(#"<xs:length value="5"/>"#)
|
||||
try compile(#"<xs:minLength value="1"/><xs:maxLength value="10"/>"#)
|
||||
try compile(#"<xs:length value="0"/>"#)
|
||||
try compile(#"<xs:totalDigits value="3"/><xs:fractionDigits value="2"/>"#, base: "xs:decimal")
|
||||
try compile(#"<xs:minLength value="2"/><xs:maxLength value="2"/>"#)
|
||||
// A well-formed nonNegativeInteger is accepted regardless of machine-integer
|
||||
// range, and tolerates a leading "+" and leading zeros.
|
||||
try compile(#"<xs:length value="+5"/>"#)
|
||||
try compile(#"<xs:length value="007"/>"#)
|
||||
try compile(#"<xs:length value="999999999999999999999999999999"/>"#)
|
||||
try compile(#"<xs:minLength value="2"/><xs:maxLength value="99999999999999999999"/>"#)
|
||||
}
|
||||
|
||||
@Test("a malformed length-family lexical is rejected")
|
||||
func test_malformedLexical() {
|
||||
#expect(rejects(#"<xs:length value=""/>"#))
|
||||
#expect(rejects(#"<xs:length value="-1"/>"#))
|
||||
#expect(rejects(#"<xs:length value="a"/>"#))
|
||||
#expect(rejects(#"<xs:length value="1e2"/>"#))
|
||||
#expect(rejects(#"<xs:minLength value="x"/>"#))
|
||||
#expect(rejects(#"<xs:maxLength value="-5"/>"#))
|
||||
}
|
||||
|
||||
@Test("totalDigits must be a positiveInteger")
|
||||
func test_totalDigitsPositive() {
|
||||
#expect(rejects(#"<xs:totalDigits value="0"/>"#, base: "xs:decimal"))
|
||||
#expect(rejects(#"<xs:totalDigits value="-1"/>"#, base: "xs:decimal"))
|
||||
}
|
||||
|
||||
@Test("length may not co-occur with minLength or maxLength")
|
||||
func test_lengthCoOccurrence() {
|
||||
#expect(rejects(#"<xs:length value="5"/><xs:minLength value="1"/>"#))
|
||||
#expect(rejects(#"<xs:length value="5"/><xs:maxLength value="10"/>"#))
|
||||
}
|
||||
|
||||
@Test("integer facet ranges must be ordered")
|
||||
func test_rangeOrder() {
|
||||
#expect(rejects(#"<xs:minLength value="10"/><xs:maxLength value="1"/>"#))
|
||||
#expect(rejects(#"<xs:totalDigits value="2"/><xs:fractionDigits value="5"/>"#, base: "xs:decimal"))
|
||||
}
|
||||
|
||||
@Test("the rejection message names the offending facet")
|
||||
func test_locatedMessage() {
|
||||
do {
|
||||
try compile(#"<xs:length value="a"/>"#)
|
||||
Issue.record("expected the malformed facet to be rejected")
|
||||
} catch {
|
||||
#expect(String(describing: error).contains("length"))
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -29,7 +29,7 @@ struct XSTSSuiteTests {
|
||||
/// XSTS` (debug is far slower and the corpus is large). Per-case deviations
|
||||
/// are written to /tmp/xsts-failures.txt for the burn-down.
|
||||
private let knownSchemaValidRejected = 72
|
||||
private let knownSchemaInvalidAccepted = 2461
|
||||
private let knownSchemaInvalidAccepted = 2082
|
||||
private let knownInstanceValidRejected = 233
|
||||
private let knownInstanceInvalidAccepted = 165
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open.
|
||||
| Bucket | Count | Reading |
|
||||
|---|---|---|
|
||||
| valid-schemas-rejected | 72 | we rarely reject a good schema |
|
||||
| **invalid-schemas-accepted** | **2461** | we rarely catch a bad one |
|
||||
| **invalid-schemas-accepted** | **2461 -> 2082** | we rarely catch a bad one (facet-definition validity landed) |
|
||||
| valid-instances-rejected | 233 | (instance side, separately tracked) |
|
||||
| invalid-instances-accepted | 165 | |
|
||||
|
||||
@ -38,16 +38,17 @@ existing pattern: collect ALL findings, report them together.
|
||||
|
||||
Each iteration targets one self-contained rule family with a clean root cause.
|
||||
|
||||
1. **Facet-definition validity** (XSD Part 2 §4.3). Each constraining facet's
|
||||
`value` must be a valid instance of the facet's own type (`length`,
|
||||
`minLength`, `maxLength`, `totalDigits`, `fractionDigits` are
|
||||
`nonNegativeInteger`; `positiveInteger` for `totalDigits`; `minInclusive`
|
||||
etc. must be valid in the base value space); facet co-occurrence rules
|
||||
(`length` excludes `minLength`/`maxLength`; `minInclusive` excludes
|
||||
`minExclusive`; `maxInclusive` excludes `maxExclusive`; `minInclusive` <=
|
||||
`maxInclusive`; `minLength` <= `maxLength`; `fractionDigits` <=
|
||||
`totalDigits`); `whiteSpace` may not weaken an inherited value. Big chunk of
|
||||
DataTypes (703) and SimpleType (139).
|
||||
1. **Facet-definition validity** (XSD Part 2 §4.3). _Partly done (2461 -> 2082)._
|
||||
Landed: length-family facet values must be `nonNegativeInteger`
|
||||
(`totalDigits` a `positiveInteger`); `length` excludes
|
||||
`minLength`/`maxLength`; `minLength` <= `maxLength`; `fractionDigits` <=
|
||||
`totalDigits`. **Still open in this family:** `minInclusive`/`maxInclusive`/
|
||||
`minExclusive`/`maxExclusive` value well-formedness in the base value space
|
||||
and their mutual exclusions/ordering; facet applicability per base
|
||||
(`fractionDigits` only on decimal-derived, `length` only on
|
||||
string/binary/list, etc.); `whiteSpace` may not weaken an inherited value;
|
||||
facet repetition. Inherited-facet co-occurrence (across derivation steps) is
|
||||
deliberately not checked yet (current pass reads only the local restriction).
|
||||
2. **Resolvable references.** Every QName a schema names (`type`, element /
|
||||
attribute / group `ref`, restriction / extension `base`, `itemType`,
|
||||
`memberTypes`, `substitutionGroup`, keyref `refer`) must resolve to a
|
||||
|
||||
@ -47,7 +47,7 @@ and to document the remainder as spec-justified, deliberate exclusions.
|
||||
|
||||
## Category A: genuine correctness bugs
|
||||
|
||||
### Fixed (thirteen root causes, ~655 deviations cleared)
|
||||
### Fixed (fourteen root causes, ~1034 deviations cleared)
|
||||
|
||||
list-datatype length facets (#146) · union pattern/enumeration facets · duration
|
||||
partial-order facets · QName length non-constraining · the full XSD `\p{Is...}`
|
||||
@ -58,7 +58,11 @@ xs:ID/xs:IDREF uniqueness and resolution · `<xs:attribute ref>` resolution ·
|
||||
mixed-content type with no element model · built-in `xsi:type` resolution gated
|
||||
by substitution validity (the XSD Part 2 derivation lattice), which also closed
|
||||
the two gaps it exposed (a simple-typed element rejecting stray attributes;
|
||||
identity fields comparing in value space when the node carries an `xsi:type`).
|
||||
identity fields comparing in value space when the node carries an `xsi:type`) ·
|
||||
compile-time constraining-facet definition validity (length-family facets are
|
||||
`nonNegativeInteger`, co-occurrence and range order), the first cut at the
|
||||
schema-validity gap (invalid schemas accepted 2461 to 2082; see
|
||||
`schema-validity-burndown.md`).
|
||||
Plus the measurement fix (exclude W3C-disputed `status="queried"` entries).
|
||||
|
||||
### Remaining, root-caused (the hard tail)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user