diff --git a/CHANGELOG.md b/CHANGELOG.md index c77806c..a9ff627 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- Constraining-facet definitions are now validated at compile time (#147), the first cut at the schema-validity gap (invalid schemas the validator was silently accepting). Per XSD Part 2 Datatypes 4.3, each length-family facet value (`length`, `minLength`, `maxLength`, `totalDigits`, `fractionDigits`) must be a valid `nonNegativeInteger` (`totalDigits` a `positiveInteger`), `length` may not co-occur with `minLength`/`maxLength`, and the integer ranges must be ordered. Malformed values (`length=""`, `"-1"`, `"a"`, `"1e2"`) were dropped by numeric parsing and the schema accepted; the definition is now read raw and rejected at compile time, with well-formedness checked independently of machine-integer range so a huge but valid value is not misreported. The findings travel through a shared diagnostics collector on the parsing context and join the model-level consistency findings. XSTS: invalid schemas accepted 2461 to 2082, with no change to any other bucket. - A built-in datatype named by `xsi:type` (`xsd:int`, `xsd:decimal`, `xsd:string`) is now resolved, gated by substitution validity (#147). The override resolved only against schema-declared types, so a built-in `xsi:type` was reported "unknown" and the element wrongly rejected. The built-in is now resolved when it validly substitutes for the declared type per Schema-Validity Assessment: the ur-type (`anyType`/`anySimpleType`) admits any type, and a bare built-in declared type admits any built-in derived from it through the XSD Part 2 datatype lattice (new `BuiltinType.derives(from:)`). An invalid built-in substitution is a located error, not a silent acceptance, closing the gap the resolution exposed. XSTS: valid instances rejected 248 to 233 (with no regression in invalid instances accepted, which the two fixes below preserve). - A simple-typed element now rejects non-schema-instance attributes (#147). A simple type declares no attribute uses, so an element whose type is simple, including one whose simple type comes from an `xsi:type` override and including an `xsi:nil` element, may carry only the `xsi:*` and namespace attributes; any other attribute is a violation. The check was absent, so resolving a built-in `xsi:type` to a simple type would otherwise have accepted stray attributes on it. - Identity-constraint fields compare in their value space when the selected node carries an `xsi:type` (#147). `unique`/`key`/`keyref` tuples compared lexically, so two `xsi:type="xsd:decimal"` fields `3.0` and `3`, the same value, were treated as distinct and a `unique` violation was missed. Each field value now carries the simple type read from the node's `xsi:type`; two values are equal in that type's value space when both share a built-in base (reusing `SimpleType.valueMatches`), otherwise by lexical form. diff --git a/Sources/Schema/SchemaDocument.swift b/Sources/Schema/SchemaDocument.swift index a193bb3..bf19cfc 100644 --- a/Sources/Schema/SchemaDocument.swift +++ b/Sources/Schema/SchemaDocument.swift @@ -120,14 +120,17 @@ public extension PureXML.Schema { targetNamespace = compiled.targetNamespace // Schema consistency through the validation framework: every named // type is checked by the composable rules (final respected, Particle - // Valid (Restriction)), and ALL findings are reported together. + // Valid (Restriction)), and ALL findings are reported together, joined + // with the schema-validity findings gathered while parsing (malformed + // facet definitions and the like). let findings = PureXML.Validation.XSDSchema.consistencyErrors( types: compiled.types, typeDerivation: compiled.typeDerivation, typeFinal: compiled.typeFinal, ) - if !findings.isEmpty { - throw SchemaError.inconsistent(findings.map { String(describing: $0) }) + let allFindings = compiled.schemaErrors + findings.map { String(describing: $0) } + if !allFindings.isEmpty { + throw SchemaError.inconsistent(allFindings) } } diff --git a/Sources/Schema/XSDContext.swift b/Sources/Schema/XSDContext.swift index 3caff96..dbfe8d3 100644 --- a/Sources/Schema/XSDContext.swift +++ b/Sources/Schema/XSDContext.swift @@ -124,6 +124,30 @@ extension PureXML.Schema { /// The schema's target namespace, so the root element's namespace can be /// checked against it. var targetNamespace: String? + /// Schema-validity findings gathered while parsing (malformed facet + /// definitions and the like), reported together with the model-level + /// consistency findings when the schema is compiled. + var schemaErrors: [String] = [] + } + + /// Collects schema-validity findings during parsing. A reference type so the + /// findings survive the value-type ``XSDContext``'s copies (`visiting(_:)`): + /// every copy shares the one collector, and the parser harvests it at the end. + final class SchemaDiagnostics { + private(set) var errors: [String] = [] + + func report(_ message: String) { + errors.append(message) + } + + /// The findings with exact-duplicate messages removed, first occurrence + /// kept. The same malformed definition is visited more than once (named + /// simple types resolve over repeated passes), so dedup keeps one finding + /// per distinct problem. + var deduplicated: [String] { + var seen: Set = [] + return errors.filter { seen.insert($0).inserted } + } } /// The parsing context: the named simple types resolved so far, plus the @@ -159,6 +183,9 @@ extension PureXML.Schema { /// Named complex types being resolved up the current `complexContent` /// derivation chain, a guard against a cyclic base reference. var visitingTypes: Set = [] + /// The shared schema-validity finding collector. A reference type, so the + /// value-type context's copies all report into the one place. + let diagnostics = SchemaDiagnostics() func visiting(_ group: String) -> XSDContext { var copy = self diff --git a/Sources/Schema/XSDParser.swift b/Sources/Schema/XSDParser.swift index f74f01c..41fabbe 100644 --- a/Sources/Schema/XSDParser.swift +++ b/Sources/Schema/XSDParser.swift @@ -40,17 +40,7 @@ extension PureXML.Schema { types[name] = .complex(complexType(node, context)) } } - var elements: [String: ElementType] = [:] - for node in allChildren(containers, named: "element") where XSDNode.attribute(node, "name") != nil { - let name = XSDNode.attribute(node, "name") ?? "" - let type = elementType(node, context) - elements[name] = type - // An `xs:element ref="name"` resolves to this declaration's type - // through a reserved key (a colon keeps it out of the NCName-only - // type namespace), letting forward and recursive refs resolve at - // validation time. - types[elementKey(name)] = type - } + let elements = globalElements(containers, context, into: &types) let (nillable, elementConstraints) = elementMetadata(containers) return XSDCompiled( elements: elements, @@ -65,9 +55,29 @@ extension PureXML.Schema { typeDerivation: derivation.typeDerivation, typeFinal: derivation.typeFinal, targetNamespace: context.targetNamespace, + schemaErrors: context.diagnostics.deduplicated, ) } + /// The global element declarations, also registering each under a reserved + /// `element:name` key in `types` so an `xs:element ref="name"` resolves to + /// this declaration's type (a colon keeps it out of the NCName-only type + /// namespace), letting forward and recursive refs resolve at validation time. + private static func globalElements( + _ containers: [XSDTree], + _ context: XSDContext, + into types: inout [String: ElementType], + ) -> [String: ElementType] { + var elements: [String: ElementType] = [:] + for node in allChildren(containers, named: "element") where XSDNode.attribute(node, "name") != nil { + let name = XSDNode.attribute(node, "name") ?? "" + let type = elementType(node, context) + elements[name] = type + types[elementKey(name)] = type + } + return elements + } + private static func allChildren(_ containers: [XSDTree], named name: String) -> [XSDTree] { containers.flatMap { XSDNode.children($0, named: name) } } diff --git a/Sources/Schema/XSDSimpleParser.swift b/Sources/Schema/XSDSimpleParser.swift index 5463f83..ba6d787 100644 --- a/Sources/Schema/XSDSimpleParser.swift +++ b/Sources/Schema/XSDSimpleParser.swift @@ -27,9 +27,70 @@ extension PureXML.Schema { facets.enumeration = nil } applyFacets(restriction, into: &facets) + for error in facetDefinitionErrors(restriction) { + context.diagnostics.report(error) + } return SimpleType(base: baseType.base, facets: facets, variety: baseType.variety) } + /// Schema-validity findings for the constraining facets declared on one + /// `restriction` (XSD Part 2 Datatypes 4.3): each length-family facet value + /// must be a valid `nonNegativeInteger` (`totalDigits` a `positiveInteger`), + /// `length` may not co-occur with `minLength`/`maxLength`, and the integer + /// ranges must be ordered. Reads the raw lexical from the restriction, since + /// numeric facet parsing silently drops a malformed value. Limited to the + /// facets declared on this step, so an inherited facet is never flagged here. + static func facetDefinitionErrors(_ restriction: XSDTree) -> [String] { + var errors: [String] = [] + var values: [String: String] = [:] // facet name -> canonical digit string + let countFacets: Set = ["length", "minLength", "maxLength", "totalDigits", "fractionDigits"] + for facet in XSDNode.elementChildren(restriction) { + guard let name = XSDNode.localName(facet), countFacets.contains(name) else { continue } + let raw = XSDNode.attribute(facet, "value") + guard let canonical = raw.flatMap(canonicalNonNegativeInteger) else { + errors.append("facet '\(name)' value '\(raw ?? "")' is not a valid nonNegativeInteger") + continue + } + if name == "totalDigits", canonical == "0" { + errors.append("facet 'totalDigits' value '\(raw ?? "")' must be a positiveInteger") + } + values[name] = canonical + } + if values["length"] != nil, values["minLength"] != nil { + errors.append("facets 'length' and 'minLength' may not both be specified") + } + if values["length"] != nil, values["maxLength"] != nil { + errors.append("facets 'length' and 'maxLength' may not both be specified") + } + if let low = values["minLength"], let high = values["maxLength"], greater(low, than: high) { + errors.append("facet 'minLength' (\(low)) exceeds 'maxLength' (\(high))") + } + if let fraction = values["fractionDigits"], let total = values["totalDigits"], greater(fraction, than: total) { + errors.append("facet 'fractionDigits' (\(fraction)) exceeds 'totalDigits' (\(total))") + } + return errors + } + + /// The canonical digit string of a lexical `nonNegativeInteger` (optional + /// leading `+`, ASCII digits, surrounding whitespace tolerated), with the + /// sign and leading zeros stripped (`"0"` for zero), or nil when the lexical + /// is not one (`-1`, ``, `a`, `1e2`). Independent of machine-integer range, + /// so a huge but well-formed value is canonicalized, not misreported as + /// malformed; the canonical form orders two values by length then lexically. + private static func canonicalNonNegativeInteger(_ lexical: String) -> String? { + var digits = Substring(lexical.trimmingXMLWhitespace()) + if digits.first == "+" { digits = digits.dropFirst() } + guard !digits.isEmpty, digits.allSatisfy({ $0.isASCII && $0.isNumber }) else { return nil } + let trimmed = digits.drop { $0 == "0" } + return trimmed.isEmpty ? "0" : String(trimmed) + } + + /// Whether canonical nonNegativeInteger `lhs` is greater than `rhs`: a longer + /// digit string is larger, and same-length strings compare lexicographically. + private static func greater(_ lhs: String, than rhs: String) -> Bool { + lhs.count != rhs.count ? lhs.count > rhs.count : lhs > rhs + } + static func simpleTypeReference(_ typeName: String, _ context: XSDContext) -> SimpleType { let local = XSDNode.stripPrefix(typeName) if let builtin = BuiltinType(rawValue: local) { return SimpleType(base: builtin) } diff --git a/Tests/SchemaFacetDefinitionTests.swift b/Tests/SchemaFacetDefinitionTests.swift new file mode 100644 index 0000000..6c346d0 --- /dev/null +++ b/Tests/SchemaFacetDefinitionTests.swift @@ -0,0 +1,83 @@ +@testable import PureXML +import Testing + +/// Schema-validity of constraining-facet definitions (XSD Part 2 Datatypes 4.3): +/// a length-family facet value must be a valid `nonNegativeInteger` +/// (`totalDigits` a `positiveInteger`), `length` may not co-occur with +/// `minLength`/`maxLength`, and the integer ranges must be ordered. Such a schema +/// is invalid and must be rejected at compile time; previously the malformed +/// value was silently dropped and the schema accepted (XSTS invalid-schema +/// MS DataTypes facet cases). +@Suite("Facet definition validity") +struct SchemaFacetDefinitionTests { + private func compile(_ facets: String, base: String = "xs:string") throws { + _ = try PureXML.Schema.Document(""" + + + + \(facets) + + + + """) + } + + private func rejects(_ facets: String, base: String = "xs:string") -> Bool { + do { try compile(facets, base: base) + return false + } catch { return true } + } + + @Test("a well-formed facet definition compiles") + func test_valid() throws { + try compile(#""#) + try compile(#""#) + try compile(#""#) + try compile(#""#, base: "xs:decimal") + try compile(#""#) + // A well-formed nonNegativeInteger is accepted regardless of machine-integer + // range, and tolerates a leading "+" and leading zeros. + try compile(#""#) + try compile(#""#) + try compile(#""#) + try compile(#""#) + } + + @Test("a malformed length-family lexical is rejected") + func test_malformedLexical() { + #expect(rejects(#""#)) + #expect(rejects(#""#)) + #expect(rejects(#""#)) + #expect(rejects(#""#)) + #expect(rejects(#""#)) + #expect(rejects(#""#)) + } + + @Test("totalDigits must be a positiveInteger") + func test_totalDigitsPositive() { + #expect(rejects(#""#, base: "xs:decimal")) + #expect(rejects(#""#, base: "xs:decimal")) + } + + @Test("length may not co-occur with minLength or maxLength") + func test_lengthCoOccurrence() { + #expect(rejects(#""#)) + #expect(rejects(#""#)) + } + + @Test("integer facet ranges must be ordered") + func test_rangeOrder() { + #expect(rejects(#""#)) + #expect(rejects(#""#, base: "xs:decimal")) + } + + @Test("the rejection message names the offending facet") + func test_locatedMessage() { + do { + try compile(#""#) + Issue.record("expected the malformed facet to be rejected") + } catch { + #expect(String(describing: error).contains("length")) + } + } +} diff --git a/Tests/XSTSSuiteTests.swift b/Tests/XSTSSuiteTests.swift index 61d8414..fb9b96f 100644 --- a/Tests/XSTSSuiteTests.swift +++ b/Tests/XSTSSuiteTests.swift @@ -29,7 +29,7 @@ struct XSTSSuiteTests { /// XSTS` (debug is far slower and the corpus is large). Per-case deviations /// are written to /tmp/xsts-failures.txt for the burn-down. private let knownSchemaValidRejected = 72 - private let knownSchemaInvalidAccepted = 2461 + private let knownSchemaInvalidAccepted = 2082 private let knownInstanceValidRejected = 233 private let knownInstanceInvalidAccepted = 165 diff --git a/docs/schema-validity-burndown.md b/docs/schema-validity-burndown.md index ec22199..f11adae 100644 --- a/docs/schema-validity-burndown.md +++ b/docs/schema-validity-burndown.md @@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open. | Bucket | Count | Reading | |---|---|---| | valid-schemas-rejected | 72 | we rarely reject a good schema | -| **invalid-schemas-accepted** | **2461** | we rarely catch a bad one | +| **invalid-schemas-accepted** | **2461 -> 2082** | we rarely catch a bad one (facet-definition validity landed) | | valid-instances-rejected | 233 | (instance side, separately tracked) | | invalid-instances-accepted | 165 | | @@ -38,16 +38,17 @@ existing pattern: collect ALL findings, report them together. Each iteration targets one self-contained rule family with a clean root cause. -1. **Facet-definition validity** (XSD Part 2 §4.3). Each constraining facet's - `value` must be a valid instance of the facet's own type (`length`, - `minLength`, `maxLength`, `totalDigits`, `fractionDigits` are - `nonNegativeInteger`; `positiveInteger` for `totalDigits`; `minInclusive` - etc. must be valid in the base value space); facet co-occurrence rules - (`length` excludes `minLength`/`maxLength`; `minInclusive` excludes - `minExclusive`; `maxInclusive` excludes `maxExclusive`; `minInclusive` <= - `maxInclusive`; `minLength` <= `maxLength`; `fractionDigits` <= - `totalDigits`); `whiteSpace` may not weaken an inherited value. Big chunk of - DataTypes (703) and SimpleType (139). +1. **Facet-definition validity** (XSD Part 2 §4.3). _Partly done (2461 -> 2082)._ + Landed: length-family facet values must be `nonNegativeInteger` + (`totalDigits` a `positiveInteger`); `length` excludes + `minLength`/`maxLength`; `minLength` <= `maxLength`; `fractionDigits` <= + `totalDigits`. **Still open in this family:** `minInclusive`/`maxInclusive`/ + `minExclusive`/`maxExclusive` value well-formedness in the base value space + and their mutual exclusions/ordering; facet applicability per base + (`fractionDigits` only on decimal-derived, `length` only on + string/binary/list, etc.); `whiteSpace` may not weaken an inherited value; + facet repetition. Inherited-facet co-occurrence (across derivation steps) is + deliberately not checked yet (current pass reads only the local restriction). 2. **Resolvable references.** Every QName a schema names (`type`, element / attribute / group `ref`, restriction / extension `base`, `itemType`, `memberTypes`, `substitutionGroup`, keyref `refer`) must resolve to a diff --git a/docs/xsts-deviations.md b/docs/xsts-deviations.md index 37ecdb9..f81c3da 100644 --- a/docs/xsts-deviations.md +++ b/docs/xsts-deviations.md @@ -47,7 +47,7 @@ and to document the remainder as spec-justified, deliberate exclusions. ## Category A: genuine correctness bugs -### Fixed (thirteen root causes, ~655 deviations cleared) +### Fixed (fourteen root causes, ~1034 deviations cleared) list-datatype length facets (#146) · union pattern/enumeration facets · duration partial-order facets · QName length non-constraining · the full XSD `\p{Is...}` @@ -58,7 +58,11 @@ xs:ID/xs:IDREF uniqueness and resolution · `` resolution · mixed-content type with no element model · built-in `xsi:type` resolution gated by substitution validity (the XSD Part 2 derivation lattice), which also closed the two gaps it exposed (a simple-typed element rejecting stray attributes; -identity fields comparing in value space when the node carries an `xsi:type`). +identity fields comparing in value space when the node carries an `xsi:type`) · +compile-time constraining-facet definition validity (length-family facets are +`nonNegativeInteger`, co-occurrence and range order), the first cut at the +schema-validity gap (invalid schemas accepted 2461 to 2082; see +`schema-validity-burndown.md`). Plus the measurement fix (exclude W3C-disputed `status="queried"` entries). ### Remaining, root-caused (the hard tail)