From 084716a1e38ac6de770f1c3ca4c880009d05761a Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Tue, 23 Jun 2026 04:56:17 +0200 Subject: [PATCH] fix(schema): attribute-wildcard union/intersection per Errata E1-10 (#251) Generalise the wildcard {namespace constraint} model to the four XSD 1.0 (with Errata E1-10) expressible forms -- any; a set (with absent); a pair of not and a namespace name; a pair of not and absent -- replacing the any/##other/set model that could not represent the union/intersection results. Both not forms exclude absent, so a union admitting "every namespace except a name, plus absent" (E1-10 rule 5.1.3) has no expressible form. A new SchemaWildcardExpressibility compile rule rejects a complexContent extension whose attribute-wildcard union with its base is not expressible (src-ct.5), closing XSTS wildZ013 (test328873i). The expressible unions are computed exactly in XSDComplexContent, so a derived type's wildcard admits the correct attributes at instance time, closing wildZ013a and wildZ013d. invalid-schemas-accepted 12 -> 11, invalid-instances-accepted 17 -> 15, with valid-schemas-rejected and valid-instances-rejected held at 0 (full XSTS archive). The model is additive for element wildcards (which never produce the new forms); admits/narrows/UPA-overlap updated and verified unchanged for the ##other cases. The intersection of two not forms negating different namespace names is not expressible (src-ct.4), a deferred rule: rather than drop the wildcard and over-reject valid cross-namespace attributes on an accepted schema, the intersection stays the less-restrictive not(lhs) over-approximation (FP-safe, matching prior behaviour). Union/intersection algebra taken clean-room from Xerces-J XSWildcardDecl. Guarded by SchemaWildcardUnionExpressibilityTests. --- CHANGELOG.md | 2 + Sources/Schema/ComplexType.swift | 157 +++++++++++------- ...ContentModelDeterminismCompositional.swift | 17 +- Sources/Schema/ParticleRestriction.swift | 15 +- Sources/Schema/SchemaCompileValidation.swift | 8 + .../Schema/SchemaWildcardExpressibility.swift | 49 ++++++ Sources/Schema/XSDComplexContent.swift | 9 +- Sources/Schema/XSDWildcardParser.swift | 65 ++++++-- ...hemaWildcardUnionExpressibilityTests.swift | 109 ++++++++++++ Tests/ValidationConfigurationTests.swift | 1 + Tests/XSTSSuiteTests.swift | 12 +- docs/release-hurdles.md | 10 +- 12 files changed, 360 insertions(+), 94 deletions(-) create mode 100644 Sources/Schema/SchemaWildcardExpressibility.swift create mode 100644 Tests/SchemaWildcardUnionExpressibilityTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index eb3696f..a2d61c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- Attribute-wildcard union and intersection now follow XSD 1.0 §3.10.6 with Errata E1-10, including the "not expressible" cases (false-negative burndown, stopper #2). The `{namespace constraint}` model was generalised from `any`/`##other`/set to the four expressible XSD forms: `any`; a set (with absent); a pair of `not` and a namespace name; a pair of `not` and absent. Both `not` forms exclude absent, so a union admitting "every namespace except a name, plus absent" (E1-10 rule 5.1.3) has no expressible form. PureXML previously widened such a union to `##any` and accepted the schema; the new `SchemaWildcardExpressibility` rule rejects a complexContent extension whose attribute-wildcard union with its base is not expressible (`src-ct.5`), closing XSTS wildZ013. The expressible unions are now computed exactly, so a derived type's wildcard admits the correct attributes at instance time, closing wildZ013a and wildZ013d. Net: `invalid-schemas-accepted` 12 -> 11 and `invalid-instances-accepted` 17 -> 15, with `valid-schemas-rejected` and `valid-instances-rejected` held at 0 (full XSTS archive); the generalised model is additive for element wildcards (which never produce the new forms). The union/intersection algebra was taken clean-room from Xerces-J `XSWildcardDecl` (factual rule numbers only). Guarded by `SchemaWildcardUnionExpressibilityTests` and the existing intersection tests. Disclosed under-rejection: the intersection-not-expressible case (`src-ct.4`, two `not` forms negating different namespace names within one type, e.g. a type intersecting its own `##other` with a cross-imported attribute group's `##other`) is not flagged as a schema error; the intersection stays the less-restrictive `not(lhs)` over-approximation so a valid cross-namespace attribute is never wrongly rejected (the prior behaviour, kept FP-safe), and `src-ct.4` detection is a deferred rule. + - A negated character-class subtraction now excludes both the negated base and the subtrahend (false-negative burndown, stopper #2). XSD `charClassSub ::= (posCharGroup | negCharGroup) '-' charClassExpr` is the set difference of the (possibly negated) group and the subtrahend, so for `[^cde-[ag]]` negation applies to the base first and the subtrahend is removed from that result: the class admits everything except `{a, c, d, e, g}`. `CharClass.matches` applied the subtraction to the positive membership and negated afterwards, so it excluded only `{c, d, e}` and wrongly matched `a`/`g`, accepting instances a correct engine rejects. Negation now applies to the base before subtraction. Only negated classes that also carry a subtraction change behaviour. This closes XSTS RegexTest_430 and RegexTest_422, dropping `invalid-instances-accepted` from 19 to 17 with `valid-instances-rejected` held at 0 (full XSTS archive). A regression is guarded by `RegexCategoryTests.test_negatedSubtraction`. - An unknown `\p{...}` category or block name in a `pattern` facet is now rejected at schema-compile time (false-negative burndown, stopper #2). XSD 1.0 `charProp` is `IsCategory | IsBlock`, an exhaustive set: every Unicode general category plus a fixed Unicode-3.1 block list. The block table (`CategoryMatcher.blocks`) was missing the nine supplementary-plane blocks (Old Italic, Gothic, Deseret, the Byzantine/Musical/Mathematical symbol blocks, CJK Unified Ideographs Extension B, CJK Compatibility Ideographs Supplement, Tags) and the three surrogate blocks, so `parseCategory` could not tell a real-but-untabulated block from a bogus name and stayed lenient (`RegexError.unsupported`), accepting `\p{IsaA0-a9}` and the schema declaring it. The table is now the complete XSD set (supplementary blocks added with their ranges; surrogate blocks added as valid names whose ranges hold no scalar, so they match nothing), making the recognised set authoritative, so an unrecognised name throws the rejecting `RegexError.invalidProperty`. The now-dead `RegexError.unsupported` case and the `isMalformedProperty` heuristic are removed. This closes XSTS reK88, dropping `invalid-schemas-accepted` from 13 to 12 with `valid-schemas-rejected` held at 0 (full XSTS archive) and the instance buckets unchanged (`invalid-instances-accepted` 19). As a side effect, a pattern naming a supplementary-plane block (e.g. `\p{IsMusicalSymbols}`) now matches its characters instead of failing to compile. Block names/ranges were taken from the XSD-supported list in Xerces-J `Token.java` (clean-room: factual data only). Regressions are guarded by `RegexCategoryTests` (supplementary/surrogate matching, unknown-name rejection) and `SchemaPatternFacetTests`. diff --git a/Sources/Schema/ComplexType.swift b/Sources/Schema/ComplexType.swift index 5bb8772..c2319be 100644 --- a/Sources/Schema/ComplexType.swift +++ b/Sources/Schema/ComplexType.swift @@ -9,16 +9,25 @@ public extension PureXML.Schema { case all } - /// The namespace constraint of a wildcard (`xs:any`/`xs:anyAttribute`). + /// The namespace constraint of a wildcard (`xs:any`/`xs:anyAttribute`). These + /// are exactly the XSD 1.0 (with Errata E1-10) expressible {namespace + /// constraint} forms: `any`; a set of namespace names possibly including + /// absent; a pair of `not` and a namespace name; a pair of `not` and absent. + /// Both `not` forms exclude absent (E1-10 "Wildcard allows Namespace Name"), so + /// there is NO "everything except N but including absent" form -- a derivation + /// whose attribute-wildcard union/intersection would need one is `not + /// expressible` (`src-ct.4`/`src-ct.5`), checked in ``SchemaWildcardExpressibility``. enum WildcardNamespace: Sendable, Equatable { /// `##any`: a name in any namespace, or none. case any - /// `##other`: a name in any namespace other than the target, and not in no - /// namespace. - case other /// A specific set of namespace URIs; the empty string stands for no /// namespace (`##local`). case enumerated(Set) + /// A pair of `not` and a namespace name: any namespace name other than the + /// given one, and NOT absent. `##other` is `notNamespace(targetNamespace)`. + case notNamespace(String) + /// A pair of `not` and absent: any (non-absent) namespace name. + case notAbsent } /// How a wildcard-matched item is validated: `skip` (no validation), `lax` @@ -30,6 +39,16 @@ public extension PureXML.Schema { case strict } + /// The outcome of combining two wildcard constraints: a constraint (possibly + /// `nil`, meaning no wildcard), or `notExpressible` when the XSD 1.0 (Errata + /// E1-10) union/intersection cannot be written as one of the four + /// `WildcardNamespace` forms -- a `src-ct.4` (intersection) or `src-ct.5` + /// (union) Complex Type Definition Representation error. + enum WildcardCombination: Sendable, Equatable { + case constraint(Wildcard?) + case notExpressible + } + /// An element or attribute wildcard (`xs:any`/`xs:anyAttribute`): which /// namespaces it admits, and how matched items are validated. struct Wildcard: Sendable, Equatable { @@ -54,39 +73,60 @@ public extension PureXML.Schema { switch namespace { case .any: return true - case .other: - return !namespaceURI.isEmpty && namespaceURI != (targetNamespace ?? "") case let .enumerated(uris): return uris.contains(namespaceURI) + case let .notNamespace(excluded): + return !namespaceURI.isEmpty && namespaceURI != excluded + case .notAbsent: + return !namespaceURI.isEmpty } } - /// The union of two wildcard constraints (XSD 1.0 `cos-aw-union`), used when - /// a complexContent extension adds its own `anyAttribute`. - func union(with other: Wildcard) -> Wildcard { - Wildcard( - namespace: Self.unionNamespace(namespace, other.namespace), + /// The union of two wildcard constraints (XSD 1.0 `cos-aw-union` + Errata + /// E1-10), used when a complexContent extension adds its own `anyAttribute`. + func union(with other: Wildcard) -> WildcardCombination { + guard let namespace = Self.unionNamespace(namespace, other.namespace) else { return .notExpressible } + return .constraint(Wildcard( + namespace: namespace, processContents: Self.unionProcessContents(processContents, other.processContents), targetNamespace: targetNamespace ?? other.targetNamespace, - ) + )) } - /// Combines optional wildcards from a base type and a derivation. - static func union(_ left: Wildcard?, _ right: Wildcard?) -> Wildcard? { + /// Combines optional wildcards from a base type and a derivation by union. + static func union(_ left: Wildcard?, _ right: Wildcard?) -> WildcardCombination { switch (left, right) { - case (nil, nil): nil - case (let left?, nil): left - case (nil, let right?): right + case (nil, nil): .constraint(nil) + case (let left?, nil): .constraint(left) + case (nil, let right?): .constraint(right) case let (left?, right?): left.union(with: right) } } - private static func unionNamespace(_ left: WildcardNamespace, _ right: WildcardNamespace) -> WildcardNamespace { + /// The union of two namespace constraints (XSD 1.0 §3.10.6 with Errata + /// E1-10), or `nil` when the result is not expressible. Both `not` forms + /// exclude absent, so a union that would admit "everything except a name, + /// plus absent" (rule 5.1.3) has no expressible form. + static func unionNamespace(_ left: WildcardNamespace, _ right: WildcardNamespace) -> WildcardNamespace? { switch (left, right) { - case (.any, _), (_, .any): .any - case (.other, .other): .other - case let (.enumerated(lhs), .enumerated(rhs)): .enumerated(lhs.union(rhs)) - case (.other, _), (_, .other): .any + case (.any, _), (_, .any): + return .any + case let (.enumerated(lhs), .enumerated(rhs)): + return .enumerated(lhs.union(rhs)) + case let (.notNamespace(lhsName), .notNamespace(rhsName)): + // not(x) admits the other's excluded name, so different names union + // to "any named, no absent"; the same name is unchanged. + return lhsName == rhsName ? .notNamespace(lhsName) : .notAbsent + case (.notAbsent, .notNamespace), (.notNamespace, .notAbsent), (.notAbsent, .notAbsent): + return .notAbsent + case let (.notNamespace(name), .enumerated(set)), let (.enumerated(set), .notNamespace(name)): + let hasName = set.contains(name), hasAbsent = set.contains("") + if hasName, hasAbsent { return .any } // E1-10 5.1.1 + if hasName { return .notAbsent } // E1-10 5.1.2 + if hasAbsent { return nil } // NOT EXPRESSIBLE, E1-10 5.1.3 + return .notNamespace(name) // E1-10 5.1.4 + case let (.notAbsent, .enumerated(set)), let (.enumerated(set), .notAbsent): + return set.contains("") ? .any : .notAbsent } } @@ -96,54 +136,57 @@ public extension PureXML.Schema { return .skip } - /// The intersection of two wildcard constraints (XSD 1.0 `cos-aw-intersect`), - /// used to combine the `anyAttribute` wildcards a complex type or attribute - /// group draws from its own declaration and its referenced attribute groups: - /// the effective wildcard admits only what every source admits. - func intersection(with other: Wildcard) -> Wildcard { - Wildcard( - namespace: Self.intersectNamespace(self, other), + /// The intersection of two wildcard constraints (XSD 1.0 `cos-aw-intersect` + /// + Errata E1-10), used to combine the `anyAttribute` wildcards a complex + /// type or attribute group draws from its own declaration and its referenced + /// attribute groups: the effective wildcard admits only what every source + /// admits. + func intersection(with other: Wildcard) -> WildcardCombination { + .constraint(Wildcard( + namespace: Self.intersectNamespace(namespace, other.namespace), processContents: Self.intersectProcessContents(processContents, other.processContents), targetNamespace: targetNamespace ?? other.targetNamespace, - ) + )) } /// Combines optional wildcards from several sources by intersection. A nil - /// source contributes no constraint (the unconstrained `##any` identity), so - /// a single wildcard is returned unchanged: only a type drawing on two or - /// more `anyAttribute`s narrows. - static func intersection(_ left: Wildcard?, _ right: Wildcard?) -> Wildcard? { + /// source contributes no constraint (the unconstrained `##any` identity). + static func intersection(_ left: Wildcard?, _ right: Wildcard?) -> WildcardCombination { switch (left, right) { - case (nil, nil): nil - case (let left?, nil): left - case (nil, let right?): right + case (nil, nil): .constraint(nil) + case (let left?, nil): .constraint(left) + case (nil, let right?): .constraint(right) case let (left?, right?): left.intersection(with: right) } } - private static func intersectNamespace(_ left: Wildcard, _ right: Wildcard) -> WildcardNamespace { - switch (left.namespace, right.namespace) { - case (.any, _): right.namespace - case (_, .any): left.namespace - case let (.enumerated(lhs), .enumerated(rhs)): .enumerated(lhs.intersection(rhs)) - case let (.enumerated(uris), .other): .enumerated(Self.exclude(uris, otherThan: right.targetNamespace)) - case let (.other, .enumerated(uris)): .enumerated(Self.exclude(uris, otherThan: left.targetNamespace)) - case (.other, .other): - // Two `##other`s with the same target namespace intersect to the - // same `##other`; with different targets the true intersection - // (neither target, nor absent) is not expressible in XSD 1.0, so - // keep the less restrictive `##other` rather than over-reject. - .other + /// The intersection of two namespace constraints (XSD 1.0 §3.10.6 with + /// Errata E1-10). Two `not` forms negating DIFFERENT namespace names have a + /// true intersection of `not({a,b})` excluding absent, which has no single + /// expressible form (`src-ct.4`). Detecting that as a schema error is a + /// deferred rule; until then this returns the less restrictive `not(lhs)` (an + /// over-approximation, so an attribute is never WRONGLY rejected -- the prior + /// behaviour), rather than dropping the wildcard and over-rejecting valid + /// cross-namespace attributes on an accepted schema. + static func intersectNamespace(_ left: WildcardNamespace, _ right: WildcardNamespace) -> WildcardNamespace { + switch (left, right) { + case (.any, _): right + case (_, .any): left + case let (.enumerated(lhs), .enumerated(rhs)): + .enumerated(lhs.intersection(rhs)) + case let (.enumerated(set), .notNamespace(name)), let (.notNamespace(name), .enumerated(set)): + .enumerated(set.subtracting([name, ""])) // not(name) also excludes absent + case let (.enumerated(set), .notAbsent), let (.notAbsent, .enumerated(set)): + .enumerated(set.subtracting([""])) + case let (.notNamespace(lhsName), .notNamespace(_)): + .notNamespace(lhsName) // same name: unchanged; different: lenient over-approximation + case (.notAbsent, .notAbsent): + .notAbsent + case let (.notNamespace(name), .notAbsent), let (.notAbsent, .notNamespace(name)): + .notNamespace(name) } } - /// The URIs of `uris` that `##other` (relative to `targetNamespace`) also - /// admits: a non-empty namespace other than the target. The empty string - /// (absent / `##local`) and the target namespace are dropped. - private static func exclude(_ uris: Set, otherThan targetNamespace: String?) -> Set { - uris.filter { !$0.isEmpty && $0 != (targetNamespace ?? "") } - } - private static func intersectProcessContents(_ left: ProcessContents, _ right: ProcessContents) -> ProcessContents { // When the two differ the intersection is strictly not expressible; keep // the less strict so a matched attribute is never validated more harshly diff --git a/Sources/Schema/ContentModelDeterminismCompositional.swift b/Sources/Schema/ContentModelDeterminismCompositional.swift index 3dfbce5..89eae17 100644 --- a/Sources/Schema/ContentModelDeterminismCompositional.swift +++ b/Sources/Schema/ContentModelDeterminismCompositional.swift @@ -237,16 +237,19 @@ private struct CompWalk { } static func wildcardsOverlap(_ lhs: PureXML.Schema.Wildcard, _ rhs: PureXML.Schema.Wildcard) -> Bool { + // Two wildcards overlap when some namespace name is admitted by both. Any + // two `not` forms always share infinitely many named namespaces; a `not` + // form overlaps a set when the set holds a named namespace it admits. switch (lhs.namespace, rhs.namespace) { case (.any, _), (_, .any): true - case (.other, .other): true case let (.enumerated(lhsSet), .enumerated(rhsSet)): !lhsSet.isDisjoint(with: rhsSet) - case let (.other, .enumerated(set)): otherOverlapsEnumerated(set, target: lhs.targetNamespace) - case let (.enumerated(set), .other): otherOverlapsEnumerated(set, target: rhs.targetNamespace) + case let (.enumerated(set), .notNamespace(name)), let (.notNamespace(name), .enumerated(set)): + set.contains { !$0.isEmpty && $0 != name } + case let (.enumerated(set), .notAbsent), let (.notAbsent, .enumerated(set)): + set.contains { !$0.isEmpty } + case (.notNamespace, .notNamespace), (.notNamespace, .notAbsent), + (.notAbsent, .notNamespace), (.notAbsent, .notAbsent): + true } } - - static func otherOverlapsEnumerated(_ set: Set, target: String?) -> Bool { - set.contains { !$0.isEmpty && $0 != (target ?? "") } - } } diff --git a/Sources/Schema/ParticleRestriction.swift b/Sources/Schema/ParticleRestriction.swift index 1e71b67..549496e 100644 --- a/Sources/Schema/ParticleRestriction.swift +++ b/Sources/Schema/ParticleRestriction.swift @@ -264,15 +264,22 @@ extension PureXML.Schema { guard processContentsRank(restricted.processContents) >= processContentsRank(base.processContents) else { return false } + // Wildcard Subset (XSD 1.0 §3.10.6 / Errata E1-10): the restriction's + // namespace constraint must admit only what the base does. Both `not` + // forms exclude absent. return switch (restricted.namespace, base.namespace) { case (_, .any): true - case (.other, .other): - restricted.targetNamespace == base.targetNamespace case let (.enumerated(narrow), .enumerated(wide)): narrow.isSubset(of: wide) - case let (.enumerated(narrow), .other): - !narrow.contains("") && !narrow.contains(base.targetNamespace ?? "") + case let (.enumerated(narrow), .notNamespace(excluded)): + !narrow.contains(excluded) && !narrow.contains("") + case let (.enumerated(narrow), .notAbsent): + !narrow.contains("") + case let (.notNamespace(narrow), .notNamespace(wide)): + narrow == wide + case (.notNamespace, .notAbsent), (.notAbsent, .notAbsent): + true default: false } diff --git a/Sources/Schema/SchemaCompileValidation.swift b/Sources/Schema/SchemaCompileValidation.swift index 6cd1547..e48332e 100644 --- a/Sources/Schema/SchemaCompileValidation.swift +++ b/Sources/Schema/SchemaCompileValidation.swift @@ -222,6 +222,13 @@ extension PureXML.Validation { } } + static var attributeWildcardUnionsExpressible: Validation { + compileRule("Attribute wildcard unions in type extensions are expressible") { document in + guard let namedTypes = document.namedTypes else { return [] } + return PureXML.Schema.XSDParser.wildcardUnionExpressibilityFindings(document.schema, document.context, namedTypes) + } + } + /// The default validator for checks that run before named types resolve. static func preCompileValidator() -> Validator { Validator.defaults( @@ -256,6 +263,7 @@ extension PureXML.Validation { AnyValidation(simpleTypeBasesAreSimple), AnyValidation(simpleTypeVarietiesValid), AnyValidation(notationsValid), AnyValidation(simpleTypeFacetsAreValid), + AnyValidation(attributeWildcardUnionsExpressible), ], reference: [ AnyValidation(schemaReferencesResolve), diff --git a/Sources/Schema/SchemaWildcardExpressibility.swift b/Sources/Schema/SchemaWildcardExpressibility.swift new file mode 100644 index 0000000..7348ff3 --- /dev/null +++ b/Sources/Schema/SchemaWildcardExpressibility.swift @@ -0,0 +1,49 @@ +private typealias WildExprNode = PureXML.Schema.XSDNode + +extension PureXML.Schema.XSDParser { + /// cos-ct-extends.1.1 / `src-ct.5`: a complexContent extension's {attribute + /// wildcard} is the wildcard union of the extension's own attribute wildcard and + /// the base type's (XSD 1.0 §3.4.2). With Errata E1-10 some unions are NOT + /// expressible -- both `not` namespace-constraint forms exclude absent, so a + /// union admitting "every namespace except a name, PLUS absent" (rule 5.1.3, + /// e.g. base `##other` unioned with `##local` and other names) has no single + /// expressible form, and the schema is invalid. PureXML formerly widened such a + /// union to `##any` and accepted it (XSTS wildZ013, test328873i). + /// + /// The union RESULT for the expressible cases is already computed correctly in + /// `XSDComplexContent` (so instance validation admits exactly the right + /// attributes, XSTS wildZ013a/d); this rule reports the not-expressible ones. + /// + /// Disclosed under-rejection (deferred): the INTERSECTION not-expressible case + /// (`src-ct.4` / `src-attribute_group.2`, two `not` forms negating different + /// namespace names within one type or attribute group) is not flagged here; it + /// needs no corpus case and is a separate, rarer rule. + /// + /// Located on the `extension` node, the component a developer would correct. + /// Self-contained schemas only (the base must resolve to this document's target + /// namespace), matching the other derivation rules. + static func wildcardUnionExpressibilityFindings( + _ schema: XSDTree, + _ context: PureXML.Schema.XSDContext, + _ types: [String: PureXML.Schema.ElementType], + ) -> [PureXML.Schema.SchemaLocatedFinding] { + guard !skipsCrossDocumentRules(schema, compositionLoaded: context.compositionLoaded) else { return [] } + let bindings = context.namespaceBindings + var findings: [PureXML.Schema.SchemaLocatedFinding] = [] + for content in descendants(schema, named: "complexContent") { + guard let extensionNode = WildExprNode.firstChild(content, named: "extension"), + let base = WildExprNode.attribute(extensionNode, "base"), + WildExprNode.referenceNamespace(base, bindings) == context.targetNamespace, + case let .complex(complex)? = types[WildExprNode.stripPrefix(base)] + else { continue } + let ownWildcard = attributeWildcard(under: extensionNode, context) + if case .notExpressible = PureXML.Schema.Wildcard.union(ownWildcard, complex.attributeWildcard) { + findings.append(PureXML.Schema.SchemaLocatedFinding( + reason: "the union of this extension's attribute wildcard with the base type's attribute wildcard is not expressible (src-ct.5)", + node: extensionNode, + )) + } + } + return findings + } +} diff --git a/Sources/Schema/XSDComplexContent.swift b/Sources/Schema/XSDComplexContent.swift index 76c2c67..fb7acb9 100644 --- a/Sources/Schema/XSDComplexContent.swift +++ b/Sources/Schema/XSDComplexContent.swift @@ -51,9 +51,16 @@ extension PureXML.Schema.XSDParser { let derivationType = XSDContentNode.localName(derivationNode) if derivationType == "extension", let base = resolvedBase(derivationNode, context) { + // Best-effort wildcard for type building: a not-expressible union makes + // the schema invalid (flagged by SchemaWildcardExpressibility); fall back + // to the base's wildcard so the dead type is still well-formed. + let unionWildcard: PureXML.Schema.Wildcard? = switch PureXML.Schema.Wildcard.union(ownWildcard, base.attributeWildcard) { + case let .constraint(wildcard): wildcard + case .notExpressible: base.attributeWildcard + } return XSDComplexType( attributes: base.attributes + ownAttributes, - attributeWildcard: PureXML.Schema.Wildcard.union(ownWildcard, base.attributeWildcard), + attributeWildcard: unionWildcard, content: extendedContent(base: base.content, own: ownParticle, mixed: mixed), ) } else if derivationType == "restriction" { diff --git a/Sources/Schema/XSDWildcardParser.swift b/Sources/Schema/XSDWildcardParser.swift index f2e1876..f0ec239 100644 --- a/Sources/Schema/XSDWildcardParser.swift +++ b/Sources/Schema/XSDWildcardParser.swift @@ -6,37 +6,66 @@ extension PureXML.Schema.XSDParser { _ context: PureXML.Schema.XSDContext, visited: Set = [], ) -> PureXML.Schema.Wildcard? { + // Best-effort view for type building: a not-expressible intersection makes + // the schema invalid (flagged by SchemaWildcardExpressibility), so the type + // it would have carried is never used to validate a valid instance. + switch attributeWildcardCombination(under: node, context, visited: visited) { + case let .constraint(wildcard): wildcard + case .notExpressible: nil + } + } + + /// The effective `xs:anyAttribute` wildcard under `node`, or `notExpressible` + /// when intersecting the sources yields no expressible {namespace constraint}. + static func attributeWildcardCombination( + under node: XSDTree, + _ context: PureXML.Schema.XSDContext, + visited: Set = [], + ) -> PureXML.Schema.WildcardCombination { // A complex type's (or attribute group's) effective {attribute wildcard} is // the INTERSECTION of the wildcards it draws from its own `anyAttribute` and // from every referenced attribute group (XSD 1.0 `cos-aw-intersect`): an // attribute is admitted only if every source admits it. (An extension's // union with its base wildcard is handled separately, in XSDComplexContent.) - var combined: PureXML.Schema.Wildcard? + var combined: PureXML.Schema.WildcardCombination = .constraint(nil) + func merge(_ next: PureXML.Schema.Wildcard?) { + guard case let .constraint(current) = combined else { return } + combined = PureXML.Schema.Wildcard.intersection(current, next) + } if let direct = PureXML.Schema.XSDNode.firstChild(node, named: "anyAttribute").map({ wildcard($0, context) }) { - combined = PureXML.Schema.Wildcard.intersection(combined, direct) + merge(direct) } for child in PureXML.Schema.XSDNode.elementChildren(node) { + guard case .constraint = combined else { break } guard PureXML.Schema.XSDNode.localName(child) == "attributeGroup", let ref = PureXML.Schema.XSDNode.attribute(child, "ref") else { continue } - let name = PureXML.Schema.XSDNode.stripPrefix(ref) - if visited.contains(name) { - guard context.redefinedAttributeGroups.contains(name), - let base = context.baseAttributeGroups[name], - let found = attributeWildcard(under: base, context, visited: visited) - else { continue } - combined = PureXML.Schema.Wildcard.intersection(combined, found) - continue - } - guard let group = context.attributeGroups[name] else { continue } - let scoped = context.scoped(for: PureXML.Schema.XSDNode.schemaOwner(group)) - if let found = attributeWildcard(under: group, scoped, visited: visited.union([name])) { - combined = PureXML.Schema.Wildcard.intersection(combined, found) + switch referencedGroupWildcard(PureXML.Schema.XSDNode.stripPrefix(ref), context, visited: visited) { + case let .constraint(found): merge(found) + case .notExpressible: combined = .notExpressible } } return combined } + /// The effective wildcard combination of the attribute group referenced by + /// `name`, resolving a redefine self-reference against the group's base. + private static func referencedGroupWildcard( + _ name: String, + _ context: PureXML.Schema.XSDContext, + visited: Set, + ) -> PureXML.Schema.WildcardCombination { + if visited.contains(name) { + guard context.redefinedAttributeGroups.contains(name), + let base = context.baseAttributeGroups[name] + else { return .constraint(nil) } + return attributeWildcardCombination(under: base, context, visited: visited) + } + guard let group = context.attributeGroups[name] else { return .constraint(nil) } + let scoped = context.scoped(for: PureXML.Schema.XSDNode.schemaOwner(group)) + return attributeWildcardCombination(under: group, scoped, visited: visited.union([name])) + } + /// Parses a wildcard's `namespace` and `processContents` constraints. static func wildcard(_ node: XSDTree, _ context: PureXML.Schema.XSDContext) -> PureXML.Schema.Wildcard { let processContents: PureXML.Schema.ProcessContents = switch PureXML.Schema.XSDNode.attribute(node, "processContents") { @@ -51,7 +80,11 @@ extension PureXML.Schema.XSDParser { private static func wildcardNamespace(_ value: String?, _ context: PureXML.Schema.XSDContext) -> PureXML.Schema.WildcardNamespace { switch value { case nil, "##any": return .any - case "##other": return .other + case "##other": + // `##other` = any namespace name other than the target, never absent. + // With no target namespace it is simply "any non-absent namespace". + if let target = context.targetNamespace, !target.isEmpty { return .notNamespace(target) } + return .notAbsent default: let uris = (value ?? "").split(whereSeparator: \.isWhitespace).map { token -> String in switch token { diff --git a/Tests/SchemaWildcardUnionExpressibilityTests.swift b/Tests/SchemaWildcardUnionExpressibilityTests.swift new file mode 100644 index 0000000..bf38fe8 --- /dev/null +++ b/Tests/SchemaWildcardUnionExpressibilityTests.swift @@ -0,0 +1,109 @@ +import Testing +@testable import PureXML + +/// XSD 1.0 (Errata E1-10) cos-ct-extends / src-ct.5: a complexContent extension's +/// {attribute wildcard} is the wildcard UNION of the extension's own attribute +/// wildcard and the base type's. Both `not` namespace-constraint forms exclude +/// absent, so a union admitting "every namespace except a name, plus absent" has +/// no expressible form and the schema is invalid. The expressible unions must be +/// computed correctly (the FP-guard) and the inexpressible one rejected (XSTS +/// wildZ013 / test328873i and the test328873a/d instances). +@Suite("Attribute wildcard union expressibility (Errata E1-10)") +struct SchemaWildcardUnionExpressibilityTests { + private typealias Wild = PureXML.Schema.Wildcard + private typealias NSpace = PureXML.Schema.WildcardNamespace + + @Test("E1-10 union of not(a) with a set follows rule 5.1") + func test_unionNotWithSet() { + // 5.1.3: set excludes the negated name but includes absent -> NOT expressible. + #expect(Wild.unionNamespace(.notNamespace("a"), .enumerated(["", "b", "c"])) == nil) + // 5.1.1: set includes both the name and absent -> any. + #expect(Wild.unionNamespace(.notNamespace("a"), .enumerated(["a", "", "b"])) == NSpace.any) + // 5.1.2: set includes the name, not absent -> not(absent). + #expect(Wild.unionNamespace(.notNamespace("a"), .enumerated(["a", "b"])) == NSpace.notAbsent) + // 5.1.4: set includes neither -> unchanged not(a). + #expect(Wild.unionNamespace(.notNamespace("a"), .enumerated(["b", "c"])) == NSpace.notNamespace("a")) + } + + @Test("E1-10 intersection of two not forms") + func test_intersectNotForms() { + // Different negated names: the true not({a,b}) is not expressible (src-ct.4, + // a deferred rule); to avoid over-rejecting valid cross-namespace attributes + // on an accepted schema, intersection stays lenient (not(lhs)). + #expect(Wild.intersectNamespace(.notNamespace("a"), .notNamespace("b")) == NSpace.notNamespace("a")) + // Same negated name is unchanged. + #expect(Wild.intersectNamespace(.notNamespace("a"), .notNamespace("a")) == NSpace.notNamespace("a")) + // not(a) and not(absent) -> named except a. + #expect(Wild.intersectNamespace(.notNamespace("a"), .notAbsent) == NSpace.notNamespace("a")) + } + + /// FP-guard (the critic's case): a type intersecting its own `##other` with an + /// imported attribute group's `##other` (different target namespaces) must still + /// admit an attribute in a THIRD namespace that both `##other`s allow, not drop + /// the wildcard. src-ct.4 (intersection not expressible) is a deferred rule, so + /// the schema is accepted and the lenient wildcard must not over-reject. + @Test("cross-namespace ##other intersection stays lenient, not dropped") + func test_crossNamespaceOtherIntersection() throws { + let imported = """ + + + + """ + let schema = try PureXML.Schema.Document(""" + + + + + + + + """, schemaLoader: { _ in imported }) + // urn:c is admitted by not(urn:a) AND not(urn:b): must NOT be rejected. + #expect(try schema.validate(#""#).isEmpty) + } + + /// XSTS wildZ013 (test328873i): `##other` extended with `##local b c` unions to + /// "everything except the target, plus absent" -> not expressible -> invalid. + @Test("an extension with a not-expressible attribute-wildcard union is rejected") + func test_notExpressibleUnionRejected() { + #expect(throws: (any Error).self) { + _ = try PureXML.Schema.Document(""" + + + + + + + + + + + + """) + } + } + + /// FP-guard: `##other` extended with `##targetNamespace b c` unions to not(absent) + /// (rule 5.1.2), which IS expressible, so the schema is valid and the resulting + /// wildcard admits a named attribute but not a no-namespace one. + @Test("an expressible union compiles and is applied correctly") + func test_expressibleUnionCompilesAndApplies() throws { + let schema = try PureXML.Schema.Document(""" + + + + + + + + + + + + """) + // not(absent): a named (urn:b) attribute is admitted... + #expect(try schema.validate(#""#).isEmpty) + // ...but a no-namespace (absent) attribute is rejected. + #expect(try !schema.validate(#""#).isEmpty) + } +} diff --git a/Tests/ValidationConfigurationTests.swift b/Tests/ValidationConfigurationTests.swift index fd84e40..712ec3d 100644 --- a/Tests/ValidationConfigurationTests.swift +++ b/Tests/ValidationConfigurationTests.swift @@ -115,6 +115,7 @@ struct ValidationConfigurationTests { "Simple-type varieties are declared consistently", "Notation declarations are valid", "Simple-type facets are valid for their base type", + "Attribute wildcard unions in type extensions are expressible", "Every schema reference resolves to a declared component", "Substitution-group members derive correctly from their head", ]) diff --git a/Tests/XSTSSuiteTests.swift b/Tests/XSTSSuiteTests.swift index 5085087..664a123 100644 --- a/Tests/XSTSSuiteTests.swift +++ b/Tests/XSTSSuiteTests.swift @@ -35,8 +35,9 @@ struct XSTSSuiteTests { /// unescaped `[` inside a pattern-facet character class is a syntax error per /// XSD Appendix F (RegexTest_993, RegexTest_1477); 13 -> 12, an unknown /// `\p{...}` category or block name is rejected now the block set is complete - /// (reK88, `\p{IsaA0-a9}`). - private let knownSchemaInvalidAccepted = 12 + /// (reK88, `\p{IsaA0-a9}`); 12 -> 11, an attribute-wildcard union in a type + /// extension that is not expressible per Errata E1-10 is rejected (wildZ013). + private let knownSchemaInvalidAccepted = 11 private let knownInstanceValidRejected = 0 /// Instance bucket (133 -> 22), per-step deltas in CHANGELOG.md: xsi:type must derive from the declared /// type; anyType cannot stand in for anySimpleType; an untyped substitutionGroup member inherits its head's @@ -47,9 +48,12 @@ struct XSTSSuiteTests { /// a nilled element may not have a fixed value constraint, cvc-elt.3.2.2 (addB065); /// an optional xs:all group present in the instance still requires its members (mgZ001); /// a defaulted/fixed IDREF/IDREFS must resolve to a matching ID, cvc-id (idZ012). - /// Latest (19 -> 17): a negated character-class subtraction excludes both the + /// 19 -> 17: a negated character-class subtraction excludes both the /// negated base and the subtrahend (`[^cde-[ag]]`), RegexTest_430 and _422. - private let knownInstanceInvalidAccepted = 17 + /// Latest (17 -> 15): the attribute-wildcard union of a type extension is + /// computed per Errata E1-10, so the resulting wildcard admits exactly the + /// right attributes (wildZ013a, wildZ013d). + private let knownInstanceInvalidAccepted = 15 /// Suspect instance tests excluded from the counts: the Ethiopic-digit `\d` /// cases contradict `\d` = `\p{Nd}` at Unicode 3.2 and are tracked as disputed. private static let specDivergentInstances: Set = ["reS17.v", "reS38.v", "reZ004v.v"] diff --git a/docs/release-hurdles.md b/docs/release-hurdles.md index f735533..8618fd8 100644 --- a/docs/release-hurdles.md +++ b/docs/release-hurdles.md @@ -6,8 +6,8 @@ evidence, what actually stands between the current engine and a 1.0 fit to put i front of a developer as the authority on XML/XSD correctness. Snapshot at time of writing: v0.2.0 released; full 2006-11-06 XSTS archive (14383 -groups) with `valid-schemas-rejected = 0`, `invalid-schemas-accepted = 12`, -`valid-instances-rejected = 0`, `invalid-instances-accepted = 17`. +groups) with `valid-schemas-rejected = 0`, `invalid-schemas-accepted = 11`, +`valid-instances-rejected = 0`, `invalid-instances-accepted = 15`. ## Status against the four stoppers @@ -16,7 +16,7 @@ groups) with `valid-schemas-rejected = 0`, `invalid-schemas-accepted = 12`, libxml2 differential over 9618 schemas. **Met.** This is the doc's bar to be "an authority"; it is cleared on the measured corpora. 2. **Silently accepting faulty input (false negatives).** `invalid-schemas-accepted - = 12`, `invalid-instances-accepted = 17`. **Not at 0.** This is the bulk of named + = 11`, `invalid-instances-accepted = 15`. **Not at 0.** This is the bulk of named milestone work (M2, M3) and the subject of Part 1 below. 3. **Uncharacterized correctness.** Differential harness vs libxml2 and a fuzz suite exist (M1, ~done). Gap: real-world corpora beyond XSTS, and the silent @@ -28,8 +28,8 @@ groups) with `valid-schemas-rejected = 0`, `invalid-schemas-accepted = 12`, ## The biggest hurdle, stated plainly -Driving the false-negative buckets (`invalid-schemas-accepted = 12`, -`invalid-instances-accepted = 17`) to zero is the largest remaining body of named +Driving the false-negative buckets (`invalid-schemas-accepted = 11`, +`invalid-instances-accepted = 15`) to zero is the largest remaining body of named work, and it is hard for a specific reason: **the ~95% already closed were the tractable cases; every case left fights the cardinal rule.** The remainder sits in shared-oracle / contested-spec / multi-document-composition territory where the fix