From 9a1bb652a73ff70346e34a7e10c392fb3b909ac4 Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Sun, 14 Jun 2026 22:48:12 +0200 Subject: [PATCH] fix(schema): content-free restriction of emptiable base, substitution-group declaration order Two more Particle Valid (Restriction) over-rejections. A derived particle that can never contribute a child (its own maxOccurs="0", or a group whose every member is content-free) accepts only the empty sequence, so it validly restricts any emptiable base. Checked before the structural cases, which previously recursed into the members of a maxOccurs="0" group and rejected it for their own occurrences (particlesW006). Substitution-group expansion now lists members in declaration order: the closure walk used a popLast() stack that reversed them, so a reference to a head expanded to choice(head, m2, m1) and an order-preserving RecurseLax restriction by choice(m1, m2) could not map (elemZ027a). The choice/choice restriction stays order-preserving, as the W3C suite requires (choice(b|a) is not a valid restriction of choice(a|b), particlesT002); only the expansion order is fixed. The expansion is shared with instance validation and the determinism check, both order-neutral. XSTS: valid schemas rejected 4 to 2, no other bucket raised. The pinned XSTS baselines (never asserted under a plain swift test, the suite being opt-in) are corrected to the measured 2/520/180/160. --- CHANGELOG.md | 2 + Sources/Schema/ParticleRestriction.swift | 24 +++++--- Sources/Schema/XSDContext.swift | 14 ++++- .../XSDRestrictionContentFreeOrderTests.swift | 57 +++++++++++++++++++ Tests/XSTSSuiteTests.swift | 6 +- docs/schema-validity-burndown.md | 2 +- 6 files changed, 90 insertions(+), 15 deletions(-) create mode 100644 Tests/XSDRestrictionContentFreeOrderTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b21a02..4107fb7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- Two more Particle Valid (Restriction) over-rejections closed (#148). A derived particle that can never contribute a child (its own `maxOccurs="0"`, or a group whose every member is content-free) accepts only the empty sequence, so it is now recognized as a valid restriction of any emptiable base before the structural cases run; previously a `maxOccurs="0"` group whose members carried their own non-trivial occurrences was wrongly recursed into and rejected (`particlesW006`). And substitution-group expansion now lists members in declaration order: the closure walk used a `popLast()` stack that reversed them, so a reference to a head expanded to `choice(head, m2, m1)` and an order-preserving RecurseLax restriction by `choice(m1, m2)` could not map (`elemZ027a`). The choice/choice restriction stays order-preserving, as the W3C suite requires (a `choice(b|a)` is not a valid restriction of `choice(a|b)`, `particlesT002`); only the expansion order is corrected. The expansion is shared with instance validation and the determinism check, both order-neutral, so neither is affected. XSTS: valid schemas rejected 4 to 2, no other bucket raised. (The pinned XSTS baselines, which the opt-in suite never asserts under a plain `swift test`, were also corrected to the measured 2/520/180/160.) + - An `` now resolves to the namespace its prefix is bound to, not the schema's target namespace (#164). The reference's QName is resolved by standard XML namespace rules against the schema root's `xmlns:` bindings: a bound prefix gives its namespace (so a reference to an imported element lands in the imported namespace), an unprefixed name resolves through the default namespace, and a name with no applicable binding is in no namespace. Previously the prefix was stripped and the element was assumed to be in the target namespace, which mis-qualified every reference to an imported element: a restriction of an `##other` wildcard by such a reference was wrongly rejected (the element looked like it was in the target namespace, which `##other` forbids), and the same mis-qualification rejected valid instance documents whose elements were correctly in the imported namespace. Because the content model is shared between schema-restriction checks and instance validation, the fix improves both. XSTS: valid schemas rejected 32 to 4, invalid schemas accepted 526 to 520, valid instances rejected 214 to 180, invalid instances accepted held at 160. Disclosed limitations, unchanged by this fix: a prefix re-bound on an inner element (rather than the schema root) is not honored, substitution-group members are still resolved to the target namespace (cross-namespace substitution is not modeled), and a dangling reference is not yet flagged as a schema error. - Particle Valid (Restriction) now derives its pairwise check over the normalized content model, and applies occurrence subsumption only where the two particles' own ranges are comparable (#163). The W3C restriction algorithm is defined over a content model with pointless particles eliminated, so both the derived and base particles are first normalized (`maxOccurs="0"` members dropped, a `{1,1}` single-member group unwrapped to its member, a nested same-compositor `{1,1}` group spliced into its parent), all language-preserving. The outer occurrence-range check (`rangeSubsumed`) was being applied to every pairing, including the cross-kind ones where it compares mismatched ranges: a derived element's range against a base group's, or a derived group's against a base wildcard's. It is now applied only to the same-kind pairings (element/element, wildcard/wildcard, group/group) and element/wildcard; for element/group (RecurseAsIfGroup), group/element, and group/wildcard the occurrence is matched inside the case, against a member or as an effective total range. This unblocks a single-member derived sequence narrowing an occurrence (`sequence(e1{2,3})` restricting `sequence(e1{1,3}, ...)`, the `e1{2,3}` no longer compared to the base sequence's own `{1,1}`). A group restricting a wildcard (NSRecurseCheckCardinality) is now valid when every leaf it can contain is admitted by the wildcard and the group's effective total occurrence range lies within the wildcard's; a group restricting a single element is bounded the same way, by that element's name, type derivation, and occurrence range. XSTS: valid schemas rejected 45 to 32 (13 correct schemas no longer wrongly rejected, the `particlesHa` family of element-restricts-wildcard cardinality among them), invalid schemas accepted 528 to 526 (the cardinality bound also catches two over-wide restrictions), no other bucket raised. - Particle Valid (Restriction) no longer over-rejects valid restrictions (#148). Three corrections, all language-preserving: a particle that can never occur (`maxOccurs="0"`, a "pointless particle") is removed from both the derived and base content before mapping, so it no longer consumes a base particle and starves the ones after it (e.g. `(e1 maxOccurs="0", e2)` restricting `(any)` is now valid); a content-free restriction (an empty ``, or a group whose members all cannot occur) is a valid restriction of an EMPTY base, since both accept only the empty sequence; and a derived group that reduces to empty after that removal is a valid restriction of its base only when the base is itself emptiable (by structure or its own occurrence), closing a vacuous-acceptance gap the removal would otherwise open. XSTS: valid schemas rejected 72 to 45 (27 correct schemas no longer wrongly rejected), with one schema (`addB32`) moving to invalid-accepted because its only invalidity is a malformed imported document PureXML does not yet load (tracked by #161), not the restriction. diff --git a/Sources/Schema/ParticleRestriction.swift b/Sources/Schema/ParticleRestriction.swift index 89ecf7c..ffab94d 100644 --- a/Sources/Schema/ParticleRestriction.swift +++ b/Sources/Schema/ParticleRestriction.swift @@ -56,6 +56,15 @@ extension PureXML.Schema { /// `sequence(e1{1,3},...)` once the pointless single-member group is /// normalized to the bare `e1{2,3}` (`3 > 1` against the base sequence). static func valid(_ restricted: Particle, _ base: Particle, _ types: [String: ElementType], _ derivation: [String: TypeDerivation]) -> Bool { + // A restriction that can never contribute a child (its own `maxOccurs=0`, + // or a group whose every member is content-free) accepts only the empty + // sequence, so it is a valid restriction of any emptiable base. Checked + // before the structural cases, which would otherwise recurse into members + // that never occur and wrongly reject (e.g. a `maxOccurs="0"` sequence + // whose members have their own non-trivial occurrences). + if contentFree(restricted) { + return emptiable(base) + } switch (restricted.term, base.term) { case let (.element(restrictedName, _, restrictedTypeName), .element(baseName, _, baseTypeName)): return rangeSubsumed(restricted, base) @@ -89,14 +98,9 @@ extension PureXML.Schema { wildcardMax: base.maxOccurs, ) case let (.group(restrictedGroup), .group(baseGroup)): - // If every member of the derived group is a pointless (maxOccurs=0) - // particle, the group accepts only the empty sequence; that is a valid - // restriction iff the base particle is itself emptiable (by structure - // or its own occurrence). `emptiable(base)` accounts for both, which a - // structural check inside `groupValid` could not. - if restrictedGroup.particles.allSatisfy({ $0.maxOccurs == 0 }) { - return emptiable(base) - } + // A content-free derived group was already handled above; here the + // derived group can contribute content, so its outer occurrence must be + // subsumed and its members must map onto the base's. return rangeSubsumed(restricted, base) && groupValid(restrictedGroup, baseGroup, types, derivation) case let (.group, .wildcard(wildcard)): // NSRecurseCheckCardinality: every leaf the group can contain is @@ -126,6 +130,10 @@ extension PureXML.Schema { case (.sequence, .sequence), (.all, .all): return recurse(restrictedParticles, baseParticles, skippedMustBeEmptiable: true, types, derivation) case (.choice, .choice): + // RecurseLax: an order-preserving mapping (W3C requires it; a choice + // `(b|a)` is NOT a valid restriction of `(a|b)`), but skipped base + // branches need not be emptiable. The base's substitution-group + // expansion must therefore be in document order for this to be right. return recurse(restrictedParticles, baseParticles, skippedMustBeEmptiable: false, types, derivation) case (.sequence, .choice): // MapAndSum (simplified): each restricted particle fits some branch. diff --git a/Sources/Schema/XSDContext.swift b/Sources/Schema/XSDContext.swift index 49af3c3..3f46613 100644 --- a/Sources/Schema/XSDContext.swift +++ b/Sources/Schema/XSDContext.swift @@ -110,13 +110,21 @@ extension PureXML.Schema { } var closure: [String: [String]] = [:] for head in direct.keys { + // Breadth-first in document order: a `popLast()` stack would reverse the + // members, but the substitution-group expansion must stay in declaration + // order. The order-preserving RecurseLax restriction check relies on it + // (a base `choice(head, m1, m2)` must list `m1` before `m2` so a derived + // `choice(m1, m2)` maps in order). var members: [String] = [] - var stack = direct[head] ?? [] + var queue = direct[head] ?? [] var seen: Set = [] - while let member = stack.popLast() { + var index = 0 + while index < queue.count { + let member = queue[index] + index += 1 guard seen.insert(member).inserted else { continue } members.append(member) - stack += direct[member] ?? [] + queue += direct[member] ?? [] } closure[head] = members } diff --git a/Tests/XSDRestrictionContentFreeOrderTests.swift b/Tests/XSDRestrictionContentFreeOrderTests.swift new file mode 100644 index 0000000..2b1ebb2 --- /dev/null +++ b/Tests/XSDRestrictionContentFreeOrderTests.swift @@ -0,0 +1,57 @@ +@testable import PureXML +import Testing + +@Suite("XSD restriction: content-free and substitution-group order") +struct XSDRestrictionContentFreeAndOrderTests { + private func compiles(_ source: String) -> Bool { + (try? PureXML.Schema.Document(source)) != nil + } + + @Test("A never-occurring (maxOccurs=0) group restricts an emptiable base, but not a required one") + func test_contentFreeRestriction() { + // particlesW006: a sequence with maxOccurs=0 accepts only the empty sequence, + // whatever its members' own occurrences, so it validly restricts a base that + // also never occurs (or is otherwise emptiable). + let base = "" + + "" + let restriction = "" + + "" + + "" + + "" + #expect(compiles("\(base)\(restriction)")) + // But a content-free restriction of a REQUIRED base is invalid: the base + // requires content the empty restriction cannot supply. + let requiredBase = "" + let emptyRestriction = "" + + "" + + "" + #expect(!compiles("\(requiredBase)\(emptyRestriction)")) + } + + @Test("A choice of substitution-group members restricts a reference to the head, in declaration order") + func test_substitutionGroupChoiceOrder() { + // elemZ027a: a ref to a head expands to choice(head, m1, m2) in declaration + // order; a derived choice(m1, m2) maps onto it in order (RecurseLax). + let decls = "" + + "" + let base = "" + func derived(_ first: String, _ second: String) -> String { + "" + + "" + + "" + } + let head = "\(decls)\(base)" + #expect(compiles(head + derived("m1", "m2") + "")) + } + + @Test("Reordering a base choice's branches is not a valid restriction (RecurseLax is order-preserving)") + func test_choiceReorderRejected() { + // particlesT002: choice(c2, c1) is NOT a valid restriction of choice(c1, c2). + let base = "" + + "" + let reordered = "" + + "" + + "" + #expect(!compiles("\(base)\(reordered)")) + } +} diff --git a/Tests/XSTSSuiteTests.swift b/Tests/XSTSSuiteTests.swift index 4905d89..6907b50 100644 --- a/Tests/XSTSSuiteTests.swift +++ b/Tests/XSTSSuiteTests.swift @@ -28,9 +28,9 @@ struct XSTSSuiteTests { /// `swift test` are unaffected; run it with `swift test -c release --filter /// XSTS` (debug is far slower and the corpus is large). Per-case deviations /// are written to /tmp/xsts-failures.txt for the burn-down. - private let knownSchemaValidRejected = 32 - private let knownSchemaInvalidAccepted = 526 - private let knownInstanceValidRejected = 214 + private let knownSchemaValidRejected = 2 + private let knownSchemaInvalidAccepted = 520 + private let knownInstanceValidRejected = 180 private let knownInstanceInvalidAccepted = 160 @Test("Every XSTS case behaves: compile, reject, validate, invalidate") diff --git a/docs/schema-validity-burndown.md b/docs/schema-validity-burndown.md index e1ba1e1..5fe5cfa 100644 --- a/docs/schema-validity-burndown.md +++ b/docs/schema-validity-burndown.md @@ -9,7 +9,7 @@ irreducible tail. Do not deviate to new features while this is open. | Bucket | Count | Reading | |---|---|---| -| valid-schemas-rejected | **72 -> 4** | production target is 0: rejecting a correct schema is the most user-hostile defect (#148). Particle-restriction over-rejection fixed: pointless `maxOccurs=0` particles removed, content-free restriction of an EMPTY base, empty-derived emptiable-base, and (#163) the pairwise check over the normalized content model with occurrence subsumption applied only to same-kind pairings (cross-kind element/group, group/element, group/wildcard match occurrence inside, by effective total range / member). #164 resolved element-ref namespaces by prefix binding (the `particlesJj/Jk/Jm/Jn/Js`, `particlesQ/R` cluster, -28). Remaining 4 (`elemZ027_a`, `particlesV003`, `particlesW006`, `particlesZ001`). | +| valid-schemas-rejected | **72 -> 4** | production target is 0: rejecting a correct schema is the most user-hostile defect (#148). Particle-restriction over-rejection fixed: pointless `maxOccurs=0` particles removed, content-free restriction of an EMPTY base, empty-derived emptiable-base, and (#163) the pairwise check over the normalized content model with occurrence subsumption applied only to same-kind pairings (cross-kind element/group, group/element, group/wildcard match occurrence inside, by effective total range / member). #164 resolved element-ref namespaces by prefix binding (the `particlesJj/Jk/Jm/Jn/Js`, `particlesQ/R` cluster, -28). Content-free (`maxOccurs=0`) derived groups and substitution-group declaration order closed `particlesW006`/`elemZ027_a`. Remaining 2 (`particlesV003`, `particlesZ001`): MapAndSum effective-total-range for a group restricting a repeated base choice (deferred, a careful larger change). | | **invalid-schemas-accepted** | **2461 -> 520** | we rarely catch a bad one (facet, default/fixed value, particle-restriction name+type, id, structural content-model, UPA determinism, content order, identity-constraint XPath-subset, wildcard-namespace, final/block, simpleType content, group/wildcard and group/element cardinality, element-ref namespace landed). | | valid-instances-rejected | **233 -> 180** | (instance side; #164's element-ref namespace fix is shared with instance validation, -34) | | invalid-instances-accepted | 160 | |