fix(schema): normalize content model, scope occurrence subsumption in particle restriction

Particle Valid (Restriction) derives its pairwise check over the spec's
normalized content model: pointless particles are eliminated (maxOccurs=0
members dropped, a {1,1} single-member group unwrapped, a nested
same-compositor {1,1} group spliced), all language-preserving. The outer
occurrence-range check was wrongly applied to the cross-kind pairings, where
it compares a derived particle's own range against a base group's or
wildcard's. It is now applied only to the same-kind pairings (element/element,
wildcard/wildcard, group/group) and element/wildcard; element/group
(RecurseAsIfGroup), group/element, and group/wildcard match occurrence inside
the case, against a member or as an effective total range.

A group restricting a wildcard (NSRecurseCheckCardinality) is valid when every
leaf is admitted and the effective total occurrence range is within the
wildcard's; a group restricting a single element is bounded the same way, by
that element's name, type derivation, and occurrence range.

XSTS: valid schemas rejected 45 to 32, invalid schemas accepted 528 to 526,
no other bucket raised.

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Particle Valid (Restriction) now derives its pairwise check over the normalized content model, and applies occurrence subsumption only where the two particles' own ranges are comparable (#163). The W3C restriction algorithm is defined over a content model with pointless particles eliminated, so both the derived and base particles are first normalized (`maxOccurs="0"` members dropped, a `{1,1}` single-member group unwrapped to its member, a nested same-compositor `{1,1}` group spliced into its parent), all language-preserving. The outer occurrence-range check (`rangeSubsumed`) was being applied to every pairing, including the cross-kind ones where it compares mismatched ranges: a derived element's range against a base group's, or a derived group's against a base wildcard's. It is now applied only to the same-kind pairings (element/element, wildcard/wildcard, group/group) and element/wildcard; for element/group (RecurseAsIfGroup), group/element, and group/wildcard the occurrence is matched inside the case, against a member or as an effective total range. This unblocks a single-member derived sequence narrowing an occurrence (`sequence(e1{2,3})` restricting `sequence(e1{1,3}, ...)`, the `e1{2,3}` no longer compared to the base sequence's own `{1,1}`). A group restricting a wildcard (NSRecurseCheckCardinality) is now valid when every leaf it can contain is admitted by the wildcard and the group's effective total occurrence range lies within the wildcard's; a group restricting a single element is bounded the same way, by that element's name, type derivation, and occurrence range. XSTS: valid schemas rejected 45 to 32 (13 correct schemas no longer wrongly rejected, the `particlesHa` family of element-restricts-wildcard cardinality among them), invalid schemas accepted 528 to 526 (the cardinality bound also catches two over-wide restrictions), no other bucket raised.
 - Particle Valid (Restriction) no longer over-rejects valid restrictions (#148). Three corrections, all language-preserving: a particle that can never occur (`maxOccurs="0"`, a "pointless particle") is removed from both the derived and base content before mapping, so it no longer consumes a base particle and starves the ones after it (e.g. `(e1 maxOccurs="0", e2)` restricting `(any)` is now valid); a content-free restriction (an empty `<sequence/>`, or a group whose members all cannot occur) is a valid restriction of an EMPTY base, since both accept only the empty sequence; and a derived group that reduces to empty after that removal is a valid restriction of its base only when the base is itself emptiable (by structure or its own occurrence), closing a vacuous-acceptance gap the removal would otherwise open. XSTS: valid schemas rejected 72 to 45 (27 correct schemas no longer wrongly rejected), with one schema (`addB32`) moving to invalid-accepted because its only invalidity is a malformed imported document PureXML does not yet load (tracked by #161), not the restriction.
 - A `simpleType` must contain exactly one of `restriction`, `list`, or `union` (#158). Per XSD 1.0 Datatypes its content is `(annotation?, (restriction | list | union))`, but the membership table admitted each name without the cardinality, so an empty `simpleType` or one combining two derivations (two `union`s, a `union` and a `list`, two `restriction`s) was accepted (the stB family). The exactly-one rule is now enforced. XSTS: invalid schemas accepted 537 to 527, no other bucket raised (valid schemas rejected held at 72).
 - The `final` and `block` derivation-control attributes are now checked against their value space (#158). Each is `#all` or a whitespace list of method tokens, but the admitted tokens depend on the component: an element's `final` is `extension`/`restriction` and its `block` adds `substitution`; a complexType's `final`/`block` are `extension`/`restriction`; a simpleType's `final` is `list`/`union`/`restriction`. The tokens were parsed leniently (unknown tokens ignored, `substitution` admitted everywhere), so `final="substitution"` on an element, a misspelled or miscased token, or `#all` mixed with other tokens was accepted (the elemF/st_final families). The list is now validated against the component's exact set; `finalDefault`/`blockDefault` (the wider schema-level value space) and foreign attributes are left alone. XSTS: invalid schemas accepted 580 to 537, no other bucket raised (valid schemas rejected held at 72).

Sources/Schema/ParticleNormalization.swift

@@ -0,0 +1,55 @@
+extension PureXML.Schema.Particle {
+    /// Eliminates pointless particles so the restriction algorithm sees the spec's
+    /// normalized content model: a `maxOccurs=0` member contributes nothing and is
+    /// dropped; a nested same-compositor `{1,1}` group is spliced into its parent;
+    /// a `{1,1}` group with a single member is unwrapped to that member.
+    /// Language-preserving.
+    func normalized() -> Self {
+        guard case let .group(group) = term else { return self }
+        var members: [Self] = []
+        for child in group.particles {
+            let normalized = child.normalized()
+            if normalized.maxOccurs == 0 { continue }
+            let splice = normalized.minOccurs == 1 && normalized.maxOccurs == 1
+            if case let .group(inner) = normalized.term, inner.compositor == group.compositor, splice {
+                members.append(contentsOf: inner.particles)
+            } else {
+                members.append(normalized)
+            }
+        }
+        if members.count == 1, minOccurs == 1, maxOccurs == 1 {
+            return members[0]
+        }
+        return Self(minOccurs: minOccurs, maxOccurs: maxOccurs, term: .group(.init(compositor: group.compositor, particles: members)))
+    }
+
+    /// The minimum number of elements this particle can contribute (effective total
+    /// range, lower bound): a leaf is its `minOccurs`; a sequence/all sums its
+    /// members, a choice takes the smallest; times the particle's own `minOccurs`.
+    func effectiveOccurrenceMin() -> Int {
+        switch term {
+        case .element, .wildcard:
+            return minOccurs
+        case let .group(group):
+            let inner: Int = switch group.compositor {
+            case .choice: group.particles.map { $0.effectiveOccurrenceMin() }.min() ?? 0
+            case .sequence, .all: group.particles.map { $0.effectiveOccurrenceMin() }.reduce(0, +)
+            }
+            return minOccurs * inner
+        }
+    }
+
+    /// The maximum number of elements this particle can contribute (nil = unbounded).
+    func effectiveOccurrenceMax() -> Int? {
+        switch term {
+        case .element, .wildcard:
+            return maxOccurs
+        case let .group(group):
+            let childMaxes = group.particles.map { $0.effectiveOccurrenceMax() }
+            guard let particleMax = maxOccurs, !childMaxes.contains(where: { $0 == nil }) else { return nil }
+            let maxes = childMaxes.compactMap(\.self)
+            let inner = group.compositor == .choice ? (maxes.max() ?? 0) : maxes.reduce(0, +)
+            return particleMax * inner
+        }
+    }
+}

Sources/Schema/ParticleRestriction.swift

@@ -31,26 +31,63 @@ extension PureXML.Schema {
             case let (.elementOnly(restrictedParticle), .elementOnly(baseParticle)),
                  let (.elementOnly(restrictedParticle), .mixed(baseParticle)),
                  let (.mixed(restrictedParticle), .mixed(baseParticle)):
-                valid(restrictedParticle, baseParticle, types, derivation)
+                // The W3C restriction algorithm is defined over the normalized content
+                // model (pointless particles eliminated): a `maxOccurs=0` member
+                // removed, a `{1,1}` single-member group unwrapped, a nested
+                // same-compositor `{1,1}` group spliced. Without this, e.g. a base
+                // `sequence(any{2,3})` is not seen as the wildcard it equals.
+                valid(restrictedParticle.normalized(), baseParticle.normalized(), types, derivation)
                     ? nil
                     : "the restricted content model is not a subset of the base's"
             }
         }
 
         /// Whether `restricted` accepts a subset of `base` (the pairwise check).
+        ///
+        /// Occurrence subsumption (`rangeSubsumed`) is applied only to the
+        /// same-kind pairings (element/element, wildcard/wildcard, group/group) and
+        /// element/wildcard, where the two particles' own occurrence ranges are
+        /// directly comparable. For the cross-kind pairings (element/group and
+        /// group/element via RecurseAsIfGroup; group/wildcard via
+        /// NSRecurseCheckCardinality) the occurrence is matched *inside* the case,
+        /// against a member or as an effective total range, never the derived
+        /// particle's own occurrence against the base group/wildcard's. Applying the
+        /// outer check there over-rejects: e.g. `sequence(e1{2,3})` restricting
+        /// `sequence(e1{1,3},...)` once the pointless single-member group is
+        /// normalized to the bare `e1{2,3}` (`3 > 1` against the base sequence).
         static func valid(_ restricted: Particle, _ base: Particle, _ types: [String: ElementType], _ derivation: [String: TypeDerivation]) -> Bool {
-            guard rangeSubsumed(restricted, base) else { return false }
             switch (restricted.term, base.term) {
             case let (.element(restrictedName, _, restrictedTypeName), .element(baseName, _, baseTypeName)):
-                return sameName(restrictedName, baseName) && elementTypeRestrictionOK(restrictedTypeName, baseTypeName, types, derivation)
+                return rangeSubsumed(restricted, base)
+                    && sameName(restrictedName, baseName)
+                    && elementTypeRestrictionOK(restrictedTypeName, baseTypeName, types, derivation)
             case let (.element(name, _, _), .wildcard(wildcard)):
-                return wildcard.admits(name)
+                return rangeSubsumed(restricted, base) && wildcard.admits(name)
             case let (.wildcard(restrictedWildcard), .wildcard(baseWildcard)):
-                return narrows(restrictedWildcard, baseWildcard)
+                return rangeSubsumed(restricted, base) && narrows(restrictedWildcard, baseWildcard)
             case let (.element, .group(baseGroup)):
                 // RecurseAsIfGroup: the lone element, as a one-particle sequence,
-                // against the base group.
-                return groupValid(Group(compositor: .sequence, particles: [restricted.withUnitRange()]), baseGroup, types, derivation)
+                // against the base group. The element's occurrence is matched against
+                // a base member inside `groupValid`, not against the base group's own
+                // occurrence, so no outer `rangeSubsumed` here.
+                return groupValid(Group(compositor: .sequence, particles: [restricted]), baseGroup, types, derivation)
+            case let (.group, .element(baseName, _, baseTypeName)):
+                // The mirror of RecurseAsIfGroup, by the same cardinality argument as
+                // group/wildcard: a base element accepts only its own name, its own
+                // occurrence count of times. The derived group is a valid restriction
+                // iff every leaf it can contain is that element (same name, and a type
+                // that derives from the base's) AND the group's effective total
+                // occurrence range is within the base element's. Comparing the derived
+                // group's *own* occurrence to the element's would be wrong (a derived
+                // `sequence(a,a)` emits two a's at occurrence {1,1}); the effective
+                // range is the count that must be bounded.
+                return leavesRestrict(restricted, toElement: baseName, baseTypeName, types, derivation)
+                    && rangeWithinWildcard(
+                        derivedMin: restricted.effectiveOccurrenceMin(),
+                        derivedMax: restricted.effectiveOccurrenceMax(),
+                        wildcardMin: base.minOccurs,
+                        wildcardMax: base.maxOccurs,
+                    )
             case let (.group(restrictedGroup), .group(baseGroup)):
                 // If every member of the derived group is a pointless (maxOccurs=0)
                 // particle, the group accepts only the empty sequence; that is a valid
@@ -60,11 +97,19 @@ extension PureXML.Schema {
                 if restrictedGroup.particles.allSatisfy({ $0.maxOccurs == 0 }) {
                     return emptiable(base)
                 }
-                return groupValid(restrictedGroup, baseGroup, types, derivation)
-            case let (.group(restrictedGroup), .wildcard(wildcard)):
-                // NSRecurseCheckCardinality, namespace part: every leaf element
-                // the group can contain must be admitted by the wildcard.
-                return leafNames(of: restrictedGroup).allSatisfy { wildcard.admits($0) }
+                return rangeSubsumed(restricted, base) && groupValid(restrictedGroup, baseGroup, types, derivation)
+            case let (.group, .wildcard(wildcard)):
+                // NSRecurseCheckCardinality: every leaf the group can contain is
+                // admitted by the wildcard (an element by name, a nested wildcard by
+                // narrowing), AND the group's effective total occurrence range is
+                // within the wildcard's occurrence range.
+                return leavesAdmitted(restricted, by: wildcard)
+                    && rangeWithinWildcard(
+                        derivedMin: restricted.effectiveOccurrenceMin(),
+                        derivedMax: restricted.effectiveOccurrenceMax(),
+                        wildcardMin: base.minOccurs,
+                        wildcardMax: base.maxOccurs,
+                    )
             default:
                 return false
             }
@@ -271,22 +316,44 @@ extension PureXML.Schema {
             lhs.localName == rhs.localName && (lhs.namespaceURI ?? "") == (rhs.namespaceURI ?? "")
         }
 
-        /// Every element name a group can contain (its alphabet).
-        private static func leafNames(of group: Group) -> [PureXML.Model.QualifiedName] {
-            group.particles.flatMap { particle -> [PureXML.Model.QualifiedName] in
-                switch particle.term {
-                case let .element(name, _, _): [name]
-                case let .group(inner): leafNames(of: inner)
-                case .wildcard: []
-                }
+        /// Whether every leaf a particle can contain is admitted by `wildcard`: an
+        /// element by name, a nested wildcard by narrowing.
+        private static func leavesAdmitted(_ particle: Particle, by wildcard: Wildcard) -> Bool {
+            switch particle.term {
+            case let .element(name, _, _): wildcard.admits(name)
+            case let .wildcard(inner): narrows(inner, wildcard)
+            case let .group(group): group.particles.allSatisfy { leavesAdmitted($0, by: wildcard) }
             }
         }
+
+        /// Whether every leaf a particle can contain is the named base element (same
+        /// name, and a type that derives from the base element's). A wildcard leaf is
+        /// rejected: it can match names other than the base element's, so it is not a
+        /// subset. Used when the base content model is a single element.
+        private static func leavesRestrict(
+            _ particle: Particle,
+            toElement baseName: PureXML.Model.QualifiedName,
+            _ baseTypeName: String?,
+            _ types: [String: ElementType],
+            _ derivation: [String: TypeDerivation],
+        ) -> Bool {
+            switch particle.term {
+            case let .element(name, _, typeName):
+                sameName(name, baseName) && elementTypeRestrictionOK(typeName, baseTypeName, types, derivation)
+            case .wildcard:
+                false
+            case let .group(group):
+                group.particles.allSatisfy { leavesRestrict($0, toElement: baseName, baseTypeName, types, derivation) }
+            }
+        }
+
+        /// Whether a derived effective total range fits within a base wildcard's
+        /// occurrence range: at least the wildcard's minimum, at most its maximum.
+        private static func rangeWithinWildcard(derivedMin: Int, derivedMax: Int?, wildcardMin: Int, wildcardMax: Int?) -> Bool {
+            guard derivedMin >= wildcardMin else { return false }
+            guard let wildcardMax else { return true }
+            guard let derivedMax else { return false }
+            return derivedMax <= wildcardMax
+        }
     }
 }
-
-private extension PureXML.Schema.Particle {
-    /// A copy occurring exactly once, for RecurseAsIfGroup's wrapping.
-    func withUnitRange() -> PureXML.Schema.Particle {
-        .init(minOccurs: minOccurs, maxOccurs: maxOccurs, term: term)
-    }
-}

Tests/XSDParticleRestrictionTests.swift

@@ -136,6 +136,76 @@ struct XSDParticleRestrictionTests {
         #expect(error?.contains("not a valid restriction") == true)
     }
 
+    @Test("A single-member derived sequence narrowing an occurrence restricts a wider base sequence")
+    func test_singleMemberSequenceNarrowsOccurrence() {
+        // particlesW008: the derived sequence has one member, so it normalizes to the
+        // bare element e1{2,3}; that must still be matched as a member of the base
+        // sequence (RecurseAsIfGroup), not have its {2,3} compared to the base
+        // sequence's own {1,1} occurrence.
+        #expect(compiles(
+            base: "<xs:sequence><xs:element name=\"e1\" maxOccurs=\"3\"/>"
+                + "<xs:element name=\"e2\" minOccurs=\"0\" maxOccurs=\"3\"/>"
+                + "<xs:element name=\"e3\" minOccurs=\"0\" maxOccurs=\"3\"/></xs:sequence>",
+            restriction: "<xs:sequence><xs:element name=\"e1\" minOccurs=\"2\" maxOccurs=\"3\"/></xs:sequence>",
+        ))
+        // particlesM002: a derived single-branch choice normalizes to the bare element
+        // and still restricts a choice of sequences whose first branch begins with it.
+        #expect(compiles(
+            base: "<xs:choice><xs:sequence minOccurs=\"1\" maxOccurs=\"3\">"
+                + "<xs:element name=\"c1\" minOccurs=\"2\" maxOccurs=\"2\"/><xs:element name=\"c2\" minOccurs=\"0\"/></xs:sequence>"
+                + "<xs:sequence><xs:element name=\"d1\"/><xs:element name=\"d2\" minOccurs=\"0\"/></xs:sequence></xs:choice>",
+            restriction: "<xs:choice><xs:element name=\"c1\" minOccurs=\"2\" maxOccurs=\"2\"/></xs:choice>",
+        ))
+    }
+
+    @Test("A group restricting a wildcard is valid only when its effective count fits the wildcard's range")
+    func test_groupRestrictsWildcardCardinality() {
+        // particlesHa: a sequence of named elements restricts a wildcard when every leaf
+        // is admitted AND the effective total count lies within the wildcard's range.
+        #expect(compiles(
+            base: "<xs:sequence><xs:any namespace=\"##any\" processContents=\"lax\" minOccurs=\"2\" maxOccurs=\"3\"/></xs:sequence>",
+            restriction: "<xs:sequence><xs:element name=\"a\"/><xs:element name=\"b\"/></xs:sequence>",
+        ))
+        // Too many leaves: three elements exceed the wildcard's maxOccurs of 2.
+        #expect(restrictionError(
+            base: "<xs:sequence><xs:any namespace=\"##any\" processContents=\"lax\" minOccurs=\"2\" maxOccurs=\"2\"/></xs:sequence>",
+            restriction: "<xs:sequence><xs:element name=\"a\"/><xs:element name=\"b\"/><xs:element name=\"c\"/></xs:sequence>",
+        )?.contains("not a valid restriction") == true)
+        // Too few leaves: one element is below the wildcard's minOccurs of 2.
+        #expect(restrictionError(
+            base: "<xs:sequence><xs:any namespace=\"##any\" processContents=\"lax\" minOccurs=\"2\" maxOccurs=\"3\"/></xs:sequence>",
+            restriction: "<xs:sequence><xs:element name=\"a\"/></xs:sequence>",
+        )?.contains("not a valid restriction") == true)
+    }
+
+    @Test("A group restricting a base that is a single element is bounded by that element's range")
+    func test_groupRestrictsSingleElement() {
+        // mgE014: base sequence(title?) normalizes to the bare element title{0,1};
+        // the derived sequence(title{0,0}) normalizes to the empty group, which
+        // accepts only the empty sequence, valid against an emptiable element.
+        #expect(compiles(
+            base: "<xs:sequence><xs:element name=\"title\" minOccurs=\"0\"/></xs:sequence>",
+            restriction: "<xs:sequence><xs:element name=\"title\" minOccurs=\"0\" maxOccurs=\"0\"/></xs:sequence>",
+        ))
+        // A derived choice repeated within the element's range is valid (the base
+        // element, normalized from a single-member sequence, is compositor-agnostic).
+        #expect(compiles(
+            base: "<xs:sequence><xs:element name=\"a\" maxOccurs=\"3\"/></xs:sequence>",
+            restriction: "<xs:choice maxOccurs=\"2\"><xs:element name=\"a\"/></xs:choice>",
+        ))
+        // A derived group whose effective count exceeds the base element's maxOccurs is
+        // rejected: sequence(a){1,10} emits up to ten a's, the base allows three.
+        #expect(restrictionError(
+            base: "<xs:sequence><xs:element name=\"a\" maxOccurs=\"3\"/></xs:sequence>",
+            restriction: "<xs:sequence maxOccurs=\"10\"><xs:element name=\"a\"/></xs:sequence>",
+        )?.contains("not a valid restriction") == true)
+        // A derived leaf with a different name than the base element is rejected.
+        #expect(restrictionError(
+            base: "<xs:sequence><xs:element name=\"a\" maxOccurs=\"3\"/></xs:sequence>",
+            restriction: "<xs:choice maxOccurs=\"2\"><xs:element name=\"b\"/></xs:choice>",
+        )?.contains("not a valid restriction") == true)
+    }
+
     @Test("Conformance: the restriction corpus passes through the harness")
     func test_conformanceCases() {
         let valid = compiles(

Tests/XSTSSuiteTests.swift

@@ -28,8 +28,8 @@ struct XSTSSuiteTests {
     /// `swift test` are unaffected; run it with `swift test -c release --filter
     /// XSTS` (debug is far slower and the corpus is large). Per-case deviations
     /// are written to /tmp/xsts-failures.txt for the burn-down.
-    private let knownSchemaValidRejected = 45
-    private let knownSchemaInvalidAccepted = 528
+    private let knownSchemaValidRejected = 32
+    private let knownSchemaInvalidAccepted = 526
     private let knownInstanceValidRejected = 214
     private let knownInstanceInvalidAccepted = 160
 

docs/schema-validity-burndown.md

@@ -9,8 +9,8 @@ irreducible tail. Do not deviate to new features while this is open.
 
 | Bucket | Count | Reading |
 |---|---|---|
-| valid-schemas-rejected | **72 -> 45** | production target is 0: rejecting a correct schema is the most user-hostile defect (#148). Particle-restriction over-rejection fixed: pointless `maxOccurs=0` particles removed, content-free restriction of an EMPTY base, empty-derived emptiable-base. |
-| **invalid-schemas-accepted** | **2461 -> 528** | we rarely catch a bad one (facet, default/fixed value, particle-restriction name+type, id, structural content-model, UPA determinism, content order, identity-constraint XPath-subset, wildcard-namespace, final/block, simpleType content landed). The 527->528 step is `addB32`, invalid only via a malformed import (#161), unmasked by removing a wrong restriction rejection. |
+| valid-schemas-rejected | **72 -> 32** | production target is 0: rejecting a correct schema is the most user-hostile defect (#148). Particle-restriction over-rejection fixed: pointless `maxOccurs=0` particles removed, content-free restriction of an EMPTY base, empty-derived emptiable-base, and (#163) the pairwise check over the normalized content model with occurrence subsumption applied only to same-kind pairings (cross-kind element/group, group/element, group/wildcard match occurrence inside, by effective total range / member). Remaining 32 are mostly element-ref namespace resolution (`particlesJj/Jk/Jm`, #164). |
+| **invalid-schemas-accepted** | **2461 -> 526** | we rarely catch a bad one (facet, default/fixed value, particle-restriction name+type, id, structural content-model, UPA determinism, content order, identity-constraint XPath-subset, wildcard-namespace, final/block, simpleType content, group/wildcard and group/element cardinality landed). |
 | valid-instances-rejected | 233 | (instance side, separately tracked) |
 | invalid-instances-accepted | 165 | |