fix(schema): content-free restriction of emptiable base, substitution-group declaration order

Two more Particle Valid (Restriction) over-rejections.

A derived particle that can never contribute a child (its own maxOccurs="0", or
a group whose every member is content-free) accepts only the empty sequence, so
it validly restricts any emptiable base. Checked before the structural cases,
which previously recursed into the members of a maxOccurs="0" group and rejected
it for their own occurrences (particlesW006).

Substitution-group expansion now lists members in declaration order: the closure
walk used a popLast() stack that reversed them, so a reference to a head expanded
to choice(head, m2, m1) and an order-preserving RecurseLax restriction by
choice(m1, m2) could not map (elemZ027a). The choice/choice restriction stays
order-preserving, as the W3C suite requires (choice(b|a) is not a valid
restriction of choice(a|b), particlesT002); only the expansion order is fixed.
The expansion is shared with instance validation and the determinism check, both
order-neutral.

XSTS: valid schemas rejected 4 to 2, no other bucket raised. The pinned XSTS
baselines (never asserted under a plain swift test, the suite being opt-in) are
corrected to the measured 2/520/180/160.
This commit is contained in:
Mihaela Mihaljevic 2026-06-14 22:48:12 +02:00
parent 828ed195f0
commit 9a1bb652a7
6 changed files with 90 additions and 15 deletions

View File

@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Two more Particle Valid (Restriction) over-rejections closed (#148). A derived particle that can never contribute a child (its own `maxOccurs="0"`, or a group whose every member is content-free) accepts only the empty sequence, so it is now recognized as a valid restriction of any emptiable base before the structural cases run; previously a `maxOccurs="0"` group whose members carried their own non-trivial occurrences was wrongly recursed into and rejected (`particlesW006`). And substitution-group expansion now lists members in declaration order: the closure walk used a `popLast()` stack that reversed them, so a reference to a head expanded to `choice(head, m2, m1)` and an order-preserving RecurseLax restriction by `choice(m1, m2)` could not map (`elemZ027a`). The choice/choice restriction stays order-preserving, as the W3C suite requires (a `choice(b|a)` is not a valid restriction of `choice(a|b)`, `particlesT002`); only the expansion order is corrected. The expansion is shared with instance validation and the determinism check, both order-neutral, so neither is affected. XSTS: valid schemas rejected 4 to 2, no other bucket raised. (The pinned XSTS baselines, which the opt-in suite never asserts under a plain `swift test`, were also corrected to the measured 2/520/180/160.)
- An `<xs:element ref="p:foo">` now resolves to the namespace its prefix is bound to, not the schema's target namespace (#164). The reference's QName is resolved by standard XML namespace rules against the schema root's `xmlns:` bindings: a bound prefix gives its namespace (so a reference to an imported element lands in the imported namespace), an unprefixed name resolves through the default namespace, and a name with no applicable binding is in no namespace. Previously the prefix was stripped and the element was assumed to be in the target namespace, which mis-qualified every reference to an imported element: a restriction of an `##other` wildcard by such a reference was wrongly rejected (the element looked like it was in the target namespace, which `##other` forbids), and the same mis-qualification rejected valid instance documents whose elements were correctly in the imported namespace. Because the content model is shared between schema-restriction checks and instance validation, the fix improves both. XSTS: valid schemas rejected 32 to 4, invalid schemas accepted 526 to 520, valid instances rejected 214 to 180, invalid instances accepted held at 160. Disclosed limitations, unchanged by this fix: a prefix re-bound on an inner element (rather than the schema root) is not honored, substitution-group members are still resolved to the target namespace (cross-namespace substitution is not modeled), and a dangling reference is not yet flagged as a schema error.
- Particle Valid (Restriction) now derives its pairwise check over the normalized content model, and applies occurrence subsumption only where the two particles' own ranges are comparable (#163). The W3C restriction algorithm is defined over a content model with pointless particles eliminated, so both the derived and base particles are first normalized (`maxOccurs="0"` members dropped, a `{1,1}` single-member group unwrapped to its member, a nested same-compositor `{1,1}` group spliced into its parent), all language-preserving. The outer occurrence-range check (`rangeSubsumed`) was being applied to every pairing, including the cross-kind ones where it compares mismatched ranges: a derived element's range against a base group's, or a derived group's against a base wildcard's. It is now applied only to the same-kind pairings (element/element, wildcard/wildcard, group/group) and element/wildcard; for element/group (RecurseAsIfGroup), group/element, and group/wildcard the occurrence is matched inside the case, against a member or as an effective total range. This unblocks a single-member derived sequence narrowing an occurrence (`sequence(e1{2,3})` restricting `sequence(e1{1,3}, ...)`, the `e1{2,3}` no longer compared to the base sequence's own `{1,1}`). A group restricting a wildcard (NSRecurseCheckCardinality) is now valid when every leaf it can contain is admitted by the wildcard and the group's effective total occurrence range lies within the wildcard's; a group restricting a single element is bounded the same way, by that element's name, type derivation, and occurrence range. XSTS: valid schemas rejected 45 to 32 (13 correct schemas no longer wrongly rejected, the `particlesHa` family of element-restricts-wildcard cardinality among them), invalid schemas accepted 528 to 526 (the cardinality bound also catches two over-wide restrictions), no other bucket raised.
- Particle Valid (Restriction) no longer over-rejects valid restrictions (#148). Three corrections, all language-preserving: a particle that can never occur (`maxOccurs="0"`, a "pointless particle") is removed from both the derived and base content before mapping, so it no longer consumes a base particle and starves the ones after it (e.g. `(e1 maxOccurs="0", e2)` restricting `(any)` is now valid); a content-free restriction (an empty `<sequence/>`, or a group whose members all cannot occur) is a valid restriction of an EMPTY base, since both accept only the empty sequence; and a derived group that reduces to empty after that removal is a valid restriction of its base only when the base is itself emptiable (by structure or its own occurrence), closing a vacuous-acceptance gap the removal would otherwise open. XSTS: valid schemas rejected 72 to 45 (27 correct schemas no longer wrongly rejected), with one schema (`addB32`) moving to invalid-accepted because its only invalidity is a malformed imported document PureXML does not yet load (tracked by #161), not the restriction.

View File

@ -56,6 +56,15 @@ extension PureXML.Schema {
/// `sequence(e1{1,3},...)` once the pointless single-member group is
/// normalized to the bare `e1{2,3}` (`3 > 1` against the base sequence).
static func valid(_ restricted: Particle, _ base: Particle, _ types: [String: ElementType], _ derivation: [String: TypeDerivation]) -> Bool {
// A restriction that can never contribute a child (its own `maxOccurs=0`,
// or a group whose every member is content-free) accepts only the empty
// sequence, so it is a valid restriction of any emptiable base. Checked
// before the structural cases, which would otherwise recurse into members
// that never occur and wrongly reject (e.g. a `maxOccurs="0"` sequence
// whose members have their own non-trivial occurrences).
if contentFree(restricted) {
return emptiable(base)
}
switch (restricted.term, base.term) {
case let (.element(restrictedName, _, restrictedTypeName), .element(baseName, _, baseTypeName)):
return rangeSubsumed(restricted, base)
@ -89,14 +98,9 @@ extension PureXML.Schema {
wildcardMax: base.maxOccurs,
)
case let (.group(restrictedGroup), .group(baseGroup)):
// If every member of the derived group is a pointless (maxOccurs=0)
// particle, the group accepts only the empty sequence; that is a valid
// restriction iff the base particle is itself emptiable (by structure
// or its own occurrence). `emptiable(base)` accounts for both, which a
// structural check inside `groupValid` could not.
if restrictedGroup.particles.allSatisfy({ $0.maxOccurs == 0 }) {
return emptiable(base)
}
// A content-free derived group was already handled above; here the
// derived group can contribute content, so its outer occurrence must be
// subsumed and its members must map onto the base's.
return rangeSubsumed(restricted, base) && groupValid(restrictedGroup, baseGroup, types, derivation)
case let (.group, .wildcard(wildcard)):
// NSRecurseCheckCardinality: every leaf the group can contain is
@ -126,6 +130,10 @@ extension PureXML.Schema {
case (.sequence, .sequence), (.all, .all):
return recurse(restrictedParticles, baseParticles, skippedMustBeEmptiable: true, types, derivation)
case (.choice, .choice):
// RecurseLax: an order-preserving mapping (W3C requires it; a choice
// `(b|a)` is NOT a valid restriction of `(a|b)`), but skipped base
// branches need not be emptiable. The base's substitution-group
// expansion must therefore be in document order for this to be right.
return recurse(restrictedParticles, baseParticles, skippedMustBeEmptiable: false, types, derivation)
case (.sequence, .choice):
// MapAndSum (simplified): each restricted particle fits some branch.

View File

@ -110,13 +110,21 @@ extension PureXML.Schema {
}
var closure: [String: [String]] = [:]
for head in direct.keys {
// Breadth-first in document order: a `popLast()` stack would reverse the
// members, but the substitution-group expansion must stay in declaration
// order. The order-preserving RecurseLax restriction check relies on it
// (a base `choice(head, m1, m2)` must list `m1` before `m2` so a derived
// `choice(m1, m2)` maps in order).
var members: [String] = []
var stack = direct[head] ?? []
var queue = direct[head] ?? []
var seen: Set<String> = []
while let member = stack.popLast() {
var index = 0
while index < queue.count {
let member = queue[index]
index += 1
guard seen.insert(member).inserted else { continue }
members.append(member)
stack += direct[member] ?? []
queue += direct[member] ?? []
}
closure[head] = members
}

View File

@ -0,0 +1,57 @@
@testable import PureXML
import Testing
@Suite("XSD restriction: content-free and substitution-group order")
struct XSDRestrictionContentFreeAndOrderTests {
private func compiles(_ source: String) -> Bool {
(try? PureXML.Schema.Document(source)) != nil
}
@Test("A never-occurring (maxOccurs=0) group restricts an emptiable base, but not a required one")
func test_contentFreeRestriction() {
// particlesW006: a sequence with maxOccurs=0 accepts only the empty sequence,
// whatever its members' own occurrences, so it validly restricts a base that
// also never occurs (or is otherwise emptiable).
let base = "<xs:complexType name=\"B\"><xs:sequence minOccurs=\"0\" maxOccurs=\"0\">"
+ "<xs:element name=\"e1\" maxOccurs=\"3\"/><xs:element name=\"e2\" maxOccurs=\"3\"/></xs:sequence></xs:complexType>"
let restriction = "<xs:complexType name=\"R\"><xs:complexContent><xs:restriction base=\"B\">"
+ "<xs:sequence minOccurs=\"0\" maxOccurs=\"0\">"
+ "<xs:element name=\"e1\" minOccurs=\"4\" maxOccurs=\"5\"/><xs:element name=\"e2\" minOccurs=\"3\" maxOccurs=\"3\"/></xs:sequence>"
+ "</xs:restriction></xs:complexContent></xs:complexType>"
#expect(compiles("<xs:schema xmlns:xs=\"http://www.w3.org/2001/XMLSchema\">\(base)\(restriction)</xs:schema>"))
// But a content-free restriction of a REQUIRED base is invalid: the base
// requires content the empty restriction cannot supply.
let requiredBase = "<xs:complexType name=\"B\"><xs:sequence><xs:element name=\"e1\"/></xs:sequence></xs:complexType>"
let emptyRestriction = "<xs:complexType name=\"R\"><xs:complexContent><xs:restriction base=\"B\">"
+ "<xs:sequence minOccurs=\"0\" maxOccurs=\"0\"><xs:element name=\"e1\"/></xs:sequence>"
+ "</xs:restriction></xs:complexContent></xs:complexType>"
#expect(!compiles("<xs:schema xmlns:xs=\"http://www.w3.org/2001/XMLSchema\">\(requiredBase)\(emptyRestriction)</xs:schema>"))
}
@Test("A choice of substitution-group members restricts a reference to the head, in declaration order")
func test_substitutionGroupChoiceOrder() {
// elemZ027a: a ref to a head expands to choice(head, m1, m2) in declaration
// order; a derived choice(m1, m2) maps onto it in order (RecurseLax).
let decls = "<xs:element name=\"head\"/><xs:element name=\"m1\" substitutionGroup=\"head\"/>"
+ "<xs:element name=\"m2\" substitutionGroup=\"head\"/>"
let base = "<xs:complexType name=\"base\"><xs:sequence><xs:element ref=\"head\"/></xs:sequence></xs:complexType>"
func derived(_ first: String, _ second: String) -> String {
"<xs:complexType name=\"derived\"><xs:complexContent><xs:restriction base=\"base\"><xs:sequence>"
+ "<xs:choice><xs:element ref=\"\(first)\"/><xs:element ref=\"\(second)\"/></xs:choice>"
+ "</xs:sequence></xs:restriction></xs:complexContent></xs:complexType>"
}
let head = "<xs:schema xmlns:xs=\"http://www.w3.org/2001/XMLSchema\">\(decls)\(base)"
#expect(compiles(head + derived("m1", "m2") + "</xs:schema>"))
}
@Test("Reordering a base choice's branches is not a valid restriction (RecurseLax is order-preserving)")
func test_choiceReorderRejected() {
// particlesT002: choice(c2, c1) is NOT a valid restriction of choice(c1, c2).
let base = "<xs:complexType name=\"B\"><xs:sequence><xs:choice>"
+ "<xs:element name=\"c1\"/><xs:element name=\"c2\"/></xs:choice><xs:element name=\"foo\"/></xs:sequence></xs:complexType>"
let reordered = "<xs:complexType name=\"R\"><xs:complexContent><xs:restriction base=\"B\"><xs:sequence><xs:choice>"
+ "<xs:element name=\"c2\"/><xs:element name=\"c1\"/></xs:choice><xs:element name=\"foo\"/></xs:sequence>"
+ "</xs:restriction></xs:complexContent></xs:complexType>"
#expect(!compiles("<xs:schema xmlns:xs=\"http://www.w3.org/2001/XMLSchema\">\(base)\(reordered)</xs:schema>"))
}
}

View File

@ -28,9 +28,9 @@ struct XSTSSuiteTests {
/// `swift test` are unaffected; run it with `swift test -c release --filter
/// XSTS` (debug is far slower and the corpus is large). Per-case deviations
/// are written to /tmp/xsts-failures.txt for the burn-down.
private let knownSchemaValidRejected = 32
private let knownSchemaInvalidAccepted = 526
private let knownInstanceValidRejected = 214
private let knownSchemaValidRejected = 2
private let knownSchemaInvalidAccepted = 520
private let knownInstanceValidRejected = 180
private let knownInstanceInvalidAccepted = 160
@Test("Every XSTS case behaves: compile, reject, validate, invalidate")

View File

@ -9,7 +9,7 @@ irreducible tail. Do not deviate to new features while this is open.
| Bucket | Count | Reading |
|---|---|---|
| valid-schemas-rejected | **72 -> 4** | production target is 0: rejecting a correct schema is the most user-hostile defect (#148). Particle-restriction over-rejection fixed: pointless `maxOccurs=0` particles removed, content-free restriction of an EMPTY base, empty-derived emptiable-base, and (#163) the pairwise check over the normalized content model with occurrence subsumption applied only to same-kind pairings (cross-kind element/group, group/element, group/wildcard match occurrence inside, by effective total range / member). #164 resolved element-ref namespaces by prefix binding (the `particlesJj/Jk/Jm/Jn/Js`, `particlesQ/R` cluster, -28). Remaining 4 (`elemZ027_a`, `particlesV003`, `particlesW006`, `particlesZ001`). |
| valid-schemas-rejected | **72 -> 4** | production target is 0: rejecting a correct schema is the most user-hostile defect (#148). Particle-restriction over-rejection fixed: pointless `maxOccurs=0` particles removed, content-free restriction of an EMPTY base, empty-derived emptiable-base, and (#163) the pairwise check over the normalized content model with occurrence subsumption applied only to same-kind pairings (cross-kind element/group, group/element, group/wildcard match occurrence inside, by effective total range / member). #164 resolved element-ref namespaces by prefix binding (the `particlesJj/Jk/Jm/Jn/Js`, `particlesQ/R` cluster, -28). Content-free (`maxOccurs=0`) derived groups and substitution-group declaration order closed `particlesW006`/`elemZ027_a`. Remaining 2 (`particlesV003`, `particlesZ001`): MapAndSum effective-total-range for a group restricting a repeated base choice (deferred, a careful larger change). |
| **invalid-schemas-accepted** | **2461 -> 520** | we rarely catch a bad one (facet, default/fixed value, particle-restriction name+type, id, structural content-model, UPA determinism, content order, identity-constraint XPath-subset, wildcard-namespace, final/block, simpleType content, group/wildcard and group/element cardinality, element-ref namespace landed). |
| valid-instances-rejected | **233 -> 180** | (instance side; #164's element-ref namespace fix is shared with instance validation, -34) |
| invalid-instances-accepted | 160 | |