diff --git a/CHANGELOG.md b/CHANGELOG.md index 302fb3d..2e300d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- Two references to the same `` that overlap (`(g?, g?)`, `(g | g)`, `(g*, g)`) are now correctly rejected as a Unique Particle Attribution violation, as the inline forms `(x?, x?)` / `(x | x)` already were and as libxml2/Xerces reject them. The compositional determinism check summarises each group once and reused the group's node identity across references, so the distinct-identity overlap test could not see that two references are two expanded occurrences, and PureXML accepted these models (an invalid-schemas-accepted false negative). It also let the per-instance content matcher run on such a non-UPA model, where a multiply-referenced nullable group is exponential: a depth-18 optional doubling chain was accepted and then hung instance validation, a keystroke-time interactive-safety hang on a schema PureXML itself accepted. The fix dedupes each decision set (first set, sequence gap, `all` cross-follow) by particle identity and treats a same-identity collision as the conflict, while the repetition self-loop (`a*`, `(a*)*`) keeps its non-flagging dedupe so a single particle looping is not misread as two occurrences. This also bounds the check: the deduped first/follow sets stay `O(labels)` instead of `2^K`, so the same depth-40 chain is now rejected at compile in 11 ms and no instance is ever matched against it. A redefine self-extension (`` inside the redefinition of `T`) lists a type's own content twice through `effectiveModelNodes`; that assembly artifact is deduped before the walk so it is not read as a conflict (addB094, schZ007). Verified over the full W3C XSTS corpus in release: `valid-schemas-rejected` and `valid-instances-rejected` stay 0 and `invalid-*-accepted` are unchanged (the corpus does not exercise this shape in its pinned buckets), with an adversarial over-rejection critic finding no valid schema newly rejected across the group-overlap shape space. + - XSTS invalid-schemas-accepted (#145): a particle restriction whose restricting element has a union-typed base element now applies Type Derivation OK (Simple) (`cos-st-derived-ok`) rather than structural member subsetting. The NameAndTypeOK type clause (§3.9.6) requires the restricting element's type to be validly derived from the base element's type; PureXML accepted a *separately declared* union whenever its member types were a subset of the base union's members, which is not the spec's derivation relation. The restricting type is now valid over a union base only if it is the base type, derives from it through a declared restriction chain, or (clause 2.2.4) validly derives from one of the union's member types. Catches particlesIk011 (a fresh `union(integer)` is not a restriction of `union(integer, string)`): invalid-schemas-accepted 9 to 8, with valid-schemas-rejected held at 0 (verified over the full XSTS; the change is scoped to the element-restriction path, so no attribute or simple-type derivation check moved). Disclosed divergence: particlesIk025/Ik027, where the restricting type derives from a union *member*, are accepted per `cos-st-derived-ok` clause 2.2.4, a contested reading their XSTS `invalid` expectation does not share. ## [0.4.8] - 2026-06-27 diff --git a/Sources/Schema/ContentModelDeterminismCompositional.swift b/Sources/Schema/ContentModelDeterminismCompositional.swift index 3aa723b..39fd2aa 100644 --- a/Sources/Schema/ContentModelDeterminismCompositional.swift +++ b/Sources/Schema/ContentModelDeterminismCompositional.swift @@ -43,7 +43,16 @@ extension PureXML.Schema { /// sequence (a complexContent extension's effective content), or nil. static func violation(_ models: [XSDTree], _ bindings: [String: String], _ context: XSDContext) -> String? { var walk = CompWalk(bindings: bindings, context: context) - let summary = walk.sequence(models.map { walk.summarize($0, visiting: []) }) + // Dedupe the effective-model nodes by identity. A redefine self-extension + // (`` inside the redefinition of `T`) resolves its base to + // the redefinition itself, so `effectiveModelNodes` lists this type's own + // content twice: an assembly artifact, not two real occurrences. Counting the + // same node once keeps the genuine two-reference detection (a group referenced + // twice WITHIN a model) while not inventing a conflict from the duplication + // (addB094, schZ007). + var seenModel: Set = [] + let uniqueModels = models.filter { seenModel.insert(ObjectIdentifier($0)).inserted } + let summary = walk.sequence(uniqueModels.map { walk.summarize($0, visiting: []) }) // The root's entry decision set, plus the followpos of each item that can // end the whole model (nothing follows the root, so its followpos is just // its internal follow). @@ -51,6 +60,11 @@ extension PureXML.Schema { for entry in summary.lastFollow { walk.emit(entry.follow) } + // A same-particle collision (two references to one group) found during the + // walk, then the distinct-identity, same-label overlaps. + if let conflict = walk.identityConflict { + return "ambiguous content model: '\(conflict)' can be matched by more than one particle" + } for set in walk.decisionSets { if let conflict = CompWalk.overlap(set) { return "ambiguous content model: '\(conflict)' can be matched by more than one particle" @@ -68,26 +82,106 @@ private struct CompWalk { let context: PureXML.Schema.XSDContext var decisionSets: [[CompItem]] = [] var groupCache: [String: CompSummary] = [:] + /// A conflict label found because the SAME particle reaches one decision point from + /// two distinct contributions. A `` is summarized once and its items + /// reuse the group's node identity across references, so two references to the same + /// group (the `(g?, g?)` / `(g | g)` overlap) would otherwise be invisible to the + /// distinct-identity `overlap` check, even though XSD expands each reference into a + /// separate occurrence. Detecting the collision here both reports that conflict and, + /// because the merge dedupes, keeps a multiply-referenced nullable group's first and + /// follow sets `O(labels)` instead of `2^K`. + var identityConflict: String? mutating func emit(_ set: [CompItem]) { if set.count > 1 { decisionSets.append(set) } } + /// Dedupe a decision set (first set or follow set) by particle identity. A repeated + /// identity means the same particle reaches this point from two contributions, which + /// XSD treats as two expanded occurrences: record it as a conflict. The deduped set + /// is still handed to `overlap` for the distinct-identity, same-label case. + mutating func merge(_ lists: [CompItem]...) -> [CompItem] { + mergeList(lists) + } + + mutating func mergeList(_ lists: [[CompItem]]) -> [CompItem] { + var seen: Set = [] + var result: [CompItem] = [] + for item in lists.joined() { + if seen.insert(item.particle).inserted { + result.append(item) + } else if identityConflict == nil { + identityConflict = labelName(item.label) + } + } + return result + } + + /// Dedupe by particle identity WITHOUT flagging a conflict, for combinations where a + /// repeated identity is one particle looping back on itself, not two occurrences: a + /// repetition's loop edge (`a*`, and nesting like `(a*)*`) folds the body's first + /// into its own follow, which is legitimate. Only combinations of DISTINCT child + /// particles (first sets, sequence gaps, `all` cross-follows) use the flagging merge. + func dedupOnly(_ lists: [[CompItem]]) -> [CompItem] { + var seen: Set = [] + var result: [CompItem] = [] + for item in lists.joined() where seen.insert(item.particle).inserted { + result.append(item) + } + return result + } + + /// Dedupe `lastFollow` end-items by particle, unioning their follow sets, so a + /// multiply-referenced nullable group does not double the carried follow list at + /// every level. End-item identity collision is not itself a conflict (the overlap of + /// the two references is decided where they START, in a merged first set), so this + /// does not flag `identityConflict`. + mutating func mergeLastFollow(_ entries: [(item: CompItem, follow: [CompItem])]) -> [(item: CompItem, follow: [CompItem])] { + var indexByParticle: [ObjectIdentifier: Int] = [:] + var result: [(item: CompItem, follow: [CompItem])] = [] + for entry in entries { + if let index = indexByParticle[entry.item.particle] { + result[index].follow = dedupOnly([result[index].follow, entry.follow]) + } else { + indexByParticle[entry.item.particle] = result.count + result.append((entry.item, dedupOnly([entry.follow]))) + } + } + return result + } + + private func labelName(_ label: PureXML.Schema.TermLabel) -> String { + if case let .name(qname) = label { return qname.localName } + return "any" + } + mutating func summarize(_ node: PureXML.Model.TreeNode, visiting: Set) -> CompSummary { let (minimum, maximum) = PureXML.Schema.XSDNode.occurrence(node) if let maximum, maximum <= 0 { return CompSummary(nullable: true, first: [], lastFollow: []) } var body = bodySummary(node, visiting: visiting) - // A repeat edge (last -> first) is added for a true repetition (max unbounded or >= 2) with min < max. A - // fixed `{n,n}` clamps to copies sharing the body's particle identities (no new - // conflict); a `{0,1}` is optional, not a repetition. The loop folds the body's - // first into every end-item's follow, so a later gap (or the root) checks the - // combined loop+continuation decision set. - if minimum != maximum, maximum == nil || (maximum ?? 0) > 1 { - body.lastFollow = body.lastFollow.map { (item: $0.item, follow: $0.follow + body.first) } - } + // A repeat edge folds the body's first into every end-item's follow. Kept in a + // separate method, not inline: `summarize` recurses on nesting depth, so any + // locals it holds multiply the per-level stack frame, and a deep group chain + // already overflows a small (WASI) stack. The repeat-edge work runs as the + // recursion unwinds, so its frame is not on the deepest descent path. + body.lastFollow = withRepeatEdge(body, minimum: minimum, maximum: maximum) return CompSummary(nullable: minimum == 0 || body.nullable, first: body.first, lastFollow: body.lastFollow) } + /// Adds the repetition loop edge (`last -> first`) for a true repetition (max + /// unbounded or `>= 2`, with `min < max`). A fixed `{n,n}` clamps to copies sharing + /// the body's particle identities (no new conflict); `{0,1}` is optional, not a + /// repetition. The loop fold uses the non-flagging dedupe: a single particle looping + /// is not two occurrences. + private mutating func withRepeatEdge(_ body: CompSummary, minimum: Int, maximum: Int?) -> [(item: CompItem, follow: [CompItem])] { + guard minimum != maximum, maximum == nil || (maximum ?? 0) > 1 else { return body.lastFollow } + var looped: [(item: CompItem, follow: [CompItem])] = [] + for entry in body.lastFollow { + looped.append((entry.item, dedupOnly([entry.follow, body.first]))) + } + return looped + } + private mutating func bodySummary(_ node: PureXML.Model.TreeNode, visiting: Set) -> CompSummary { switch PureXML.Schema.XSDNode.localName(node) { case "element": @@ -129,29 +223,29 @@ private struct CompWalk { } guard !suffixFirst.isEmpty else { continue } for entry in parts[index].lastFollow { - emit(entry.follow + suffixFirst) + emit(merge(entry.follow, suffixFirst)) } } - var first: [CompItem] = [] + var prefixFirsts: [[CompItem]] = [] for part in parts { - first.append(contentsOf: part.first) + prefixFirsts.append(part.first) if !part.nullable { break } } - var lastFollow: [(item: CompItem, follow: [CompItem])] = [] + var trailing: [(item: CompItem, follow: [CompItem])] = [] for part in parts.reversed() { - lastFollow.append(contentsOf: part.lastFollow) + trailing.append(contentsOf: part.lastFollow) if !part.nullable { break } } - return CompSummary(nullable: parts.allSatisfy(\.nullable), first: first, lastFollow: lastFollow) + return CompSummary(nullable: parts.allSatisfy(\.nullable), first: mergeList(prefixFirsts), lastFollow: mergeLastFollow(trailing)) } private mutating func choice(_ parts: [CompSummary]) -> CompSummary { - let first = parts.flatMap(\.first) + let first = mergeList(parts.map(\.first)) emit(first) return CompSummary( nullable: parts.isEmpty || parts.contains(where: \.nullable), first: first, - lastFollow: parts.flatMap(\.lastFollow), + lastFollow: mergeLastFollow(parts.flatMap(\.lastFollow)), ) } @@ -161,19 +255,19 @@ private struct CompWalk { /// follow is recorded in `followlast` so a parent gap combines it with the suffix /// (mirrors `ContentModelDeterminism.all`). private mutating func all(_ parts: [CompSummary]) -> CompSummary { - let first = parts.flatMap(\.first) + let first = mergeList(parts.map(\.first)) emit(first) var lastFollow: [(item: CompItem, follow: [CompItem])] = [] for (index, part) in parts.enumerated() { let others = parts.enumerated().filter { $0.offset != index }.flatMap(\.element.first) for entry in part.lastFollow { - lastFollow.append((entry.item, entry.follow + others)) + lastFollow.append((entry.item, merge(entry.follow, others))) } } return CompSummary( nullable: parts.isEmpty || parts.allSatisfy(\.nullable), first: first, - lastFollow: lastFollow, + lastFollow: mergeLastFollow(lastFollow), ) } diff --git a/Tests/ContentModelDeterminismGroupReferenceTests.swift b/Tests/ContentModelDeterminismGroupReferenceTests.swift new file mode 100644 index 0000000..b874a50 --- /dev/null +++ b/Tests/ContentModelDeterminismGroupReferenceTests.swift @@ -0,0 +1,88 @@ +import Testing +@testable import PureXML + +/// Two references to the same `` are two expanded occurrences for Unique +/// Particle Attribution, so when they overlap (`(g?, g?)`, `(g | g)`) the model is +/// ambiguous, exactly as the inline `(x?, x?)` / `(x | x)` is. The compositional +/// determinism check summarizes a group once and reused its node identity across +/// references, so the distinct-identity overlap test could not see the conflict and +/// PureXML accepted these where libxml2/Xerces reject them. That also let the per- +/// instance content matcher run on a non-UPA model, where a multiply-referenced +/// nullable group is exponential, a keystroke-time hang on an accepted schema. The +/// fix (dedup decision sets by identity, treat a same-identity collision as the +/// conflict) closes the gap and bounds the check. +@Suite("UPA on multiply-referenced group overlap") +struct DeterminismGroupReferenceTests { + private let header = "" + private let groupG = "" + + private func accepts(_ body: String) -> Bool { + (try? PureXML.Schema.Document(header + body + "")) != nil + } + + private func model(_ content: String, withGroup: Bool = true) -> String { + (withGroup ? groupG : "") + "\(content)" + } + + @Test("two overlapping references to one group are rejected as ambiguous") + func test_overlappingGroupReferencesRejected() { + // (g?, g?): a leading element of g matches the first or the second reference. + #expect(!accepts(model(""))) + // (g | g): a leading element of g matches either branch. + #expect(!accepts(model(""))) + // (g*, g): after a run of g, the next could be the repeat or the trailing g. + #expect(!accepts(model(""))) + } + + @Test("references to one group that do NOT overlap stay valid (no over-rejection)") + func test_nonOverlappingGroupReferencesAccepted() { + // (g, g): a required g then a required g is a deterministic two-element sequence. + #expect(accepts(model(""))) + // (g, g*): g then zero-or-more g is g repeated, unambiguous (the a+ shape). + #expect(accepts(model(""))) + // A single group reference, repeated, is one particle looping: valid. + #expect(accepts(model(""))) + } + + @Test("nested repetition of a single element is not a same-particle conflict") + func test_nestedSingleElementRepetitionAccepted() { + // ((a{0,M}){0,M}){0,M}: one element 'a' nested in repetitions is a*, unambiguous. + // The repeat-edge self-loop must not be read as two occurrences (#192 shape). + let content = "" + + "" + + "" + #expect(accepts(model(content, withGroup: false))) + } + + /// `g0=(g1?,g1?) … g{K-1}=(gK?,gK?), gK=(x?)`: every level optional and referenced + /// twice. Inlined this is the `2^K` overlap that hung the matcher; the determinism + /// check now rejects it in work linear in the depth, so no instance is ever matched + /// against it (the interactive-safety payoff). + private func optionalDoublingSchema(depth: Int) -> String { + var parts = [header] + for level in 0 ..< depth { + parts.append("" + + "" + + "") + } + parts.append("") + parts.append("") + parts.append("") + return parts.joined() + } + + // The determinism check is the linear-in-depth rejection's home, but it recurses on + // nesting depth, so a depth-40 chain overflows WASI's small single-threaded stack + // (the recursive check/builder depth-safety on WASI is tracked separately). The bound + // is platform-independent, so this runs on the large-stack hosts (macOS, Linux); the + // shallow correctness tests above cover the fix on WASI too. + #if !os(WASI) + @Test("a deep multiply-referenced nullable group chain is rejected in bounded work", .timeLimit(.minutes(1))) + func test_deepOptionalDoublingRejectedBounded() { + // 2^40 inlined occurrences: it must be rejected by the linear-in-depth check, + // not explored. Reaching this test's end at all is the bound (the time limit + // turns a regression to the exponential back into a failure, not a hang). + #expect((try? PureXML.Schema.Document(optionalDoublingSchema(depth: 40))) == nil) + } + #endif +}