NodetteHQ/PureXML
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

perf(schema): make the UPA determinism check depth-safe on group-reference chains (#371)

Browse Source 
A flat chain of <xs:group> references is not bounded by the parser's maxDepth (the definitions are siblings, not nested), so the compositional UPA determinism check's lazy per-reference recursion overflowed the native 8 MB stack while compiling a legal schema. Group summaries form an acyclic graph, so they are now precomputed bottom-up in dependency order (GroupDependencyOrder): the walk's group lookup is a cache hit and recursion is bounded by a single group's own model nesting. The precomputed reachable set is identical to the lazy walk's, so the verdict is unchanged. Verified verdict-equivalent over the full W3C XSTS corpus, the libxml2 determinism differential (0 FP, same 3 FN), the full suite, and an adversarial critic. Residual, tracked: the particle compiler that inlines a group reference still overflows on a far longer chain; closing it needs a content-model nesting-depth limit with a located error.
This commit is contained in:
Mihaela Mihaljevic 2026-06-28 23:15:32 +02:00 committed by GitHub
parent ae37f493f9
commit 1f8d99adfc
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 222 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGELOG.md
View File

@ -25,6 +25,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- A deeply nested chain of `<xs:group>` references (`g0` referencing `g1` referencing `g2`, and so on) no longer overflows the stack in the Unique Particle Attribution determinism check. Such a chain is flat in the schema document, the group definitions being siblings rather than nested, so it is not bounded by the parser's `maxDepth`; the compositional check summarised each group lazily by recursing through the reference at its use site, so a long chain recursed once per link and overflowed the stack while compiling a legal (if hostile) schema, a keystroke-time interactive-safety crash on the primary platform. Group summaries form an acyclic graph (a circular group reference is rejected elsewhere), so they are now precomputed bottom-up in dependency order: every reachable group is summarised once, in an order where each group's own references are already cached, so the walk's group lookup is always a cache hit and recursion is bounded by a single group's model nesting (which `maxDepth` bounds), not by the chain length. A group summary is context-independent, so this only reorders the work; the set of decision sets it produces is identical and the accept/reject verdict does not depend on their order. Verified verdict-equivalent over the full W3C XSTS corpus in release (all four buckets unchanged: `valid-schemas-rejected` and `valid-instances-rejected` stay 0, `invalid-schemas-accepted` 8, `invalid-instances-accepted` 14), the generated-schema determinism differential against libxml2 unchanged (zero false positives, the same three named false negatives), and the full 1816-test suite. Disclosed residual, not silent: the separate particle-compiler pass that inlines a group reference into the instance content model still recurses on the same chain and overflows on a far longer one (around ten thousand links); closing it needs a content-model nesting-depth limit, analogous to `maxDepth`, that rejects such a pathological chain with a located error, and is tracked.
- A repeating model whose optional trailing particle and leading particle share a name, `(c, b?, c?)*`, is now correctly rejected as a Unique Particle Attribution violation, as libxml2/Xerces reject it: after `b` the next `c` could be the trailing `c?` or the looped leading `c`, two distinct particles. The compositional determinism check formed the internal sequence gap before the enclosing repetition's loop edge was added, so the combined `{trailing, leading}` followpos was never checked (an invalid-schemas-accepted false negative the finite XSTS corpus did not contain). The sequence now carries each trailing end-item's internal suffix in its `lastFollow`, so a parent gap or the loop edge forms the end-item's full followpos. The carried suffix is always a subset of the true followpos, so the check only catches more genuine conflicts and never over-rejects: the non-repeated `(c, b?, c?)` and disjoint shapes like `(a, b?)*` stay valid. Found by the new generated-schema determinism differential against libxml2; verified over the full W3C XSTS corpus in release (`valid-schemas-rejected`/`valid-instances-rejected` stay 0, `invalid-*-accepted` unchanged) with an adversarial over-rejection critic finding no valid schema newly rejected over 47 shapes. The related ambiguity through two references to one group in a repeating sequence stays an under-rejection (the cached group identity cannot distinguish a looped from a trailing occurrence without per-occurrence tracking), named in `DeterminismDifferentialTests`.
- Two references to the same `<xs:group>` that overlap (`(g?, g?)`, `(g | g)`, `(g*, g)`) are now correctly rejected as a Unique Particle Attribution violation, as the inline forms `(x?, x?)` / `(x | x)` already were and as libxml2/Xerces reject them. The compositional determinism check summarises each group once and reused the group's node identity across references, so the distinct-identity overlap test could not see that two references are two expanded occurrences, and PureXML accepted these models (an invalid-schemas-accepted false negative). It also let the per-instance content matcher run on such a non-UPA model, where a multiply-referenced nullable group is exponential: a depth-18 optional doubling chain was accepted and then hung instance validation, a keystroke-time interactive-safety hang on a schema PureXML itself accepted. The fix dedupes each decision set (first set, sequence gap, `all` cross-follow) by particle identity and treats a same-identity collision as the conflict, while the repetition self-loop (`a*`, `(a*)*`) keeps its non-flagging dedupe so a single particle looping is not misread as two occurrences. This also bounds the check: the deduped first/follow sets stay `O(labels)` instead of `2^K`, so the same depth-40 chain is now rejected at compile in 11 ms and no instance is ever matched against it. A redefine self-extension (`<extension base="T">` inside the redefinition of `T`) lists a type's own content twice through `effectiveModelNodes`; that assembly artifact is deduped before the walk so it is not read as a conflict (addB094, schZ007). Verified over the full W3C XSTS corpus in release: `valid-schemas-rejected` and `valid-instances-rejected` stay 0 and `invalid-*-accepted` are unchanged (the corpus does not exercise this shape in its pinned buckets), with an adversarial over-rejection critic finding no valid schema newly rejected across the group-overlap shape space.

67
Sources/Schema/ContentModelDeterminismCompositional.swift
View File

@ -1,7 +1,8 @@
/// One distinct particle occurrence in a content model: its source-node identity
/// and label. A particle never conflicts with a repetition of itself, so identity,
/// not label alone, decides conflict (per the UPA semantics). File-private.
private struct CompItem: Hashable {
/// not label alone, decides conflict (per the UPA semantics). Module-internal so the
/// conflict tests (``CompConflict``) can live in their own file.
struct CompItem: Hashable {
let particle: ObjectIdentifier
let label: PureXML.Schema.TermLabel
@ -52,6 +53,10 @@ extension PureXML.Schema {
// (addB094, schZ007).
var seenModel: Set<ObjectIdentifier> = []
let uniqueModels = models.filter { seenModel.insert(ObjectIdentifier($0)).inserted }
// Summarize every reachable group up front, in dependency order, so the walk
// below never recurses through a `<group ref>` chain (which is flat in the
// document and so unbounded by `maxDepth`); see `precomputeGroupSummaries`.
walk.precomputeGroupSummaries(uniqueModels)
let summary = walk.sequence(uniqueModels.map { walk.summarize($0, visiting: []) })
// The root's entry decision set, plus the followpos of each item that can
// end the whole model (nothing follows the root, so its followpos is just
@ -66,7 +71,7 @@ extension PureXML.Schema {
return "ambiguous content model: '\(conflict)' can be matched by more than one particle"
}
for set in walk.decisionSets {
if let conflict = CompWalk.overlap(set) {
if let conflict = CompConflict.overlap(set) {
return "ambiguous content model: '\(conflict)' can be matched by more than one particle"
}
}
@ -326,31 +331,6 @@ private struct CompWalk {
.first { ["sequence", "choice", "all"].contains(PureXML.Schema.XSDNode.localName($0) ?? "") }
}
// MARK: Conflict
static func overlap(_ set: [CompItem]) -> String? {
var nameOwner: [PureXML.Model.QualifiedName: ObjectIdentifier] = [:]
var wildcards: [(card: PureXML.Schema.Wildcard, particle: ObjectIdentifier)] = []
for item in set {
switch item.label {
case let .name(qname):
if let owner = nameOwner[qname], owner != item.particle { return qname.localName }
if nameOwner[qname] == nil { nameOwner[qname] = item.particle }
case let .wildcard(card):
wildcards.append((card, item.particle))
}
}
for (index, wildcard) in wildcards.enumerated() {
// A wildcard conflicts with any name it admits, from a distinct particle,
// and with any other wildcard it intersects, from a distinct particle.
if nameOwner.contains(where: { wildcard.card.admits($0.key) && $0.value != wildcard.particle }) { return "any" }
for other in wildcards[(index + 1)...] where other.particle != wildcard.particle && wildcardsOverlap(wildcard.card, other.card) {
return "any"
}
}
return nil
}
// MARK: Helpers (mirror ContentModelDeterminism)
private func elementQName(_ node: PureXML.Model.TreeNode) -> PureXML.Model.QualifiedName {
@ -371,20 +351,23 @@ private struct CompWalk {
return PureXML.Model.QualifiedName(localName: qname, namespaceURI: bindings[""])
}
static func wildcardsOverlap(_ lhs: PureXML.Schema.Wildcard, _ rhs: PureXML.Schema.Wildcard) -> Bool {
// Two wildcards overlap when some namespace name is admitted by both. Any
// two `not` forms always share infinitely many named namespaces; a `not`
// form overlaps a set when the set holds a named namespace it admits.
switch (lhs.namespace, rhs.namespace) {
case (.any, _), (_, .any): true
case let (.enumerated(lhsSet), .enumerated(rhsSet)): !lhsSet.isDisjoint(with: rhsSet)
case let (.enumerated(set), .notNamespace(name)), let (.notNamespace(name), .enumerated(set)):
set.contains { !$0.isEmpty && $0 != name }
case let (.enumerated(set), .notAbsent), let (.notAbsent, .enumerated(set)):
set.contains { !$0.isEmpty }
case (.notNamespace, .notNamespace), (.notNamespace, .notAbsent),
(.notAbsent, .notNamespace), (.notAbsent, .notAbsent):
true
// MARK: Group summaries (depth-safe precomputation)
/// Summarizes every group reachable from `models`, in dependency order, into
/// `groupCache`, so the main walk's `group(_:)` is always a cache hit and
/// `summarize` never recurses through a `<group ref>` chain. A group reference
/// chain is FLAT in the schema document (the definitions are siblings, not
/// nested), so it is NOT bounded by the parser's `maxDepth`; summarizing lazily
/// through references therefore recursed on the chain length and overflowed the
/// stack on a long (hostile but legal) chain. Each group's own model nesting is
/// document structure, so it stays `maxDepth`-bounded. The cached summary is the
/// SAME object the lazy walk would have produced (a group summary is
/// context-independent), so this only reorders the work; the decision sets it
/// emits are identical as a set, and the verdict does not depend on their order.
mutating func precomputeGroupSummaries(_ models: [PureXML.Model.TreeNode]) {
for name in GroupDependencyOrder.reachable(from: models, context: context) where groupCache[name] == nil {
guard let definition = context.groups[name], let model = modelChild(definition) else { continue }
groupCache[name] = summarize(model, visiting: [name])
}
}
}

47
Sources/Schema/ContentModelDeterminismConflict.swift Normal file
View File

@ -0,0 +1,47 @@
/// The overlap tests for a compositional-determinism decision set: a UPA violation
/// is two DISTINCT particles in one decision set that can match the same element
/// (same element name, or a wildcard admitting the other's name/namespace). Pure
/// functions over ``CompItem`` sets, factored out of ``CompWalk`` so the walk file
/// stays within its length budget; the walk accumulates decision sets and hands
/// each to ``overlap(_:)``.
enum CompConflict {
static func overlap(_ set: [CompItem]) -> String? {
var nameOwner: [PureXML.Model.QualifiedName: ObjectIdentifier] = [:]
var wildcards: [(card: PureXML.Schema.Wildcard, particle: ObjectIdentifier)] = []
for item in set {
switch item.label {
case let .name(qname):
if let owner = nameOwner[qname], owner != item.particle { return qname.localName }
if nameOwner[qname] == nil { nameOwner[qname] = item.particle }
case let .wildcard(card):
wildcards.append((card, item.particle))
}
}
for (index, wildcard) in wildcards.enumerated() {
// A wildcard conflicts with any name it admits, from a distinct particle,
// and with any other wildcard it intersects, from a distinct particle.
if nameOwner.contains(where: { wildcard.card.admits($0.key) && $0.value != wildcard.particle }) { return "any" }
for other in wildcards[(index + 1)...] where other.particle != wildcard.particle && wildcardsOverlap(wildcard.card, other.card) {
return "any"
}
}
return nil
}
static func wildcardsOverlap(_ lhs: PureXML.Schema.Wildcard, _ rhs: PureXML.Schema.Wildcard) -> Bool {
// Two wildcards overlap when some namespace name is admitted by both. Any
// two `not` forms always share infinitely many named namespaces; a `not`
// form overlaps a set when the set holds a named namespace it admits.
switch (lhs.namespace, rhs.namespace) {
case (.any, _), (_, .any): true
case let (.enumerated(lhsSet), .enumerated(rhsSet)): !lhsSet.isDisjoint(with: rhsSet)
case let (.enumerated(set), .notNamespace(name)), let (.notNamespace(name), .enumerated(set)):
set.contains { !$0.isEmpty && $0 != name }
case let (.enumerated(set), .notAbsent), let (.notAbsent, .enumerated(set)):
set.contains { !$0.isEmpty }
case (.notNamespace, .notNamespace), (.notNamespace, .notAbsent),
(.notAbsent, .notNamespace), (.notAbsent, .notAbsent):
true
}
}
}

85
Sources/Schema/ContentModelGroupDependency.swift Normal file
View File

@ -0,0 +1,85 @@
/// Orders the model groups reachable from a set of content models so each group
/// comes after every group it references. The compositional determinism check
/// summarizes groups in this order, so its recursive walk is always a cache hit at a
/// `<group ref>` and never recurses through a reference chain. A chain is FLAT in the
/// schema document (group definitions are siblings, not nested), so it is not bounded
/// by the parser's `maxDepth`; ordering and scanning here are iterative so neither
/// step recurses on the chain length either.
enum GroupDependencyOrder {
/// Reachable group names in dependency order (dependencies first), by an iterative
/// post-order DFS from the groups the `models` reference. A name already on the
/// current path is not re-entered, so a circular reference (illegal, rejected
/// elsewhere) terminates.
static func reachable(from models: [PureXML.Model.TreeNode], context: PureXML.Schema.XSDContext) -> [String] {
var roots: [String] = []
var seen: Set<String> = []
for model in models {
for ref in referencedGroupNames(in: model) where seen.insert(ref).inserted {
roots.append(ref)
}
}
var order: [String] = []
var visited: Set<String> = []
var onPath: Set<String> = []
var work: [(name: String, exit: Bool)] = roots.reversed().map { ($0, false) }
while let (name, exit) = work.popLast() {
if exit {
onPath.remove(name)
order.append(name)
continue
}
if visited.contains(name) { continue }
visited.insert(name)
onPath.insert(name)
work.append((name, true))
for ref in dependencies(of: name, context: context).reversed() where !visited.contains(ref) && !onPath.contains(ref) {
work.append((ref, false))
}
}
return order
}
/// The group names referenced within `node`, by an iterative tree walk (so scanning
/// a deep model does not itself recurse on depth). It descends compositors but stops
/// at an `<element>`, exactly as the determinism walk's `summarize` treats an element
/// as terminal: a group inside an element's inline complex type belongs to THAT type's
/// own content model (checked separately), not this one, and must not be drawn in.
private static func referencedGroupNames(in node: PureXML.Model.TreeNode) -> [String] {
var names: [String] = []
var stack = [node]
while let current = stack.popLast() {
let local = PureXML.Schema.XSDNode.localName(current)
if local == "group", let ref = PureXML.Schema.XSDNode.attribute(current, "ref") {
names.append(PureXML.Schema.XSDNode.stripPrefix(ref))
}
if local != "element" {
stack.append(contentsOf: PureXML.Schema.XSDNode.elementChildren(current))
}
}
return names
}
/// The groups `name` references. A redefined group's base-model references are added
/// ONLY when the redefinition actually self-references (`<group ref=name>` inside the
/// redefinition of `name`): that reference expands to the base model
/// (`CompWalk.redefineSelfReference`), making the base's groups part of the effective
/// content model, so they must be ordered first. When the redefinition does NOT
/// self-reference, the base is fully replaced and dead, and pulling its refs in would
/// summarise unreachable groups: surfacing a conflict in a dead group is a false
/// positive, so it is deliberately excluded. This keeps the reachable set identical
/// to the lazy walk's.
private static func dependencies(of name: String, context: PureXML.Schema.XSDContext) -> [String] {
guard let definition = context.groups[name], let model = modelChild(definition) else { return [] }
var refs = referencedGroupNames(in: model)
if refs.contains(name), context.redefinedGroups.contains(name), let base = context.baseGroups[name], let baseModel = modelChild(base) {
refs += referencedGroupNames(in: baseModel)
}
return refs
}
/// The `sequence`/`choice`/`all` model child of a group definition, if any.
private static func modelChild(_ definition: PureXML.Model.TreeNode) -> PureXML.Model.TreeNode? {
PureXML.Schema.XSDNode.elementChildren(definition)
.first { ["sequence", "choice", "all"].contains(PureXML.Schema.XSDNode.localName($0) ?? "") }
}
}

63
Tests/SchemaGroupChainDepthSafetyTests.swift Normal file
View File

@ -0,0 +1,63 @@
#if !os(WASI)
import Foundation
import Testing
@testable import PureXML
/// A `<xs:group>` reference chain (`g0 = (e0, g1)`, `g1 = (e1, g2)`, ...) is FLAT in
/// the schema document: the group definitions are siblings, not nested, so the chain
/// is NOT bounded by the parser's `maxDepth`, and compiling it recurses once per
/// link. The Unique Particle Attribution determinism check summarised groups lazily
/// by recursing through each reference, so a long chain overflowed the stack while
/// compiling a legal (if hostile) schema. It now summarises every reachable group
/// once in dependency order, so the walk's group lookup is a cache hit and recursion
/// is bounded by a single group's own model nesting.
///
/// Runs on an 8 MB stack, the native main-thread budget a real caller has: swift
/// testing executes a test on a much smaller Task stack, where the separately
/// tracked particle-compiler group expansion (which still recurses on the chain)
/// would overflow first and mask the determinism fix. WASI cannot size a thread
/// stack, so the case is gated off there; its budget is a link-time setting.
@Suite("Schema group-chain depth safety")
struct SchemaGroupChainDepthSafetyTests {
@Test("a deep group-reference chain compiles without overflowing the stack")
func test_deepGroupChainCompiles() {
let depth = 1500
var groups = ""
for index in 0 ..< depth {
let next = index < depth - 1 ? "<xs:group ref=\"g\(index + 1)\"/>" : ""
groups += "<xs:group name=\"g\(index)\"><xs:sequence>"
+ "<xs:element name=\"e\(index)\" type=\"xs:string\"/>\(next)</xs:sequence></xs:group>"
}
let schema = "<xs:schema xmlns:xs=\"http://www.w3.org/2001/XMLSchema\">\(groups)"
+ "<xs:element name=\"doc\"><xs:complexType><xs:sequence>"
+ "<xs:group ref=\"g0\"/></xs:sequence></xs:complexType></xs:element></xs:schema>"
// The semaphore makes the worker's write happen-before the read, so the box's
// unchecked-Sendable mutation is safe (the closure must be Sendable for
// Foundation's Thread on Linux).
let result = ResultBox()
runOnNativeStack { result.compiled = (try? PureXML.Schema.Document(schema)) != nil }
// The chain is deterministic, so it must compile (not be rejected) and, above
// all, must not crash: before the dependency-order precompute the determinism
// check overflowed the stack on this shape well before this depth.
#expect(result.compiled)
}
private final class ResultBox: @unchecked Sendable {
var compiled = false
}
/// Runs `work` on a thread with the native main-thread stack budget and waits.
private func runOnNativeStack(_ work: @escaping @Sendable () -> Void) {
let done = DispatchSemaphore(value: 0)
let thread = Thread {
work()
done.signal()
}
thread.stackSize = 8 << 20
thread.start()
done.wait()
}
}
#endif