PureXML/Sources/Schema/ComplexValidator.swift
Mihaela Mihaljevic 5d38f0c82f
perf(schema): graph-structured-stack content matcher, killing the last 2^K content-model hang (#364)
* perf(schema): graph-structured-stack content matcher, sharing each xs:group (alongside)

The per-instance content-model build inlines every xs:group reference, so a
multiply-referenced nested group (g0=(g1,g1), g1=(g2,g2), ...) explodes to 2^K
states and reset actions: a depth-16 doubling chain already takes seconds, depth
20+ hangs. This compiles the same particle to a counted Recursive Transition
Network instead, where each definitionID group is compiled once and reached by a
call, and simulates it with a graph-structured stack.

ContentRTNBuilder mirrors ContentNFABuilder exactly except that a group with a
definitionID becomes a call into its shared, compiled-once fragment (memoized by
id; the fragment's accept is a return). The callee's counters are deliberately not
propagated to the caller: every counter is reset on its own particle's entry edge,
which is on the path from the callee's start, so a shared fragment resets itself on
every re-entry without the caller re-listing it. Re-listing was the residual
blow-up: it nested the reset-action lists to 2^K even when the states stayed O(K).

ContentRTNMatcher runs configurations (state, counters, gssNode): a call saves the
caller's counters on a GSS node interned by (returnState, savedCounters, position)
and a return restores them, so the shared callee never mixes counter state across
its callers; parents are a set and a popped node replays its pop to a parent added
later (the GLL completeness detail). The build drops to O(groups): a depth-24
doubling chain (2^24 inlined states) compiles to under 480 states in under a
millisecond.

It is gated by a differential (ContentRTNDifferentialTests) against the trusted
inlining matcher ContentNFA as the oracle: the two agree on the accept verdict and
the matched particle over the hand model cases, the named-over-wildcard tie-break,
the shared-group doubling chains that exercise call and return, and a 4000-case
fuzz of nested models with occurrences, wildcards, and shared groups, plus a bound
assertion. Built alongside the inlining matcher and not yet wired into validation,
so no behaviour change and the XSTS buckets are unchanged; the switch-over is gated
on this differential.

* perf(schema): add the streaming/located surface to the GSS matcher, lock it differentially

The content-model validator reads two things off the inlining matcher: matchedParticles
for per-child attribution, and the streaming startStates/step/admissible/isComplete for
located diagnostics (which child breaks the model, and the "expected a, b" hint). The GSS
matcher only had matchesWhole/matchedParticles, so it could not yet replace the inlining
matcher. This adds the streaming surface: admissible works because the closure already
expands call states into the callee's label states, so the follow set has the same shape
as ContentNFA's.

The differential gate now locks the whole surface, not just the accept verdict: driving
both matchers child by child, the first rejected child (step is nil) agrees, the
admissible expectation set agrees at the start and after every accepted child, and
isComplete agrees once the sequence is consumed. The fuzz now stamps group references the
way the parser does, a unique definitionID per distinct group (so a singly-referenced
named group also takes the call path, not only doubled groups), a shared id only for
genuinely shared structure, and some anonymous inline groups, over occurrences and
wildcards. Still alongside, not wired; no behaviour change, XSTS buckets unchanged.

* perf(schema): complete the GSS matcher surface with follow(after:), verified

follow(after:) is the last ContentNFA method the GSS matcher lacked: the completion
path (XSDCompletions) reads it for the acceptable-next labels and whether the content
may end. It composes the streaming surface, and the differential now locks its allowed
set and complete flag to ContentNFA too. The RTN matcher is now a complete drop-in for
the ContentNFA surface the validator and completions read, so the switch-over is a
mechanical builder swap gated on XSTS neutrality. Still alongside; no behaviour change.

* docs(design): record GSS Steps 1-3 done, the switch as a corpus-gated mechanical swap

Updates the counted-content-automaton Step 4 to the current state: the GSS matcher and
its full differential gate are built and green in-repo, and the production switch is now
a three-site builder swap blocked only on the XSTS neutrality measurement, which must run
on a machine that has the corpus. Records the exact swap sites and the gate so the corpus
machine can execute it mechanically.

* perf(schema): validate and complete on the GSS matcher, killing the last 2^K content-model hang

The three call sites that built the inlining content automaton once per validated
element, ComplexValidatorContentModel.sequenceStructureErrors (located structure
errors), ComplexValidator.validateParticle (per-child particle attribution), and
XSDCompletions (acceptable-next labels), now build ContentRTNBuilder and run
ContentRTNMatcher. A schema whose nested <xs:group> references inline to 2^K positions
(the g{i}=(g{i+1},g{i+1}) doubling shape) no longer makes each keystroke-time validation
of an element exponential in the nesting depth: the automaton is built in O(groups).

Validating a short instance against a depth-40 doubling schema (2^40 inlined positions),
the shape an editor sees mid-typing, now completes in milliseconds with a located error
instead of hanging; a new end-to-end test pins it.

Verified conformance-neutral on the full W3C XSTS corpus in release: valid-schemas-rejected
and valid-instances-rejected stay 0, invalid-schemas-accepted and invalid-instances-accepted
unchanged at 8 and 14. The located diagnostics, per-child attribution, and completions are
behaviour-identical by the surface differential (ContentRTNDifferentialTests), which locks
the verdict, matched particle, rejection index, admissible set, isComplete, and follow to
the inlining ContentNFA. ContentNFABuilder/ContentNFA are retained as that oracle.

* docs(design): mark GSS content-matcher Step 4 done, XSTS-neutral in production

Records that the three production sites are switched to the graph-structured-stack
matcher and the switch was measured conformance-neutral on the full W3C XSTS corpus in
release (all four buckets held). Notes the local corpus path and that ContentNFABuilder
is kept as the differential oracle rather than deleted.

* test(schema): compare full matched-particle attribution; bound a cyclic group id

From the adversarial review of the GSS matcher:

- The differential compared only the matched-particle kind (element/wildcard), so it
  could not observe the type/value metadata matchedParticles carries. Every fuzz and
  hand element now carries a per-name fixed value constraint and the comparison projects
  it, so the gate verifies the metadata is carried identically through call/return, not
  only that the kind agrees. A name maps to one constraint, so the comparison does not
  depend on the lowest-state-index tie-break (which only distinguishes same-name
  particles, a UPA violation the compiler rejects before instance validation); the
  matchedParticles doc now records that caveat.

- ContentRTNBuilder.groupFragment memoizes after compiling, so a (malformed) cyclic
  definitionID would recur without bound, the interactive-safety hazard this matcher
  removes. definitionID groups are acyclic by construction (the parser breaks cycles at
  element boundaries, the inlining builder relies on the same), but bound it anyway:
  a cycle degrades to a terminating empty fragment instead of recurring.

No production behaviour change; the differential, full suite, and the full XSTS corpus
in release (16/16, all four buckets held) stay green.
2026-06-28 05:07:37 +02:00

376 lines
21 KiB
Swift

/// File-scope aliases for the XSD complex-type validator, kept out of the
/// namespace to avoid nesting a type two levels deep.
typealias XSDFailure = PureXML.Validation.ValidationError
typealias XSDPath = [PureXML.Validation.PathKey]
private struct XSDChildValidationFrame {
let path: XSDPath
let namespaceBindings: [String: String]
}
private typealias XSDGroup = PureXML.Schema.Group
private typealias XSDTerm = PureXML.Schema.Term
private typealias XSDTermLabel = PureXML.Schema.TermLabel
private typealias XSDProcessContents = PureXML.Schema.ProcessContents
private typealias XSDElementType = PureXML.Schema.ElementType
private typealias XSDWildcard = PureXML.Schema.Wildcard
public extension PureXML.Schema {
/// Validates an element against a ``ComplexType``: its attribute uses, its
/// content model, and, recursively, each child element against the type
/// declared for its name. Every violation is located by the element's coding
/// path, and the results are ``PureXML/Validation/ValidationError`` values.
struct ComplexValidator {
/// The named type table that `ElementType.typeReference` resolves against.
let types: [String: ElementType]
/// Local names of elements declared `nillable="true"`.
private let nillableElements: Set<String>
/// The `default`/`fixed` value constraint declared on each element name.
private let elementConstraints: [String: ValueConstraint]
/// Complex types declared `abstract="true"`, keyed by namespaced identity
/// (`{ns}local`) so same-local-name types in different namespaces stay distinct.
let abstractTypes: Set<String>
/// Derivation methods each named type forbids through `xsi:type`, keyed by
/// namespaced identity (`{ns}local`).
let typeBlock: [String: Set<DerivationMethod>]
/// Derivation methods each element declaration forbids through `xsi:type`
/// (the `block` on the element, distinct from the block on its type), keyed by
/// namespaced identity (`{ns}local`).
let elementBlock: [String: Set<DerivationMethod>]
/// Each named complex type's base and derivation method, keyed by namespaced
/// identity (`{ns}local`).
let typeDerivation: [String: TypeDerivation]
/// Each named union type's member type keys (`{ns}local`), so the
/// `xsi:type`-`block` check can follow a derivation that reaches a declared
/// union through one of its members (cos-st-derived-OK 2.2.4).
let unionMembers: [String: [String]]
/// Global attribute declarations for strict/lax `anyAttribute` validation.
let globalAttributes: [String: AttributeUse]
/// Document-scoped xs:ID/xs:IDREF accumulator, filled during the typed walk
/// and reported by `idErrors()` once the whole tree has been seen. See the
/// ID/IDREF extension for `idErrors()` and `recordIDs(_:value:at:)`.
let idTracker = IDTracker()
public init(
types: [String: ElementType] = [:],
globalAttributes: [String: AttributeUse] = [:],
nillableElements: Set<String> = [],
elementConstraints: [String: ValueConstraint] = [:],
abstractTypes: Set<String> = [],
typeBlock: [String: Set<DerivationMethod>] = [:],
elementBlock: [String: Set<DerivationMethod>] = [:],
typeDerivation: [String: TypeDerivation] = [:],
unionMembers: [String: [String]] = [:],
) {
self.types = types
self.globalAttributes = globalAttributes
self.nillableElements = nillableElements
self.elementConstraints = elementConstraints
self.abstractTypes = abstractTypes
self.typeBlock = typeBlock
self.elementBlock = elementBlock
self.typeDerivation = typeDerivation
self.unionMembers = unionMembers
}
/// Validates `element` against `type` at `path`, one error per violation.
public func validate(
_ element: PureXML.Model.Element,
against type: ComplexType,
at path: [PureXML.Validation.PathKey] = [],
namespaceBindings inheritedBindings: [String: String] = [:],
) -> [PureXML.Validation.ValidationError] {
let namespaceBindings = Self.namespaceBindings(for: element, inherited: inheritedBindings)
var errors: [XSDFailure] = []
validateAttributes(element, type, at: path, into: &errors)
// An xsi:nil element keeps its attribute obligations but must have no
// content; its content model is not otherwise checked.
if let nilErrors = nilErrors(element, at: path) {
return errors + nilErrors
}
validateContent(element, type.content, at: path, namespaceBindings: namespaceBindings, into: &errors)
// A simpleContent fixed value is compared in its simple type's value space
// (cvc-elt.5.2.2.1), e.g. "05" equals an xs:int fixed "5".
let simpleContentType: SimpleType? = if case let .simpleContent(simple) = type.content { simple } else { nil }
errors += elementFixedErrors(element, valueType: simpleContentType, at: path)
return errors
}
/// The errors from an `xsi:nil="true"` element: rejecting it when the
/// element is not nillable, or when it carries content. Returns nil when
/// the element is not nilled.
func nilErrors(_ element: PureXML.Model.Element, fixedValue: String? = nil, at path: XSDPath) -> [XSDFailure]? {
let name = element.name.localName
// cvc-elt.3.1: a non-nillable element must carry no `xsi:nil` attribute at
// all, whatever its value (so `xsi:nil="false"` is equally forbidden, not
// only `xsi:nil="true"`).
if Self.nilAttributeValue(element) != nil, !nillableElements.contains(name) {
return [XSDFailure(reason: "element '\(name)' is not nillable but has xsi:nil", at: path)]
}
guard Self.isNil(element) else { return nil }
if Self.hasContent(element) {
return [XSDFailure(reason: "nil element '\(name)' must be empty", at: path)]
}
// cvc-elt.3.2.2: a nilled element may not have a fixed {value constraint}.
// The matched particle's constraint is authoritative; the by-local-name map
// is a fallback for paths without a particle (e.g. the resolved-type entry).
if (fixedValue ?? elementConstraints[name]?.fixedValue) != nil {
return [XSDFailure(reason: "nil element '\(name)' must not have a fixed value constraint", at: path)]
}
return []
}
/// The text an element's simple content is validated against: its character
/// content, or, when the element is empty, its `default`/`fixed` value (an
/// empty element takes that value, and the value, not the empty string, must
/// satisfy the type). The matched particle's constraint takes precedence over
/// the by-local-name `elementConstraints` map, exactly as the tree path's
/// `validateChild` resolves it.
func effectiveSimpleText(for element: PureXML.Model.Element, particleConstraint: ValueConstraint? = nil) -> String {
let text = Self.rawTextContent(element)
let constraint = particleConstraint ?? elementConstraints[element.name.localName]
return text.isEmpty ? (constraint?.value ?? text) : text
}
/// The error from a `fixed` element value constraint: the element's text
/// must equal the fixed value in the element type's value space when known.
func elementFixedErrors(
_ element: PureXML.Model.Element,
valueType: SimpleType? = nil,
particleFixed: String? = nil,
at path: XSDPath,
) -> [XSDFailure] {
guard let fixed = particleFixed ?? elementConstraints[element.name.localName]?.fixedValue else { return [] }
// A fixed element's content is its fixed character value: it may not carry
// element children, even under a mixed content type (cvc-elt.5.2.2.2.1).
if !element.children.compactMap(\.element).isEmpty {
return [XSDFailure(reason: "fixed element '\(element.name.localName)' must not have element children", at: path)]
}
let text = Self.rawTextContent(element)
// An empty element takes the fixed value; only present content must equal it.
if text.isEmpty { return [] }
if let valueType, valueType.valueMatches(text, literal: fixed) { return [] }
return text == fixed ? [] : [XSDFailure(reason: "element '\(element.name.localName)' is fixed and must be '\(fixed)'", at: path)]
}
/// Validates `element` against any element type at `path`, resolving a
/// `typeReference` through the type table.
public func validate(
_ element: PureXML.Model.Element,
as type: ElementType,
at path: [PureXML.Validation.PathKey] = [],
) -> [PureXML.Validation.ValidationError] {
var errors: [XSDFailure] = []
let namespaceBindings = Self.namespaceBindings(for: element)
validateChild(element, against: type, at: path, namespaceBindings: namespaceBindings, into: &errors)
return errors
}
// MARK: Content
private func validateContent(
_ element: PureXML.Model.Element,
_ content: ContentType,
at path: XSDPath,
namespaceBindings: [String: String],
into errors: inout [XSDFailure],
) {
let children = element.children.compactMap(\.element)
switch content {
case .empty:
if !children.isEmpty { errors.append(XSDFailure(reason: "element must be empty", at: path)) }
rejectText(element, at: path, into: &errors)
case let .simpleContent(type):
if !children.isEmpty { errors.append(XSDFailure(reason: "element must not have children", at: path)) }
if let error = type.validate(Self.rawTextContent(element)) {
errors.append(XSDFailure(reason: "content: \(error)", at: path))
}
case let .elementOnly(particle):
rejectText(element, at: path, into: &errors)
validateParticle(particle, children: children, at: path, namespaceBindings: namespaceBindings, into: &errors)
case let .mixed(particle):
validateParticle(particle, children: children, at: path, namespaceBindings: namespaceBindings, into: &errors)
}
}
func rejectText(_ element: PureXML.Model.Element, at path: XSDPath, into errors: inout [XSDFailure]) {
guard !Self.textContent(element).isEmpty else { return }
errors.append(XSDFailure(reason: "element must not contain text", at: path))
}
// MARK: Particles
private func validateParticle(
_ particle: Particle,
children: [PureXML.Model.Element],
at path: XSDPath,
namespaceBindings: [String: String],
into errors: inout [XSDFailure],
) {
// Locate each content-model violation at the offending child (or the
// missing one), with a recovery hint, rather than one opaque failure,
// then assess every well-placed child against the exact particle the
// content model matched it to (#180), not a by-name lookup that loses
// which of two same-named particles, or a wildcard versus a named
// particle, actually matched.
let matched: [PureXML.Schema.MatchedParticle?]
if case let .group(group) = particle.term, group.compositor == .all {
allStructureErrors(group, children: children, groupOptional: particle.occurrenceRange.minimum.isZero, at: path, into: &errors)
matched = allMatchedParticles(group, children: children)
} else {
sequenceStructureErrors(particle, children: children, at: path, into: &errors)
matched = PureXML.Schema.ContentRTNMatcher(PureXML.Schema.ContentRTNBuilder.build(particle)).matchedParticles(children.map(\.name))
}
validateChildren(
children,
matched: matched,
frame: XSDChildValidationFrame(path: path, namespaceBindings: namespaceBindings),
into: &errors,
)
}
private func validateChildren(
_ children: [PureXML.Model.Element],
matched: [PureXML.Schema.MatchedParticle?],
frame: XSDChildValidationFrame,
into errors: inout [XSDFailure],
) {
let steps = Self.childSteps(children)
for (index, child) in children.enumerated() {
let namespaceBindings = Self.namespaceBindings(for: child, inherited: frame.namespaceBindings)
let path = frame.path + [steps[index]]
switch matched[index] {
case let .element(type, valueConstraint):
// An element particle without an inlined type (a bare ref)
// resolves through its global declaration, as before.
guard let declared = type ?? globalElementDeclaration(for: child.name) else { continue }
validateChild(child, against: declared, at: path, namespaceBindings: namespaceBindings, particleConstraint: valueConstraint, into: &errors)
case let .wildcard(wildcard):
validateWildcardChild(child, processContents: wildcard.processContents, at: path, namespaceBindings: namespaceBindings, into: &errors)
case .none:
// The content model rejected this child (a structure error,
// already located above); the matched particle is undefined.
continue
}
}
}
/// Validates a wildcard-matched child by its `processContents`: skip does
/// nothing, lax validates against a global declaration when one exists, and
/// strict requires one.
private func globalElementDeclaration(for name: PureXML.Model.QualifiedName) -> PureXML.Schema.ElementType? {
// A global element declaration is always in the schema's target
// namespace, so an instance element matches one only by full qualified
// name. The namespaced key already covers a no-namespace global (its key
// is `element:{}local`); matching an unqualified instance against a
// namespaced global by bare local name would conflate namespaces, e.g.
// accepting unqualified text where a namespaced `xs:short` is declared.
types[PureXML.Schema.XSDParser.elementDeclarationKey(name)]
}
private func validateWildcardChild(
_ child: PureXML.Model.Element,
processContents: ProcessContents,
at path: XSDPath,
namespaceBindings: [String: String],
into errors: inout [XSDFailure],
) {
switch processContents {
case .skip:
return
case .lax:
if let declaration = globalElementDeclaration(for: child.name) {
validateChild(child, against: declaration, at: path, namespaceBindings: namespaceBindings, into: &errors)
}
case .strict:
if let declaration = globalElementDeclaration(for: child.name) {
validateChild(child, against: declaration, at: path, namespaceBindings: namespaceBindings, into: &errors)
} else if let type = xsiDeclaredType(for: child, namespaceBindings: namespaceBindings) {
validateChild(child, against: type, at: path, namespaceBindings: namespaceBindings, into: &errors)
} else {
errors.append(XSDFailure(reason: "no declaration for wildcard-matched element '\(child.name.localName)'", at: path))
}
}
}
/// The type named by a wildcard-matched element's `xsi:type`, when present.
private func xsiDeclaredType(for child: PureXML.Model.Element, namespaceBindings: [String: String]) -> ElementType? {
guard Self.xsiTypeAttributeValue(child) != nil else { return nil }
guard let reference = Self.xsiTypeReference(child, namespaceBindings: namespaceBindings),
let type = Self.resolveNamedType(reference, in: types)
else { return nil }
return type
}
private func validateChild(
_ child: PureXML.Model.Element,
against declared: ElementType,
at path: XSDPath,
namespaceBindings: [String: String],
particleConstraint: ValueConstraint? = nil,
into errors: inout [XSDFailure],
) {
// An instance `xsi:type` overrides the declared type, provided the named
// type exists in the schema. Resolve to the type itself rather than a
// reference so re-entry does not re-read the same xsi:type. A `block` on
// the declared type can forbid the substitution; an abstract declared
// type requires one.
if let xsiError = xsiTypeOverrideError(declared: declared, child: child, at: path, namespaceBindings: namespaceBindings) {
errors.append(xsiError)
return
}
guard let declared = overriddenType(declared, for: child, namespaceBindings: namespaceBindings, at: path, into: &errors) else { return }
switch declared {
case let .simple(simple):
// A simple type declares no attribute uses, so a simple-typed
// element (including one whose simple type comes from an xsi:type
// override, and including a nilled one) may carry only the
// schema-instance and namespace attributes; any other is a
// violation. Checked before the nil short-circuit, since an
// xsi:nil element is equally bound by it.
for attribute in child.attributes where !Self.isNamespaceDeclaration(attribute) && !Self.isSchemaInstanceAttribute(attribute) {
errors.append(XSDFailure(reason: "'\(child.name.localName)' has a simple type and must not carry attribute '\(attribute.name.localName)'", at: path))
}
if let nilErrors = nilErrors(child, fixedValue: particleConstraint?.fixedValue, at: path) {
errors += nilErrors
return
}
if !child.children.compactMap(\.element).isEmpty {
errors.append(XSDFailure(reason: "'\(child.name.localName)' must not have children", at: path))
}
// An empty element takes its `default`/`fixed` value; that value
// (not the empty string) is what must be valid against the type,
// including any xsi:type override already resolved into `simple`.
let text = Self.rawTextContent(child)
if let error = simple.validate(effectiveSimpleText(for: child, particleConstraint: particleConstraint)) {
errors.append(XSDFailure(reason: "'\(child.name.localName)': \(error)", at: path))
}
recordIDs(simple, value: text, at: path)
errors += elementFixedErrors(child, valueType: simple, particleFixed: particleConstraint?.fixedValue, at: path)
case let .complex(complex):
errors.append(contentsOf: validate(child, against: complex, at: path, namespaceBindings: namespaceBindings))
case .typeReference:
validateResolvedReference(declared, child, at: path, namespaceBindings: namespaceBindings, into: &errors)
}
}
/// Validates a child against a `typeReference` through the shared resolver,
/// reporting an unknown name or a circular chain as a located error.
private func validateResolvedReference(
_ declared: ElementType,
_ child: PureXML.Model.Element,
at path: XSDPath,
namespaceBindings: [String: String],
into errors: inout [XSDFailure],
) {
switch resolveReference(declared) {
case let .unknown(name):
errors.append(XSDFailure(reason: "unknown type '\(name)'", at: path))
case let .circular(name):
errors.append(XSDFailure(reason: "circular type reference '\(name)'", at: path))
case let .resolved(resolved):
validateChild(child, against: resolved, at: path, namespaceBindings: namespaceBindings, into: &errors)
}
}
}
}