Schema documents that form an import chain are now compiled as a single
union (Document.init(composing:)) rather than as separately compiled
documents merged after the fact. Merging froze each document's content
models with only the substitution-group members visible in its own import
closure, so a substitution member declared in an importing document never
reached the head's content model in the imported document.
This clears XSTS ctZ007, a cross-document substitution group, taking
valid-instances-rejected (#146) from 4 to 3 with no other bucket moving
(now 1/262/3/147). The union pools each document's container closure into
one context, deduplicating a document supplied directly and also reached
through another's import by target namespace and component-name set.
The XSTS runner uses the union only for import chains, detected by matching
import/include/redefine schemaLocation basenames against the listed
documents; independent roots stay on the lenient per-document merge, which a
union would wrongly trip with cross-document checks.
The remaining three valid-instance rejections are the Ethiopic-digit \d
cases (reS17, reS38, reZ004v), which contradict \d = \p{Nd}.
Consolidates the in-progress #146 work; valid-instances-rejected reaches 4
on the 2006-11-06 XSTS archive (buckets now 1/262/4/147).
Regex: match Unicode scalars rather than grapheme clusters; the empty
pattern matches only the empty string; pin the category repertoire to
Unicode 3.2, the version the corpus is consistent at, so code points added
in 3.2 (Tagalog, Cyrillic Komi) are word characters while later additions
(the Tamil digit zero) stay unassigned and match \D. New XSDCategory and a
3.2 assignment table derived from DerivedAge, replacing the old Unicode31
files.
Content model: assess each child against the particle the content automaton
matched, not a by-name lookup (#180). Two same-named particles keep their
own fixed values, and an element matching a skip wildcard is no longer
validated against its global declaration. Selection is made reproducible
when an ambiguous schema slips past the UPA check.
XSTS runner: compile and merge every schemaDocument in a schemaTest, not
just the first; this also surfaced one previously-masked invalid-instance
acceptance (recorded in the baselines).
Also folds in attribute-group anyAttribute inheritance, extension wildcard
union, namespace-qualified type tables for multi-document
xsi:schemaLocation, per-container elementFormDefault, strict and lax
attribute wildcard validation, built-in type resolution, and xs:redefine
simpleType overrides.
Schema compile findings now carry ValidationError coding paths that resolve to source spans on a ranged tree, enabling IDE underlines for id, structure, include, and facet violations.
Ship the OpenAPIKit-style validation layer (builtin rules, warnings channel,
Validator tiers, coverage gates) and wire XSD compile checks through it. Advance
#161 with chameleon include validation, multi-container reference resolution,
and include targetNamespace composition rules so merged schema sets compile
correctly when a schemaLoader is provided.
An element declaring substitutionGroup="head" whose own type is unrelated to
the head's type was accepted; XSD 1.0 e-props-correct.4 requires the member's
type be validly derived from the head's. The check runs after named types
resolve and reuses the restriction/extension + built-in-lattice derivation
oracle (an ur-type head admits any member).
It is conservative so it never rejects a valid member: it runs only for a
self-contained schema (no import/include/redefine, where the derivation table
is complete and single-namespace names are unambiguous), reads the member's
type from its own type attribute and the head's from the top-level element
declarations only (so a local element of the same name supplies neither), and
stands down when either type is a list or union (cos-st-derived-ok clauses
2.3/2.4 union-member derivation is not modelled).
The post-namedTypes consistency calls move into a postNamedTypeErrors
aggregator alongside consistencyErrors. XSTS: invalid schemas accepted
480 to 479, no other bucket raised.
An attribute or element whose type is or derives from xs:ID may not carry a
default or fixed value (a-props-correct.3 / e-props-correct.5). Such schemas
were accepted.
The check runs after named simple types are resolved, so a named user type
derived from xs:ID is recognized (not just the built-in and inline forms). The
type reference's namespace is resolved from its prefix: only an ID in the XSD
namespace is the built-in, and a user type is ID only when it resolves to one
whose base is xs:ID. An adversarial critic caught a namespace-blind over-
rejection (a user type named ID, or an imported foo:ID, mistaken for the
built-in); the namespace resolution fixes it, with a regression test.
XSTS: invalid schemas accepted 489 to 481 (attZ013a/b, elemZ032a/b,
valueconstraint01001m2/m3/m5/m6 now rejected), no other bucket raised.
A complex type's or attribute group's attribute-use set (flattened through
attributeGroup references) may not contain two distinct declarations with the
same name and namespace (ct-props-correct.4 / ag-props-correct.2), nor more than
one attribute whose type is or derives from xs:ID (ct-props-correct.5).
Declarations are deduplicated by the identity of their declaring node, so the
same declaration reached by two attribute-group paths (a diamond) is one use,
not a clash. Names carry their resolved namespace (form / attributeFormDefault
for locals, the prefix binding for refs), so a local attribute and a reference
to a same-local-name attribute in an imported namespace do not collide. The
check runs after named simple types are resolved, so a named user type derived
from xs:ID is recognized, not just the built-in and inline forms.
XSTS: invalid schemas accepted 505 to 495 (attQ008/009/012/013, attgD006/007/
009/017, ctM003/004 now rejected), no other bucket raised. An adversarial critic
caught a false positive on same-local-name-different-namespace (fixed by
resolving the ref namespace) and a missed named-user-ID type (fixed by ordering
the check after type resolution). Disclosed under-rejections: the set inherited
through extension is not composed in; a duplicate needing an unprefixed
reference with no default namespace is not detected.
A complex or simple type may not derive, transitively through its base, from
itself. Such schemas were accepted. The check walks the base chain built from
each type's complexContent/simpleContent extension/restriction base (and a
simpleType's restriction base), following an edge only when the base resolves to
the schema's own target namespace.
Three exceptions keep it from over-rejecting: element content that recurses (an
element of the type's own type) is not a derivation and is not followed; a type
redefined in xs:redefine legitimately names itself as its base, so redefined
types end the chain; and the ur-types (anyType/anySimpleType/anyAtomicType,
which the schema-for-schemas bootstraps with self-referential definitions) end
the chain. The target-namespace gate prevents a local type whose base is a
same-local-name type in another namespace from being mistaken for a self-cycle
(found by an adversarial critic; the engine resolves by local name).
XSTS: invalid schemas accepted 519 to 516 (addB101, elemM003, xsd007.e now
rejected), no other bucket raised. Union/list and genuine cross-namespace
derivation cycles are a disclosed under-rejection.
An `<xs:element ref="p:foo">` had its prefix stripped and the element assumed to
be in the schema's target namespace. That mis-qualified every reference to an
imported element: a restriction of an `##other` wildcard by such a reference was
rejected (the element looked like the target namespace, which `##other` forbids),
and the same mis-qualification rejected valid instances whose elements were
correctly in the imported namespace.
The reference's QName is now resolved by standard XML namespace rules against the
schema root's xmlns bindings: a bound prefix gives its namespace (an imported
element lands in the imported namespace), an unprefixed name the default
namespace, or no namespace when none is declared. Substitution-group members
stay in the target namespace (this schema's own globals).
The content model is shared between schema-restriction checks and instance
validation, so both improve. XSTS: valid schemas rejected 32 to 4, invalid
schemas accepted 526 to 520, valid instances rejected 214 to 180, invalid
instances accepted held at 160.
Disclosed limitations: a prefix re-bound on an inner element is not honored,
cross-namespace substitution-group members are not modeled, and a dangling
reference is not yet flagged.
Complete the element-vs-element type clause of Particle Valid (Restriction):
the restricting element's type must be, or derive by restriction from, the base
element's type. Previously only names were compared, so an unrelated, wider, or
ur-type element type was accepted under restriction.
Enrich the content-model particle: Term.element carries the element type's
derivation identity (the resolved type name, anyType for an absent type, nil for
an inline anonymous type). Record named simpleType restriction derivations in the
derivation table alongside complex ones. The element-vs-element check walks the
user restriction chain and then the built-in lattice to decide derivation by
name; an ur-type base, a union base (membership is not a restriction chain), and
an inline anonymous type are permitted, so no valid restriction is rejected.
XSTS: invalid schemas accepted 616 to 594, invalid instances accepted 165 to 160
(more invalid schemas rejected up front), valid schemas rejected held at 72,
valid instances rejected held at 214.
Reject ambiguous content models at compile time via a Glushkov position
automaton over a literal, QName-resolved view of the raw schema tree, built
just for this rule and independent of the instance-validation content model.
The literal view resolves a ref's QName through the schema's prefix bindings
(not the blanket target namespace) and does not expand substitution groups,
removing the two false-overlap sources a check over the compiled model
produced. Determinism is per particle: a particle competing with a repetition
of itself is never a violation, so a fixed count {n,n} unrolls into copies in
sequence while a true range or unbounded repeat uses a self-loop, and only two
distinct source particles with overlapping labels conflict. The overlap test
is QName-only, an under-rejection that can never over-reject.
A position cap bounds pathological nesting (skipping the check, never
over-rejecting). The element-declarations-consistent test loses its
incidentally-ambiguous (a?, a) case in favor of a deterministic (a, b, a).
XSTS: invalid schemas accepted 884 to 864, valid schemas rejected held at 72,
instance buckets unchanged.
Within a schema, global type names (simpleType and complexType share one symbol
space), global element names, global attribute names, named model-group names,
and named attribute-group names must each be unique, and identity-constraint
names must be unique across the document. A duplicate was accepted, with the
later definition silently overwriting the earlier one.
Only the document's own globals are examined (xs:redefine children and unloaded
included documents are not direct globals), so a redefinition is not mistaken
for a clash; the symbol spaces are kept separate, so a type and an element may
share a name.
XSTS: invalid schemas accepted 909 to 891 (-18); valid schemas rejected held at
72; instance buckets unchanged.
Every QName a schema names must resolve to a declared component or a built-in:
type/base/itemType/memberTypes to a type, element/attribute/group/attributeGroup
ref to the matching global component, and element substitutionGroup to a global
element. An undeclared reference was previously accepted at compile time and only
surfaced lazily during instance validation, if at all.
References are matched by local name, with the value trimmed (a QName attribute
is whiteSpace=collapse). The whole check is skipped when the document declares an
import/include/redefine, since the default compile does not load externals and
the pools would be incomplete; that guard is gated on the XSD namespace so a
foreign element named "import" does not trip it.
XSTS: invalid schemas accepted 1092 to 1051 (-41); valid schemas rejected held
at 72 (an interim version regressed one valid schema on a whitespace-padded
base=" xs:string ", fixed by trimming); instance buckets unchanged.
The general mechanism for the structural tail of the schema-validity gap, in
place of hand-coding a rule per component. A data-driven table encodes the XSD
1.0 schema-for-schemas child content model; at compile time it rejects any
component child the model does not admit, more than one annotation (or an
annotation that is not first, except on schema/redefine, which allow several),
and an identity constraint missing its selector or field.
The walk visits only XSD-namespace elements and skips appinfo/documentation
foreign content, so foreign children are never flagged; restriction/extension
use the permissive union of their simple/complex/simpleContent contexts, so a
child valid in some context is never rejected. One table covers every component.
XSTS: invalid schemas accepted 1688 to 1313 (-375); valid schemas rejected held
at 72 (no over-rejection); instance buckets unchanged.
The id attribute on any XSD component is of type xs:ID, so each value must be a
valid NCName and unique within the schema document. The values were never
checked, so a malformed (id="", id="123") or duplicated id left the schema
wrongly accepted across many categories (idA-idK, attribute groups, complex
types).
The schema-document tree is walked once, collecting id attributes. Two guards
keep valid schemas from being rejected: appinfo/documentation subtrees are
skipped (their foreign content carries its own non-xs:ID id attributes), and
only the unprefixed id is treated as the component identifier (a foreign xml:id
or pre:id is a different attribute). Findings join the shared parse-time
diagnostics.
XSTS: invalid schemas accepted 1789 to 1688; no other bucket raised.
First cut at the schema-validity gap: the validator ran instance-validity rules
but not schema-validity rules, so it silently accepted invalid schemas. Per XSD
Part 2 Datatypes 4.3, a length-family facet value (length, minLength, maxLength,
totalDigits, fractionDigits) must be a valid nonNegativeInteger (totalDigits a
positiveInteger), length may not co-occur with minLength/maxLength, and the
integer ranges must be ordered. Malformed values (length="", "-1", "a", "1e2")
were silently dropped by numeric facet parsing and the schema accepted.
The definition is now read raw from the restriction and validated at compile
time. Well-formedness is checked independently of machine-integer range (via a
canonical digit string), so a huge but valid value is not misreported as
malformed, and ordering compares by length then lexically. Findings travel
through a shared reference-type diagnostics collector on the value-type parsing
context (surviving its copies) and join the model-level consistency findings
thrown from Schema.Document.init.
Limited to facets declared on the local restriction, so an inherited facet is
never wrongly flagged. The min/maxInclusive value-space rules, facet
applicability per base, and inherited co-occurrence remain for later iterations
(see docs/schema-validity-burndown.md).
XSTS: invalid schemas accepted 2461 to 2082, no change to any other bucket.
A complexType with mixed="true" and no sequence/choice/all (only attributes)
was compiled to the empty content type, which forbids text, so a mixed element
carrying only character data was rejected. It is now mixed content over an empty
particle: text is permitted, child elements are still disallowed. Non-mixed
empty types are unchanged.
XSTS: valid instances rejected 311 to 308. Baseline updated; regression tests
cover text accepted, an empty element, a child element rejected, and the
non-mixed control still rejecting text.
Refs #147, #129.
An <xs:attribute ref="..."> reference has no name attribute, so attributeUse
returned nil and the referenced attribute was dropped, making it look undeclared
on the instance (nested attributeGroup refs were already flattened, but the
per-attribute reference was not). References now resolve through a new global
attribute index to the declaration's type, with use/default/fixed taken from the
reference site and the name qualified to the target namespace.
XSTS: valid instances rejected 338 to 311. Baseline updated; regression tests
cover a referenced attribute admitted, its type enforced, and an attribute
outside the group still rejected. The attribute-parsing functions move to
XSDAttributeParser.swift to keep XSDParser within the type-body length limit.
Refs #147, #129.
The block on an element declaration (distinct from block on its type) was
parsed into the derivation tables but never threaded to the instance validator,
so an xsi:type that substitutes a type derived by a blocked method was accepted.
elementBlock now flows through XSDCompiled, SchemaDocument, the validation
context, and both ComplexValidator construction sites to blockedSubstitutionError,
which checks the element's own block alongside the type's. It also resolves the
element-ref type indirection (a ref'd child carries an `element:foo` key that
resolves through `types` to the concrete declared type) so the derivation chain
and block checks fire for referenced children, not only the document root.
XSTS: invalid instances accepted 261 to 240. Baseline updated; regression tests
cover element block rejecting a restriction xsi:type, the no-block control, and
an unrelated block not over-rejecting.
Refs #147, #129.
The derivation final check and Particle Valid (Restriction) move from
fail-fast throws at schema compile into composable Validation rules
(finalRespected, restrictionsAreSubsets) over per-type SchemaTypeFact
subjects, so a schema with several problems reports every finding at
once. SchemaError gains .inconsistent([String]) carrying all findings
(texts unchanged) and drops .finalViolation/.invalidRestriction.
checkRedefine, checkAllGroups, and notASchema stay throws: they examine
the raw schema source, the compilation analog of well-formedness.
Make XSD validation namespace-aware and enforce the form defaults and
xs:all structural constraints that were previously ignored:
- targetNamespace is tracked: a schema's global elements are in its
target namespace, and an instance's root element must be in it.
- elementFormDefault / attributeFormDefault (and per-declaration form)
qualify local element and attribute declarations. The content-model
matcher already compares namespace URIs, so qualified declarations
now match only same-namespace instance nodes; attribute matching
became namespace-aware too.
- xs:all groups are checked at compile time: every member must be an
element occurring at most once, the group may occur at most once, and
nested model groups inside an all are rejected (a SchemaError).
When a schema has no target namespace the stamping is a no-op, so
unqualified schemas validate exactly as before.
Cross-namespace type references through xs:import resolve by local name
only; full multi-namespace type tables are not modeled.
Closes#78
Read and enforce the XSD 1.0 derivation controls that were previously
ignored:
- abstract elements may not appear in an instance directly, only through
a substitution member; an element of an abstract complex type must
supply an xsi:type naming a concrete derived type.
- block on a type forbids the disallowed xsi:type substitutions; block on
an element drops substitution-group members that derive by a blocked
method.
- final on a type rejects, at compile time, any type that derives from it
by a final method.
- xs:redefine requires the redefined type to derive from itself.
Track each named complex type's base and derivation method as the
backbone these checks walk. Also compose complexContent extension and
restriction content models, which were previously dropped: an extension
now inherits its base's attributes and content (base first, then the
extension, in sequence).
Complex-type restriction is not verified to be a structurally valid
subset of its base (Particle Valid (Restriction)); that limit is
documented at its sites.
Closes#76
Add xs:any and xs:anyAttribute wildcards with namespace and
processContents constraints (skip/lax/strict), and resolve the
list built-in datatypes (IDREFS, ENTITIES, NMTOKENS) to their
item types so each list item validates.
WildcardNamespace models ##any/##other/enumerated URI sets;
Wildcard.admits gates undeclared attributes and wildcard-matched
children. ContentNFA.follow exposes the follow set for completions.
Closes#77
Closes#75, on the validation framework. XSD instance-level features:
- nillable elements: an xsi:nil="true" element is accepted (its content model is
not checked) only when the element is declared nillable and carries no content;
xsi:nil on a non-nillable element, or a nilled element with content, is a
located error.
- fixed value constraints: a present fixed attribute must equal its fixed value,
and a fixed element's text must match.
- default/fixed are parsed onto AttributeUse (per use) and collected per element
name (nillable set + value constraints), threaded through XSDContext to the
ComplexValidator.
All findings are located ValidationErrors. Collection and helper methods move to
same-file extensions to stay within type-body limits.
Validate XSD identity constraints over the instance document in a pass that
walks the tree alongside type validation:
- xs:unique requires the selected nodes' field tuples to be distinct, excluding
any node that is missing a field.
- xs:key additionally requires every field to be present.
- xs:keyref requires each tuple to equal a tuple of the named key or unique
visible in an enclosing scope, resolved through a stack of scope frames so a
keyref and the key it refers to may sit on the same element.
The selector and field XPaths run on the existing XPath engine, relative to the
declaring element and each selected node. Constraints are gathered for every
element declaration at any depth, keyed by element name.
Closes#67
Compose schemas across documents and resolve types at the instance:
- xs:include, xs:import, and xs:redefine pull a transitive closure of schema
documents through an injected loader (resolved by schemaLocation), base
schemas first so the including schema wins; a redefine's nested definitions
override the schema it includes.
- substitution groups: an xs:element ref to a head also admits every element
that substitutes for it, expanded at parse time into a choice over the head
and its transitive members.
- xsi:type overrides an element's declared type with the named type at the
instance; XML Schema instance attributes are exempt from attribute checks.
The schema parser splits across XSDContext (shared tree helpers, document
collection, parsing context), XSDSimpleParser (simple types and facets), and
XSDParser (elements, complex types, model groups) to keep each within bounds.
Closes#66
Extend the schema parser toward full structural parity:
- list simple types validate each whitespace-separated item against the item
type and count items for the length facets; union simple types admit a value
valid for any member type. A restriction over a named list or union inherits
its variety.
- Named attribute groups are expanded (transitively) into every complex type
that references them.
- Named model groups are expanded at each xs:group ref, with a cycle guard;
element refs resolve to the referenced global element's declared type,
through a reserved type-table key so forward and recursive refs settle at
validation time.
Named simple types now resolve over repeated passes so they may restrict or
list one another regardless of document order. Simple-type and facet parsing
moves into a sibling XSDSimpleParser to keep each parser within bounds.
Closes#65
Add Schema.Document compiling an xs:schema into global element declarations
and a named-type table, and validate(_:) checking instance documents end to
end. The parser supports global and local element declarations, named and
inline simple and complex types, sequence/choice/all groups with occurrence,
attribute uses, simple content (restriction/extension), the full facet set,
and recursive types via named references resolved during validation. Extends
ComplexType with a typeReference case and ComplexValidator with a type table
so recursive schemas are representable.
Closes#10
