Consolidates the in-progress #146 work; valid-instances-rejected reaches 4
on the 2006-11-06 XSTS archive (buckets now 1/262/4/147).
Regex: match Unicode scalars rather than grapheme clusters; the empty
pattern matches only the empty string; pin the category repertoire to
Unicode 3.2, the version the corpus is consistent at, so code points added
in 3.2 (Tagalog, Cyrillic Komi) are word characters while later additions
(the Tamil digit zero) stay unassigned and match \D. New XSDCategory and a
3.2 assignment table derived from DerivedAge, replacing the old Unicode31
files.
Content model: assess each child against the particle the content automaton
matched, not a by-name lookup (#180). Two same-named particles keep their
own fixed values, and an element matching a skip wildcard is no longer
validated against its global declaration. Selection is made reproducible
when an ambiguous schema slips past the UPA check.
XSTS runner: compile and merge every schemaDocument in a schemaTest, not
just the first; this also surfaced one previously-masked invalid-instance
acceptance (recorded in the baselines).
Also folds in attribute-group anyAttribute inheritance, extension wildcard
union, namespace-qualified type tables for multi-document
xsi:schemaLocation, per-container elementFormDefault, strict and lax
attribute wildcard validation, built-in type resolution, and xs:redefine
simpleType overrides.
Ship the OpenAPIKit-style validation layer (builtin rules, warnings channel,
Validator tiers, coverage gates) and wire XSD compile checks through it. Advance
#161 with chameleon include validation, multi-container reference resolution,
and include targetNamespace composition rules so merged schema sets compile
correctly when a schemaLoader is provided.
Document the restructured GitHub issue hierarchy (epic #167, gates,
milestones M1-M5, task labels) and point README Status at the current
production-readiness focus and v0.2.0 XSTS baselines.
Cut the second pre-release: schema-validity burn-down to XSTS baselines
1/266/171/155, component child-content shape checks, and synced conformance
docs. CHANGELOG updated for v0.2.0.
Require that any restricted attribute has a type validly derived by restriction from its base counterpart. Validates atomic built-in derivation, list item type restrictions, and union member/subset relationships recursively. Updates baseline to 362 accepted invalid schemas.
Reject top level declarations that omit a name attribute. Disallow derivation by restriction of xs:anySimpleType inside simpleType definitions. Require restriction facets length, minLength, maxLength, totalDigits, and fractionDigits to restrict or equal the base type facets. Updates the conformance baseline to 383 accepted invalid schemas.
In a complexContent restriction, an attribute the restriction redeclares that the
base type marks use="required" may not be made optional or prohibited. Only
attributes explicitly redeclared that also exist in the base are checked; an
omitted attribute is inherited unchanged. The base reference must resolve to the
schema's own target namespace, and the check stands down for a schema that
composes other documents.
The matching fixed-value clause is deliberately not enforced: it requires
comparing the two fixed values in the attribute type's value space, not lexically
(a list-typed fixed value differing only in whitespace is the same value), so a
string comparison wrongly rejects valid schemas. That clause is left for a later
change (a disclosed under-rejection).
XSTS: invalid schemas accepted 394 to 390, no other bucket raised.
A simpleType nested inside a restriction/list/union/element/attribute is the
anonymous localSimpleType (no name), and a nested attributeGroup is always a
reference; named simpleType and attributeGroup definitions are top-level only (a
direct child of schema or redefine). A nested definition declaring a name was
accepted; it is now rejected. A nested anonymous simpleType, an attributeGroup
reference, and a top-level named definition are unaffected.
XSTS: invalid schemas accepted 403 to 394, no other bucket raised.
An xs:all group in a complexContent extension of a base that already has element
content was accepted. The effective content model of such an extension is
sequence(base content, extension content), which nests the all and violates the
rule that an all group must be the whole content model. The base type is read
from the compiled model: rejected only when the base resolves, in the schema's
own target namespace, to a complex type with element-only or mixed content. An
explicitly-empty base (an empty model group with mixed=false) has effectively
empty content and may still be extended by an all (the all becomes the whole
content); a simple-content, empty, foreign, or unresolved base is left alone.
Checked for a self-contained schema.
XSTS: invalid schemas accepted 411 to 403, no other bucket raised.
A simple type whose final contains list (or #all) may not be a list itemType,
and one whose final contains union (or #all) may not be a union memberType.
finalRespected already covered restriction and extension; the list and union
directions were unchecked because the derivation-method set does not model
final="list"/"union", so the final attribute is read directly. The itemType /
memberTypes reference is matched only when it resolves to the schema's own target
namespace, so a list or union of a built-in (whose final is empty) is not
confused with a user type of the same local name. Checked for a self-contained
schema.
XSTS: invalid schemas accepted 413 to 411, no other bucket raised.
A whiteSpace facet that relaxed its base type's effective whiteSpace was
accepted; XSD Part 2 4.3.6 permits only strengthening, ordered
preserve < replace < collapse. A base fixed to collapse admits only collapse,
and a replace base admits replace or collapse; relaxing it (e.g. replace over a
collapse base, or restricting xs:token to replace) is now rejected. The base's
effective whiteSpace is its own facet falling back to its built-in intrinsic;
the check is atomic-only and, when a base's own whiteSpace has not yet resolved,
can only under-detect (an unresolved base defaults to preserve, the weakest), so
it never rejects a valid restriction.
XSTS: invalid schemas accepted 422 to 413, no other bucket raised.
A top-level xs:element declaration (a direct child of schema, or of redefine) was
accepted with ref, form, minOccurs, or maxOccurs. A global element is a
declaration, never a particle or a reference, so those local-use attributes are
prohibited by the schema-for-schemas topLevelElement. The top-level-attribute
applicability scan is generalized into topLevelDeclarationErrors, driven by a
per-kind forbidden-attribute map, covering both attribute and element. A local
element particle inside a content model is unaffected.
XSTS: invalid schemas accepted 424 to 422, no other bucket raised.
An element ref carrying nillable/default/fixed/form/block, or an element or
attribute ref carrying an inline type definition, was accepted. A ref is a use
of a global declaration, not a declaration, so beyond ref it admits only
minOccurs, maxOccurs, and id (src-element.2.2). The applicability check already
excluded name and type from a ref; this extends the exclusion to the remaining
declaration-only properties and to an inline simpleType/complexType child.
minOccurs/maxOccurs, a group/attributeGroup ref, and an attribute ref's
use/default/fixed (which override on the use) are unaffected.
XSTS: invalid schemas accepted 439 to 424, no other bucket raised. Disclosed
under-detection: a key/unique/keyref child under a ref is not yet flagged.
A top-level xs:attribute declaration (a direct child of schema, or of redefine)
was accepted with use, form, or ref. Those belong to a local attribute use, not
a global declaration: the schema-for-schemas topLevelAttribute marks ref, form,
and use as prohibited, allowing only default, fixed, id, name, type and an inline
simpleType. The per-component applicability table is the permissive global/local
union, so it could not draw the distinction; a focused scan of the direct
attribute children of schema and redefine supplies the global-only constraint. A
local attribute use inside a complex type or attribute group is unaffected.
The existing test_validAttributeValues asserted a top-level attribute with use
compiles, which is itself invalid; corrected to a local attribute use.
XSTS: invalid schemas accepted 454 to 439, no other bucket raised.
A default/fixed value on an element or attribute whose type names a user-declared
type in the schema's own target namespace must be a valid value of that type
(XSD 1.0 e-props-correct.2 / a-props-correct.2). The check runs after the named
types are compiled and validates with the compiled SimpleType validator (the one
instance validation uses), so whitespace and facet handling match. Built-ins are
handled at structure time; imported, foreign, and inline anonymous types are left
alone via a namespace gate.
This exposed a pre-existing compile defect: a simpleType restriction read its base
only from the base attribute, so an inline anonymous <simpleType> base (a list or
union with no base attribute) was dropped and the type collapsed to an atomic
string, making a length facet count characters instead of list items. The
restriction branch now resolves the inline child as its base, preserving list and
union variety. This corrects both schema-time value checking and instance
validation.
XSTS: invalid schemas accepted 462 to 454; invalid instances accepted 160 to 158;
no other bucket raised.
An xs:attribute declaration carrying a default value constraint was accepted
alongside use="required" or use="prohibited"; XSD 1.0 src-attribute.2 requires
use to be optional when default is present (a default makes the attribute
optional). The check rides the existing value-constraint walk, gated to
XSD-namespace attribute declarations reading the unprefixed default/use, and is
per declaration. Only fixed (not default) may pair with use="required".
XSTS: invalid schemas accepted 469 to 462, no other bucket raised. Disclosed
under-rejection: a whitespace-padded use=" required " (NMTOKEN actual value
"required") is not flagged.
An xs:element or xs:attribute declaration carrying both a default and a fixed
value constraint was accepted; XSD 1.0 src-element.1 / src-attribute.1 make the
two mutually exclusive. The check rides the existing value-constraint structure
walk, gated to XSD-namespace declarations and reading the unprefixed
default/fixed (a foreign same-local-name attribute is not the value
constraint). It is per declaration, so a default on a global declaration and a
fixed on a reference to it (two separate items) remains valid.
XSTS: invalid schemas accepted 479 to 469, no other bucket raised.
An element declaring substitutionGroup="head" whose own type is unrelated to
the head's type was accepted; XSD 1.0 e-props-correct.4 requires the member's
type be validly derived from the head's. The check runs after named types
resolve and reuses the restriction/extension + built-in-lattice derivation
oracle (an ur-type head admits any member).
It is conservative so it never rejects a valid member: it runs only for a
self-contained schema (no import/include/redefine, where the derivation table
is complete and single-namespace names are unambiguous), reads the member's
type from its own type attribute and the head's from the top-level element
declarations only (so a local element of the same name supplies neither), and
stands down when either type is a list or union (cos-st-derived-ok clauses
2.3/2.4 union-member derivation is not modelled).
The post-namedTypes consistency calls move into a postNamedTypeErrors
aggregator alongside consistencyErrors. XSTS: invalid schemas accepted
480 to 479, no other bucket raised.
An xs:all group may only be the whole content model of a complex type. A direct
all nested in a sequence/choice was already rejected; this closes the case where
an all is reached through a <group ref> to a named group whose content is an all
and that reference appears inside a compositor.
The all-group pool is qualified by the defining schema's target namespace, and a
reference by its prefix binding, so an imported or redefined same-named group is
not confused with a local one (two over-rejections found by an adversarial
critic: a redefined all-group whose content is overridden, and an imported
all-group shadowing a local non-all group; both fixed and tested). A group
redefined to non-all content is exempt.
XSTS: invalid schemas accepted 481 to 480, no other bucket raised. Disclosed
under-rejections: a redefine that keeps the group an all, and cross-namespace
misplacement of an imported all-group.
An attribute or element whose type is or derives from xs:ID may not carry a
default or fixed value (a-props-correct.3 / e-props-correct.5). Such schemas
were accepted.
The check runs after named simple types are resolved, so a named user type
derived from xs:ID is recognized (not just the built-in and inline forms). The
type reference's namespace is resolved from its prefix: only an ID in the XSD
namespace is the built-in, and a user type is ID only when it resolves to one
whose base is xs:ID. An adversarial critic caught a namespace-blind over-
rejection (a user type named ID, or an imported foo:ID, mistaken for the
built-in); the namespace resolution fixes it, with a regression test.
XSTS: invalid schemas accepted 489 to 481 (attZ013a/b, elemZ032a/b,
valueconstraint01001m2/m3/m5/m6 now rejected), no other bucket raised.
A complex type's or attribute group's attribute-use set (flattened through
attributeGroup references) may not contain two distinct declarations with the
same name and namespace (ct-props-correct.4 / ag-props-correct.2), nor more than
one attribute whose type is or derives from xs:ID (ct-props-correct.5).
Declarations are deduplicated by the identity of their declaring node, so the
same declaration reached by two attribute-group paths (a diamond) is one use,
not a clash. Names carry their resolved namespace (form / attributeFormDefault
for locals, the prefix binding for refs), so a local attribute and a reference
to a same-local-name attribute in an imported namespace do not collide. The
check runs after named simple types are resolved, so a named user type derived
from xs:ID is recognized, not just the built-in and inline forms.
XSTS: invalid schemas accepted 505 to 495 (attQ008/009/012/013, attgD006/007/
009/017, ctM003/004 now rejected), no other bucket raised. An adversarial critic
caught a false positive on same-local-name-different-namespace (fixed by
resolving the ref namespace) and a missed named-user-ID type (fixed by ordering
the check after type resolution). Disclosed under-rejections: the set inherited
through extension is not composed in; a duplicate needing an unprefixed
reference with no default namespace is not detected.
A model group may not contain itself (mg-props-correct.2), an attribute group
may not reference itself transitively (ag-props-correct.3), and a substitution
chain may not loop (e-props-correct.6).
The model-group check follows references only within a definition's own
compositor tree, stopping at element and type boundaries: a group referenced
inside an element's content is a recursive data structure (permitted), not the
group containing itself (particlesZ010, addB077). A reference is followed only
when it resolves to the schema's own target namespace; a group or attribute
group redefined in xs:redefine ends the chain (its single self-reference is the
legal redefinition).
XSTS: invalid schemas accepted 516 to 505 (attgC010/020/031, attgD015,
groupB012-015, addB180, xsd009.e, xsd010.e now rejected), no other bucket
raised. A genuine cycle introduced across two redefined definitions is a
disclosed under-rejection (an adversarial critic's only finding).
A complex or simple type may not derive, transitively through its base, from
itself. Such schemas were accepted. The check walks the base chain built from
each type's complexContent/simpleContent extension/restriction base (and a
simpleType's restriction base), following an edge only when the base resolves to
the schema's own target namespace.
Three exceptions keep it from over-rejecting: element content that recurses (an
element of the type's own type) is not a derivation and is not followed; a type
redefined in xs:redefine legitimately names itself as its base, so redefined
types end the chain; and the ur-types (anyType/anySimpleType/anyAtomicType,
which the schema-for-schemas bootstraps with self-referential definitions) end
the chain. The target-namespace gate prevents a local type whose base is a
same-local-name type in another namespace from being mistaken for a self-cycle
(found by an adversarial critic; the engine resolves by local name).
XSTS: invalid schemas accepted 519 to 516 (addB101, elemM003, xsd007.e now
rejected), no other bucket raised. Union/list and genuine cross-namespace
derivation cycles are a disclosed under-rejection.
The W3C "Particle Valid (Restriction)" MapAndSum rule (XSD 1.0 §3.9.6): a derived
sequence restricting a base choice is valid iff each derived particle restricts
some base branch AND the derived sequence's occurrence count times its number of
particles lies within the base choice's occurrence range.
PureXML applied the same-compositor occurrence subsumption (the sequence's own
range against the choice's), which both rejected a valid restriction
(sequence{2,4}(e1,e2) of choice{3,9}(e1,e2): 4..8 elements fit 3..9) and accepted
an invalid one (a two-member sequence restricting a single-occurrence choice,
count-product 2 > 1). The count-product is the spec's measure, matched to the
reference implementation (checkMapAndSum/checkOccurrenceRange).
The (sequence, choice) branch in groupValid is retained: it is reached by the
(element, group) RecurseAsIfGroup path, which wraps an element as a one-particle
sequence against a base choice.
XSTS: valid schemas rejected 2 to 1, invalid schemas accepted 520 to 519, no
other bucket raised. The remaining valid-rejected case (particlesZ001) is a
documented spec ambiguity (RecurseAsIfGroup), recorded in docs/xsts-deviations.md.
Two more Particle Valid (Restriction) over-rejections.
A derived particle that can never contribute a child (its own maxOccurs="0", or
a group whose every member is content-free) accepts only the empty sequence, so
it validly restricts any emptiable base. Checked before the structural cases,
which previously recursed into the members of a maxOccurs="0" group and rejected
it for their own occurrences (particlesW006).
Substitution-group expansion now lists members in declaration order: the closure
walk used a popLast() stack that reversed them, so a reference to a head expanded
to choice(head, m2, m1) and an order-preserving RecurseLax restriction by
choice(m1, m2) could not map (elemZ027a). The choice/choice restriction stays
order-preserving, as the W3C suite requires (choice(b|a) is not a valid
restriction of choice(a|b), particlesT002); only the expansion order is fixed.
The expansion is shared with instance validation and the determinism check, both
order-neutral.
XSTS: valid schemas rejected 4 to 2, no other bucket raised. The pinned XSTS
baselines (never asserted under a plain swift test, the suite being opt-in) are
corrected to the measured 2/520/180/160.
An `<xs:element ref="p:foo">` had its prefix stripped and the element assumed to
be in the schema's target namespace. That mis-qualified every reference to an
imported element: a restriction of an `##other` wildcard by such a reference was
rejected (the element looked like the target namespace, which `##other` forbids),
and the same mis-qualification rejected valid instances whose elements were
correctly in the imported namespace.
The reference's QName is now resolved by standard XML namespace rules against the
schema root's xmlns bindings: a bound prefix gives its namespace (an imported
element lands in the imported namespace), an unprefixed name the default
namespace, or no namespace when none is declared. Substitution-group members
stay in the target namespace (this schema's own globals).
The content model is shared between schema-restriction checks and instance
validation, so both improve. XSTS: valid schemas rejected 32 to 4, invalid
schemas accepted 526 to 520, valid instances rejected 214 to 180, invalid
instances accepted held at 160.
Disclosed limitations: a prefix re-bound on an inner element is not honored,
cross-namespace substitution-group members are not modeled, and a dangling
reference is not yet flagged.
Particle Valid (Restriction) derives its pairwise check over the spec's
normalized content model: pointless particles are eliminated (maxOccurs=0
members dropped, a {1,1} single-member group unwrapped, a nested
same-compositor {1,1} group spliced), all language-preserving. The outer
occurrence-range check was wrongly applied to the cross-kind pairings, where
it compares a derived particle's own range against a base group's or
wildcard's. It is now applied only to the same-kind pairings (element/element,
wildcard/wildcard, group/group) and element/wildcard; element/group
(RecurseAsIfGroup), group/element, and group/wildcard match occurrence inside
the case, against a member or as an effective total range.
A group restricting a wildcard (NSRecurseCheckCardinality) is valid when every
leaf is admitted and the effective total occurrence range is within the
wildcard's; a group restricting a single element is bounded the same way, by
that element's name, type derivation, and occurrence range.
XSTS: valid schemas rejected 45 to 32, invalid schemas accepted 528 to 526,
no other bucket raised.
The bar for shipping PureXML in an IDE: correctness total and characterized,
false positives at zero, not "passes the suite." Four stoppers (reject no valid
input; accept no faulty input; characterize correctness via differential + fuzz,
no silent under-rejection debt; interactive safety: bounded resources, located
recoverable errors) plus a per-iteration checklist and definition of done.
Wired into the per-iteration protocol (schema-validity-burndown.md) and the
session Read First list (CLAUDE.md) so it is read every iteration.
