An element restricting a base choice was always checked by the
effective-total-range approximation, which exists for one reason: the
literal section 3.9.6 pairing has no allowance for occurrence counts
accumulated across the choice's own repetitions and over-rejects
W3C-read-valid schemas (particlesZ001, whose manifest calls the rule
ambiguous). That ambiguity is impossible when the choice cannot repeat:
at {max occurs} = 1 every accepted child comes from one traversal of one
branch, so the literal rule governs. elementRestrictsChoice now applies
the literal pairing (the derived element, with its own occurrence range,
must validly restrict some non-pointless branch) when base maxOccurs is
1, and keeps the effective-range reading only for repeating choices.
Closes XSTS particlesHa161 (a derived a{0,1} is not a pointwise
NameAndTypeOK restriction of a required branch a{1,1}) and particlesM034
(a derived d1+ cannot map into the branch (d1*, d2+): the required d2 is
unmapped and not emptiable). The settled-valid neighbors that pin the
boundary from the other side all still pass: Ha160, M002, M003, M035,
and the Z001 shape, each now a regression test in
XSDParticleChoiceRestrictionTests. invalid-schemas-accepted 6 -> 4.
particlesIg004 is reclassified as spec-divergent (4 -> 3): section 3.3.2
maps block="#all" and the explicit substitution/extension/restriction
list to the SAME {disallowed substitutions} set, so NameAndTypeOK clause
5's superset requirement holds and the invalid expectation has no spec
path; it is jointly unsatisfiable with the settled Ig001/002/003 (equal
sets, valid) and Ig005 (#all covers the triple, valid).
The remaining three schema deviations are documented divergences, not
open bugs: Ik025/Ik027 (cos-st-derived-ok 2.2.4 reading) and schN12,
newly adjudicated in docs/xsts-deviations.md: its chameleon redefine
fails src-redefine 6.2.2 on NameAndTypeOK clause 1 (no-namespace locals
vs qualified locals), but 6.2.2 cannot be enforced uniformly because
mgO013's settled-VALID schema also violates it; any uniform enforcement
trades a false negative for a valid-schema-rejected false positive.
Also corrects the stale deviations-doc row that listed the already-closed
particlesIk011 and omitted schN12; the failures file is the authority.
Gates: full XSTS release suite green at 0/3/0/4 (valid buckets unchanged
at zero), libxml2 differential green at 244 pinned entries (Ha161 and
M034 newly pinned, both compile under libxml2), full local gate green.
NameAndTypeOK clause 1 (XSD 1.0 section 3.9.6, Elt:Elt) pairs the declared
element declarations and requires the same {name} and {target namespace};
no clause grants substitution an allowance. The substitution-group
expansion rewrites the compiled term (an abstract head is dropped, members
join a synthetic choice), so the checker previously compared the expanded
names and a head-for-member swap looked name-identical.
Particle.declaredReference now preserves the pre-expansion identity of an
element-ref particle, and the restriction check pairs the declared
references before the structural pairing table (extracted as
structuralValid). Closes XSTS elemZ026 and particlesV020:
invalid-schemas-accepted 8 -> 6, valid buckets unchanged at zero.
Stricter than libxml2, which does not implement this edge; elemZ026 is
pinned in the differential baseline accordingly.
particlesZ008, byte-for-byte the same construct as elemZ026 but carrying
a settled valid expectation contradicting elemZ026's settled invalid, is
reclassified as a spec-divergent schema test with the adjudication
recorded in the suite (exclusion tables split into
Tests/XSTSSuiteExclusions.swift for file length).
Separately attributed: the differential baseline gains ten further pinned
entries (addB070a, addB108, addB177, attO025, elemZ027_c, errC008,
groupH021v, notatB005, notatE002, particlesEb040) that reproduce with
this change stashed. They are pre-existing drift from earlier main-line
fixes, not effects of this commit. Baseline 231 -> 242, differential
green.
particlesZ001 declares Derived as a complexContent restriction of Base and an
instance whose <elem> (type Derived) content is annotation + four elements. The
suite pairs a valid schemaTest with an invalid instanceTest, but the two are
mutually exclusive for a consistent conformant processor, and the manifest's own
documentation labels the governing rule, Particle Derivation OK (RecurseAsIfGroup),
ambiguous.
cvc-complex-type (§3.4.4) validates an instance against the element's own
{content type}: for <elem> that is Derived = sequence(annotation?, element*),
which the instance satisfies, so the instance is valid. Particle Valid
(Restriction) / RecurseAsIfGroup (§3.9.6) is a schema-assembly constraint that is
never re-run during instance validation, so there is no path from an accepted
schema to an invalid instance. The only way to reach the suite's invalid is to
reject the schema as an invalid restriction, which is a valid-schema-rejected
false positive and would still fail the suite's own valid schemaTest. libxml2
accepts both schema and instance, agreeing with PureXML.
Exclude particlesZ001.i via specDivergentInstances with the spec-grounded
justification in code and docs/xsts-deviations.md. invalid-instances-accepted
5 -> 4; full XSTS in release now 0/8/0/4, both false-positive buckets at 0. No
library behaviour change. Confirmed by an adversarial analysis that found no
spec-coherent reading yielding valid-schema + invalid-instance.
mgO013 redefines a model group `group` to `all{e1(1,1)}`, redefining a base
`all{e1(0,0)}` (e1 forbidden) without referencing itself. The suite pairs a
`valid` schemaTest with an `invalid` instanceTest, but the two are mutually
exclusive for a consistent conformant processor.
src-redefine clause 6.2.2 makes a no-self-reference group redefine a restriction
obligation: the redefined group must be a valid restriction of the original.
`all{e1(1,1)}` is not a valid restriction of `all{e1(0,0)}` (Particle Valid
(Restriction), Occurrence Range OK: derived max 1 > base max 0), so enforcing
6.2.2 rejects the schema, a valid-schema-rejected false positive against the
schemaTest's own `valid` verdict. Yet §4.2.2 makes the referencing type's group
reference resolve to the redefined `all{e1(1,1)}`, against which the single-e1
instance is valid. There is no third path: reaching instance-invalid requires
rejecting the schema. libxml2 accepts both, agreeing with PureXML and the
schemaTest. An adversarial analysis confirmed no reading yields valid-schema +
invalid-instance.
Exclude mgO013.i via specDivergentInstances with the spec-grounded justification
in code and docs/xsts-deviations.md. invalid-instances-accepted 7 -> 6; full XSTS
in release now 0/8/0/6, both false-positive buckets at 0. No library behaviour
change: matching the suite would introduce the false positive the production bar
forbids.
The sch* schema-location cluster, dissected per case rather than swept. An
adversarial analysis separated the contested tests from the genuine deviations.
schA2.i and schA5.i are contested and now excluded (invalid-instances-accepted
10 -> 8). Each group's own documentation states the inline xsi:schemaLocation
hint "is ignored" and the instance "must conform to" the collection schema, but
under exactly that prescribed behavior the instances are valid: libxml2 validates
schA2.xml against its a.xsd and schA5.xml against its a.xsd+b.xsd collection. So
the suite's invalid expectation contradicts its own documentation. Reaching
invalid requires letting the instance's hint override a schema already loaded for
that namespace, which XSD 1.0 §4.3.2 makes optional (strategy 1: do nothing when a
schema for the namespace is already known) and which would be a direct false
positive for an IDE: an instance file forcing rejection against the project's own
schema. PureXML's policy (honor hints but keep the collection's declarations on
conflict) is conformant.
schA7.i is NOT excluded. The adversarial pass showed its invalid IS spec-grounded:
with hints ignored, the instance's b:e1 (namespace ns-b) has no declaration in the
a.xsd-only collection and is demanded by root's default-strict ##any wildcard, so
§3.10.4 makes it invalid. PureXML accepts it only by honoring the ns-b hint to
gap-fill, the other conformant §4.3.2 strategy. A genuine policy-dependent
deviation, left in the count and documented.
schU3.i/schU4.i/schU5.i are also left in the count, documented as a deferred
under-rejection in an underspecified area: circular redefine chains where an
attribute group is redefined to conflicting types across an include/redefine
cycle. The suite expects the int reading (so c1="a" is invalid); PureXML resolves
the laxer type and accepts. libxml2 does not support the construct at all (rejects
the schema: multiple redefinition not supported), so matching the suite means
out-resolving the reference on a construct whose cyclic composition order the spec
does not pin down.
Full XSTS in release now 0/8/0/8, both false-positive buckets at 0. No library
behavior change; reasoning recorded in code and docs/xsts-deviations.md.
The W3C XSTS group anyURI_a004_1339 pairs a `valid` schemaTest with an `invalid`
instanceTest, but the two expectations are mutually inconsistent, so PureXML's
acceptance of the instance is spec-correct rather than an under-rejection.
The schema's include, import, and redefine carry unresolvable schemaLocations
(ftp://, gopher://, mailto:). XSD 1.0 §4.2.1 makes an unresolvable include "not
an error" (no inclusion performed), the import likewise, and the empty redefine
imposes no resolution requirement (src-redefine clause 1 gates that on
non-annotation children), so the schema is conformantly valid, agreeing with the
schemaTest. The only components the instance exercises (root, bar, st, att) are
fully locally defined, and every instance value is byte-identical to one of the
type's six anyURI enumeration members, so a strict assessment of the document
element is valid. There is no spec path to valid-schema + invalid-instance.
libxml2 is no authority here: it non-conformantly fails to compile the schema on
the unresolvable include, so it disagrees with the suite's valid-schema verdict
too. Matching the suite's invalid-instance expectation would mean rejecting a
valid instance whenever an include cannot be resolved, a false positive an IDE
editing loop hits constantly, which the production bar forbids.
Exclude it from the counts via specDivergentInstances, mirroring how the suite's
own status="queried" entries are skipped, with the spec-grounded justification
recorded in code and docs/xsts-deviations.md. invalid-instances-accepted 11 -> 10;
full XSTS in release now 0/8/0/10, both false-positive buckets at 0. Two
independent adversarial analyses failed to refute the instance's validity.
XSD 1.0 (Datatypes Appendix F) defines the regex escape \i as the characters
that may begin an XML Name (Letter | '_' | ':') and \c as the characters that
may appear in one (NameChar), citing the XML 1.0 Second Edition productions:
the enumerated, Unicode-2.0-era character classes of XML 1.0 Appendix B
(BaseChar, Ideographic, CombiningChar, Digit, Extender). \i and \c were wired
to PureXML.Parsing.XMLCharacter, which correctly implements the Fifth Edition
Name production used to parse live XML, but the two genuinely differ: the Fifth
Edition admits whole blocks where the Second Edition has gaps, so \c
over-accepted, e.g. the combining mark U+0346 (COMBINING BRIDGE ABOVE), which
Appendix B's CombiningChar excludes (its first range ends at U+0345).
Add XSDNameCharacter, transcribed clean-room from the Appendix B grammar
(identical across editions one through four) and stored as sorted, non-
overlapping code-point ranges with binary-search membership; the parser's own
name check is untouched. The two tables reproduce the W3C XSTS \i/\c
enumerations (reZ005, reZ006: 34516 and 35122 code points) exactly and in both
directions, so the more restrictive definition rejects no spec-valid input:
there is no false-positive surface.
Verified over the full W3C XSTS corpus in release: invalid-instances-accepted
12 -> 11 (reZ006i, whose value is U+0346, now rejects), both false-positive
buckets at 0. Full suite (1834 tests) green; an adversarial critic brute-forced
the binary search over the whole code-point space (0 mismatches), confirmed the
tables are sorted and non-overlapping and equal the oracle sets, and that the
parser's Fifth Edition path is left intact. Found by triaging the invalid-
instance tail to a single boundary code point (U+0345 valid vs U+0346 invalid)
and reading the XML 1.0 Appendix B grammar.
XSD 1.0 fixes the regular-expression repertoire at Unicode 3.2 (Datatypes
Appendix F), where U+1369..U+1371 (ETHIOPIC DIGIT ONE..NINE) are general
category Nd, each carrying a decimal value; a later Unicode reclassified them
to No. XSDCategory derives \d (== \p{Nd}) from the running platform's category,
pinned to 3.2 by an assigned-in-3.2 gate plus a per-code-point override table,
but the table omitted these nine, so \d used the modern No: a valid \d-constrained
Ethiopic digit was wrongly rejected (cardinal-rule false positive) and a
\D-constrained one wrongly accepted.
Pin the nine to Nd. This is the complete set of code points that were Nd at 3.2
and are not today; the reverse direction is empty (every later-Nd point was
unassigned at 3.2 and already reports Cn), verified by diffing the 3.2 and current
Unicode Nd sets. \w is unaffected: an Ethiopic numeral is category N either way,
neither punctuation, separator, nor other.
Verified over the full W3C XSTS corpus in release: invalid-instances-accepted
14 -> 12 (reT17, reT38, the \D cases, now reject), and the matching \d-valid cases
reS17/reS38/reZ004 no longer need a suspect-test exclusion and pass as ordinary
agreements, with both false-positive buckets at 0. Full suite (1830 tests) green;
an adversarial over-rejection critic re-derived the Nd delta bidirectionally and
confirmed no false-positive surface. A regression-pinning test exercises the
production regex path, including the boundary that U+1372 (No even at 3.2) is
not \d.
Triaging the 24 open XSTS failures against libxml2 (the stated reference)
separates genuine reference-aligned bugs from stricter-than-libxml2 scope
decisions. On the schema side, 9 of 10 are accepted by libxml2 too (Xerces-
level strictness, false-positive-prone); only particlesZ030_d was a real
gap, now fixed. The instance tail's reference-aligned cases are intricate
and need per-case analysis. Guides the burndown toward bugs, not non-bugs.
Adds a dated accounting to the XSTS deviation worklist: the labelled-suite
ratchet stands at 0/74/0/133 with false positives held at 0 across the campaign.
Partitions the remaining 74 into fixable-but-intricate debt (redefine
restriction, NameAndTypeOK union membership, wildcard UPA, chameleon/cross-ns
composition, heterogeneous particle edges) and irreducible-by-design or disclosed
items (RecurseAsIfGroup spec ambiguities, the deliberate regex-tolerance), with
the root cause and false-positive risk for each so the burndown is driven by
understanding. Records that the adversarial critic loop caught three genuine
false positives this session that the suite and differential both missed.
Ship the OpenAPIKit-style validation layer (builtin rules, warnings channel,
Validator tiers, coverage gates) and wire XSD compile checks through it. Advance
#161 with chameleon include validation, multi-container reference resolution,
and include targetNamespace composition rules so merged schema sets compile
correctly when a schemaLoader is provided.
Cut the second pre-release: schema-validity burn-down to XSTS baselines
1/266/171/155, component child-content shape checks, and synced conformance
docs. CHANGELOG updated for v0.2.0.
The W3C "Particle Valid (Restriction)" MapAndSum rule (XSD 1.0 §3.9.6): a derived
sequence restricting a base choice is valid iff each derived particle restricts
some base branch AND the derived sequence's occurrence count times its number of
particles lies within the base choice's occurrence range.
PureXML applied the same-compositor occurrence subsumption (the sequence's own
range against the choice's), which both rejected a valid restriction
(sequence{2,4}(e1,e2) of choice{3,9}(e1,e2): 4..8 elements fit 3..9) and accepted
an invalid one (a two-member sequence restricting a single-occurrence choice,
count-product 2 > 1). The count-product is the spec's measure, matched to the
reference implementation (checkMapAndSum/checkOccurrenceRange).
The (sequence, choice) branch in groupValid is retained: it is reached by the
(element, group) RecurseAsIfGroup path, which wraps an element as a one-particle
sequence against a base choice.
XSTS: valid schemas rejected 2 to 1, invalid schemas accepted 520 to 519, no
other bucket raised. The remaining valid-rejected case (particlesZ001) is a
documented spec ambiguity (RecurseAsIfGroup), recorded in docs/xsts-deviations.md.
A keyref's refer must name a key or unique constraint. A refer pointing at a
non-existent constraint was accepted. keyrefReferErrors collects all key/unique
names and all keyref refers across the document (so a forward refer, textually
before its key, is handled), then flags any refer with no matching name. Matched
by local name, like the rest of the compiler. Skipped when the document declares
import/include/redefine, whose externals the default compile does not load.
XSTS: invalid schemas accepted 891 to 884 (-7); valid schemas rejected held at
72; instance buckets unchanged.
Within a schema, global type names (simpleType and complexType share one symbol
space), global element names, global attribute names, named model-group names,
and named attribute-group names must each be unique, and identity-constraint
names must be unique across the document. A duplicate was accepted, with the
later definition silently overwriting the earlier one.
Only the document's own globals are examined (xs:redefine children and unloaded
included documents are not direct globals), so a redefinition is not mistaken
for a clash; the symbol spaces are kept separate, so a type and an element may
share a name.
XSTS: invalid schemas accepted 909 to 891 (-18); valid schemas rejected held at
72; instance buckets unchanged.
Within one complex type's content model, two element declarations with the same
name must have the same type definition. Two same-name elements with different
types were accepted. elementDeclsConsistentErrors collects element particles
across the content model -- descending through model groups and the content-
derivation wrappers, but never into an element's own nested type, which is a
separate content model -- and flags a name seen with more than one distinct type
key (a type reference by local name, a distinct token per inline type, or the
untyped ur-type).
Modeled clean-room on Xerces (libxml2 enforces this only partially). Substitution-
group expansion is not folded in, an accepted under-rejection.
XSTS: invalid schemas accepted 925 to 909 (-16); valid schemas rejected held at
72; instance buckets unchanged.
The attribute sibling of the content-model table. Each XSD component admits a
fixed set of attributes; an unprefixed attribute outside that set (a typo like
nullable, a stray foo, name on a sequence) was accepted because nothing checked
attribute applicability. allowedAttributes now lists each component's admitted
unprefixed attributes (the permissive union of global and local use), and
attributeApplicabilityErrors rejects the rest, plus ref co-occurring with name
or type.
Foreign (prefixed) attributes and namespace declarations are the schema author's
own and are not checked, so no valid schema is rejected.
XSTS: invalid schemas accepted 956 to 925 (-31); valid schemas rejected held at
72; instance buckets unchanged.
XSD 1.0 gives a named model-group definition's content as
(annotation?, (all | choice | sequence)) -- exactly one compositor. The
allowedChildren set-membership check allowed any number, so an xs:group with two
compositors, or with none, was accepted. namedGroupErrors now enforces the
cardinality, gated on the node being a definition (has a name) so a group
reference is never flagged.
XSTS: invalid schemas accepted 977 to 956 (-21); valid schemas rejected held at
72; instance buckets unchanged.
An occurrence range like minOccurs="5" maxOccurs="2" is empty and illegal in
XSD, but the validator accepted it (only the individual values were checked).
occurrenceOrderErrors now compares the two on a particle that carries both: the
comparison is by canonical magnitude (sign and leading zeros stripped, length
then lexicographic), so it holds beyond a machine integer; maxOccurs="unbounded"
is never exceeded and an equal single-point range is allowed.
XSTS: invalid schemas accepted 997 to 977 (-20); valid schemas rejected held at
72; instance buckets unchanged.
XSD 1.0 allows a complexType's content to be simpleContent, or complexContent,
or a model group followed by attributes, never a mix. The allowedChildren
set-membership check permitted illegal combinations, so a complexType with two
simpleContent, simpleContent beside a sequence, or two model groups was accepted.
complexTypeContentErrors now enforces, over the complexType's direct children:
at most one of simpleContent/complexContent; when one is present, no direct model
group and no direct attribute (those belong inside the extension/restriction);
and at most one model group. Only direct children are examined, so attributes
nested inside a derivation are not flagged.
XSTS: invalid schemas accepted 1051 to 997 (-54); valid schemas rejected held at
72; instance buckets unchanged. Also folds in a behavior-identical lint cleanup
of the reference-walk external-definition check.
Every QName a schema names must resolve to a declared component or a built-in:
type/base/itemType/memberTypes to a type, element/attribute/group/attributeGroup
ref to the matching global component, and element substitutionGroup to a global
element. An undeclared reference was previously accepted at compile time and only
surfaced lazily during instance validation, if at all.
References are matched by local name, with the value trimmed (a QName attribute
is whiteSpace=collapse). The whole check is skipped when the document declares an
import/include/redefine, since the default compile does not load externals and
the pools would be incomplete; that guard is gated on the XSD namespace so a
foreign element named "import" does not trip it.
XSTS: invalid schemas accepted 1092 to 1051 (-41); valid schemas rejected held
at 72 (an interim version regressed one valid schema on a whitespace-padded
base=" xs:string ", fixed by trimming); instance buckets unchanged.
Each pattern facet value is now compiled when the schema is, instead of only
lazily at instance time (try?) where an unparseable pattern was silently
ignored. A structurally malformed pattern (unbalanced parentheses, a quantifier
with nothing to repeat) is rejected.
The check is conservative against over-rejection: it rejects only the
unambiguous structural RegexError kinds (.unbalanced, .danglingQuantifier) and
tolerates the kinds that fire on otherwise-valid patterns the engine cannot
fully handle (.empty for the empty pattern, .unsupported for an untabulated
\p{Is...} block such as a surrogate or historic block, .badQuantifier for the
lenient {,m} form, and .badEscape/.badClass which could fire on valid character-
class subtraction). So a{2,1} and a trailing backslash remain accepted (an
under-rejection), but no valid schema is rejected.
XSTS: invalid schemas accepted 1129 to 1092 (-37); valid schemas rejected held
at 72; instance buckets unchanged.
Extends the compile-time attribute-model check. A component's name must be a
valid NCName, the reference attributes type/base/ref/itemType/refer/
substitutionGroup must be lexical QNames, and memberTypes a list of QNames.
These were unchecked, so name="", name="123", name="a:b", or type=":_" left the
schema wrongly accepted.
Only the lexical form is checked (a QName's prefix is resolved elsewhere), so a
valid reference is never rejected; each of these attribute names has one type
throughout XSD, so checking by local name is safe.
XSTS: invalid schemas accepted 1190 to 1129 (-61); valid schemas rejected held
at 72 (no over-rejection); instance buckets unchanged.
The attribute-model sibling of the structural content-model check, folded into
the same schema-document walk. Enumerated attributes (form,
elementFormDefault, attributeFormDefault, use, processContents, mixed, abstract,
nillable) and the occurrence attributes (minOccurs a nonNegativeInteger,
maxOccurs a nonNegativeInteger or unbounded) are checked against their fixed
value space. An out-of-space value (use="foo", mixed="yes", minOccurs="-1") was
previously ignored and the schema accepted.
Each of these names has one meaning throughout XSD, so checking by local name is
safe. Only unprefixed attributes on XSD-namespace elements are checked, so a
foreign attribute is never flagged; values are trimmed before comparison.
XSTS: invalid schemas accepted 1313 to 1190 (-123); valid schemas rejected held
at 72 (no over-rejection); instance buckets unchanged.
The general mechanism for the structural tail of the schema-validity gap, in
place of hand-coding a rule per component. A data-driven table encodes the XSD
1.0 schema-for-schemas child content model; at compile time it rejects any
component child the model does not admit, more than one annotation (or an
annotation that is not first, except on schema/redefine, which allow several),
and an identity constraint missing its selector or field.
The walk visits only XSD-namespace elements and skips appinfo/documentation
foreign content, so foreign children are never flagged; restriction/extension
use the permissive union of their simple/complex/simpleContent contexts, so a
child valid in some context is never rejected. One table covers every component.
XSTS: invalid schemas accepted 1688 to 1313 (-375); valid schemas rejected held
at 72 (no over-rejection); instance buckets unchanged.
The id attribute on any XSD component is of type xs:ID, so each value must be a
valid NCName and unique within the schema document. The values were never
checked, so a malformed (id="", id="123") or duplicated id left the schema
wrongly accepted across many categories (idA-idK, attribute groups, complex
types).
The schema-document tree is walked once, collecting id attributes. Two guards
keep valid schemas from being rejected: appinfo/documentation subtrees are
skipped (their foreign content carries its own non-xs:ID id attributes), and
only the unprefixed id is treated as the component identifier (a foreign xml:id
or pre:id is a different attribute). Findings join the shared parse-time
diagnostics.
XSTS: invalid schemas accepted 1789 to 1688; no other bucket raised.
Continue the facet-validity work. Per XSD Part 2 Datatypes 4.3, each
minInclusive/maxInclusive/minExclusive/maxExclusive and enumeration value must
be a valid value of the base type; minInclusive excludes minExclusive and
maxInclusive excludes maxExclusive; and the lower bound may not exceed the upper
bound (nor equal it when either side is exclusive, an empty value space),
compared in the base primitive's value space. Bound and enumeration values were
previously unchecked, so a malformed or out-of-space value (maxInclusive="",
enumeration="" on float) was accepted.
The bound-value check (which validates a bound against its base type) exposed a
gMonth lexical gap: only --MM was recognized, not the XSD 1.0 form --MM-- used
by valid schemas across the corpus. parseMonthOnly now accepts an optional
trailing -- (the --MM erratum form still works), with an optional timezone,
which removes 26 false schema rejections the bound check would otherwise have
caused and also fixes instance-side gMonth validation.
XSTS: invalid schemas accepted 2082 to 1789; valid instances rejected 233 to
214 (the gMonth fix); valid schemas rejected held at 72; invalid instances
accepted held at 165.
First cut at the schema-validity gap: the validator ran instance-validity rules
but not schema-validity rules, so it silently accepted invalid schemas. Per XSD
Part 2 Datatypes 4.3, a length-family facet value (length, minLength, maxLength,
totalDigits, fractionDigits) must be a valid nonNegativeInteger (totalDigits a
positiveInteger), length may not co-occur with minLength/maxLength, and the
integer ranges must be ordered. Malformed values (length="", "-1", "a", "1e2")
were silently dropped by numeric facet parsing and the schema accepted.
The definition is now read raw from the restriction and validated at compile
time. Well-formedness is checked independently of machine-integer range (via a
canonical digit string), so a huge but valid value is not misreported as
malformed, and ordering compares by length then lexically. Findings travel
through a shared reference-type diagnostics collector on the value-type parsing
context (surviving its copies) and join the model-level consistency findings
thrown from Schema.Document.init.
Limited to facets declared on the local restriction, so an inherited facet is
never wrongly flagged. The min/maxInclusive value-space rules, facet
applicability per base, and inherited co-occurrence remain for later iterations
(see docs/schema-validity-burndown.md).
XSTS: invalid schemas accepted 2461 to 2082, no change to any other bucket.
Resolve a built-in datatype named by xsi:type (xsd:int, xsd:decimal,
xsd:string), which previously resolved only against schema-declared types and
so reported "unknown" and wrongly rejected the element. The built-in is now
resolved when it validly substitutes for the declared type per Schema-Validity
Assessment: the ur-type (anyType/anySimpleType) admits any type, and a bare
built-in declared type admits any built-in derived from it through the XSD
Part 2 datatype lattice (new BuiltinType.derives(from:)). An invalid built-in
substitution is a located error, not a silent acceptance.
Resolving the override correctly exposed two pre-existing gaps, both closed so
the change carries no invalid-instances-accepted regression:
- A simple-typed element rejects non-schema-instance attributes. A simple type
declares no attribute uses, so an element of simple type (including via an
xsi:type override, and including an xsi:nil element) may carry only xsi:* and
namespace attributes. Checked before the nil short-circuit.
- Identity-constraint fields compare in their value space when the selected
node carries an xsi:type. unique/key/keyref tuples compared lexically, so two
xsi:type="xsd:decimal" fields 3.0 and 3 were treated as distinct and a unique
violation was missed. Each field value now carries the type read from the
node's xsi:type and compares via SimpleType.valueMatches when both share a
built-in base, otherwise lexically.
XSTS: valid instances rejected 248 to 233; invalid instances accepted held at
165. Identity tests corrected to spec-valid schemas (an element carrying an
attribute must be a complex type, not type="xs:string").
Updates the worklist to the current settled-expectation baseline
(72/2461/308/165) and records the twelve fixes landed. The remaining instance
deviations are root-caused individually: the particle clusters need
multi-document schema loading and xsi:schemaLocation honoring under a strict
wildcard (a new capability, the largest single lever); reI is pattern-attribute
whitespace normalization; test/elemT are heterogeneous grab-bags needing
per-case triage; the rest are identity-XPath and wildcard/simple-type edges.
The clean single-root-cause fixes are exhausted; the tail is deliberate work.
Refs #129, #122.
The runner read <expected validity> but ignored <current status>, so it counted
disagreements with W3C-disputed expectations as deviations. The 2006-11-06
archive marks 748 instanceTests (plus schemaTests) status="queried" with the
expected validity contested in W3C Bugzilla (for example reJ11.i: \p{Lu}* over
two uppercase letters, prior expectation valid, current invalid+queried, where
PureXML's accept is defensible). Those are now skipped: neither agreement nor
deviation, since the expectation is not authoritative.
Recalibrated baseline (settled expectations only): valid schemas rejected 74 to
72, invalid schemas accepted 2467 to 2461, valid instances rejected 369 to 338,
invalid instances accepted 210 to 179. About 70 apparent deviations were
disagreements with contested expectations, never bugs.
Refs #129, #122.
XSD 1.0 Datatypes 4.3.1 defines the unit of length only for string, anyURI,
binary, and list; for QName it is implementation-defined. We had counted
characters, rejecting valid QName values. Like Xerces and the XSTS NIST set we
now treat length/minLength/maxLength on QName as always satisfied. Lexical
QName validity is still enforced.
The suite's contributors disagree here (NIST: non-constraining; Microsoft: a
character count), which the spec does not resolve, so a single behavior cannot
satisfy both. Net XSTS: valid instances rejected 547 to 502; 4 Microsoft cases
move to invalid instances accepted (257 to 261), recorded as a deliberate
interpretation in docs/xsts-deviations.md.
Refs #147, #129.
Turns the raw failure list into a worklist driven by understanding each rule,
not by chasing the count. Three categories: (A) genuine correctness bugs in
instance validation worth fixing (union restriction facets ~200, QName length
~36, regex Unicode blocks ~155, duration ordering ~50, particles/int facets);
(B) schema-for-schemas structural validity, the bulk of the 2467 invalid
schemas accepted, unimplemented by design and documented as a deliberate
exclusion; (C) arguable cases needing investigation. Records the evidence
sampled per cluster.
Refs #145, #146, #147, #148.
