The two monolithic XSD content/identity rules only satisfied the OpenAPIKit
validation idiom at the wrong granularity: no per-constraint removability,
no isolable failing/near-miss tests. Split them into 32 named rules, each
finding tagged at its origin and cached per XSDContext so the walk still
runs once per document (the #101 decision against duplicating XSD's
recursive type resolution still holds). Wired the streaming path's ID/IDREF
and identity checks through the same framework instead of calling the
validators imperatively, and added the unwrapAndLookup combinator the rule
already named as required.
Also includes the mandated repo-wide swiftformat/swiftlint pass.
XSD 1.0 charProp is IsCategory | IsBlock, an exhaustive set: every Unicode
general category plus a fixed Unicode-3.1 block list. The block table was
missing the nine supplementary-plane blocks (Old Italic, Gothic, Deseret, the
Byzantine/Musical/Mathematical symbol blocks, CJK Unified Ideographs Extension
B, CJK Compatibility Ideographs Supplement, Tags) and the three surrogate
blocks, so parseCategory could not tell a real-but-untabulated block from a
bogus name and stayed lenient (RegexError.unsupported), accepting the pattern
\p{IsaA0-a9} and the schema declaring it (XSTS reK88).
Complete the table to the full XSD set: supplementary blocks with their ranges,
and surrogate blocks as valid names whose ranges hold no scalar (so they match
nothing). The recognised set is now authoritative, so parseCategory throws the
rejecting RegexError.invalidProperty for any unrecognised name. The now-dead
RegexError.unsupported case and the isMalformedProperty heuristic are removed.
Closes XSTS reK88: invalid-schemas-accepted 13 -> 12, valid-schemas-rejected
held at 0, instance buckets unchanged (full XSTS archive). Side effect: a
pattern naming a supplementary-plane block (e.g. \p{IsMusicalSymbols}) now
matches its characters instead of failing to compile, removing a latent
instance false positive. Block names/ranges taken from the XSD-supported list
in Xerces-J Token.java (clean-room: factual data only). The list is
spec-frozen at Unicode 3.1, so later blocks are rejected by design.
XSD 1.0 Datatypes Appendix F defines SingleCharNoEsc as [^\[\]] and admits a
nested class only as a -[...] subtraction, so a `[` inside a character class
that is not introduced by `-` is a syntax error. RegexParser treated such a
`[` as a literal scalar, so `[a[:xyz:]` compiled and the schema declaring it
was wrongly accepted (XSTS RegexTest_993, RegexTest_1477).
nextClassMember now throws a dedicated RegexError.unescapedClassBracket, kept
distinct from the generic, deliberately-lenient badClass so the change cannot
regress a valid-but-unsupported pattern. XSDSimpleParser.rejectsPattern treats
it as a genuine syntax error, so the declaring schema is invalid.
Closes RegexTest_993 and RegexTest_1477: invalid-schemas-accepted 15 -> 13,
valid-schemas-rejected held at 0, instance buckets unchanged (full XSTS
archive). Escaped [a\[c] and subtraction [a-z-[aeiou]] stay valid; the
unterminated case [a[:xyz: now faults on the inner [.
A \p{...}/\P{...} property name that can be no XSD charProp under any
repertoire (empty, the bare prefix Is, a non-block name carrying a non-letter,
or Is followed by a non-block character) is now a schema-validity error,
distinct from a well-formed but unrecognised category or block name, which
stays a lenient engine limitation.
XSTS invalid-schemas-accepted 203 to 200, no other bucket moved.
A `pattern` facet's value must be a valid XSD regular expression. Two
truncation cases were compiled with a non-rejecting error and the schema was
wrongly accepted:
- a trailing `\` with no character to escape (`a\`), and
- a `[` character class the expression ends before closing with `]`
(`a[`, `[a[:xyz:`).
Both are unambiguously invalid, never an engine limitation on an otherwise
valid pattern, so they now raise the dedicated `RegexError.incompleteEscape`
and `RegexError.unterminatedClass` and `rejectsPattern` treats them as
schema-validity errors. A completed escape (`\[`, `\\`), a closed class, and a
class containing an escaped `]` or a subtraction stay valid.
Caught thirteen invalid schemas: XSTS invalid-schemas-accepted 230 -> 217, no
other bucket moved.
A pattern facet with an unambiguously malformed regular expression was
compiled leniently and the schema accepted. Three such constructs are now
rejected: a reversed character-class range ([b-a]), an empty class ([]),
and a reversed bounded quantifier ({37,17}). The parser raises distinct
RegexError cases (reversedRange, emptyClass, reversedQuantifier) that the
schema-validity check treats as invalid.
Constructs the engine merely does not support (an untabulated \p{Is...}
block, the lenient {,m} form) stay non-rejecting, so a valid pattern is
never refused. XSTS invalid-schemas-accepted 262 to 244, no other bucket
moved; the 27 regex unit tests still pass.
Consolidates the in-progress #146 work; valid-instances-rejected reaches 4
on the 2006-11-06 XSTS archive (buckets now 1/262/4/147).
Regex: match Unicode scalars rather than grapheme clusters; the empty
pattern matches only the empty string; pin the category repertoire to
Unicode 3.2, the version the corpus is consistent at, so code points added
in 3.2 (Tagalog, Cyrillic Komi) are word characters while later additions
(the Tamil digit zero) stay unassigned and match \D. New XSDCategory and a
3.2 assignment table derived from DerivedAge, replacing the old Unicode31
files.
Content model: assess each child against the particle the content automaton
matched, not a by-name lookup (#180). Two same-named particles keep their
own fixed values, and an element matching a skip wildcard is no longer
validated against its global declaration. Selection is made reproducible
when an ambiguous schema slips past the UPA check.
XSTS runner: compile and merge every schemaDocument in a schemaTest, not
just the first; this also surfaced one previously-masked invalid-instance
acceptance (recorded in the baselines).
Also folds in attribute-group anyAttribute inheritance, extension wildcard
union, namespace-qualified type tables for multi-document
xsi:schemaLocation, per-container elementFormDefault, strict and lax
attribute wildcard validation, built-in type resolution, and xs:redefine
simpleType overrides.
Running the full W3C XSTS archive to completion exposed three crash and hang
classes, now fixed with regression tests, which unblocked the #129 first
measurement.
- Content-model and regex NFA builders now cap total state count (2^20),
degrading further occurrence/quantifier repetition to star, so nested
high-maxOccurs schemas no longer OOM (was 8 GB on the MS Particles set).
- Content-model validation advances one active state-set per child instead of
re-walking the prefix each time, removing the O(children^2) spin.
- Regex class subtraction [abc-[b]] no longer traps on a reversed range, and a
reversed range such as [z-a] is rejected cleanly.
XSTS first measurement baselined: 75 valid schemas rejected, 2467 invalid
schemas accepted, 604 valid instances rejected, 582 invalid instances accepted
across 14383 groups. The suite runs green in 8s, opt-in via XSTS_ROOT.
Refs #129, #122.
Close the two XSD-regex gaps from the audit:
- \p{...} and \P{...} now resolve Unicode general categories (one-letter
groups like \p{L} and two-letter codes like \p{Lu}/\p{Nd}) through
Swift's own scalar generalCategory, so no character tables are vendored.
A curated set of well-known \p{Is<Block>} blocks is supported; an
unknown category or block name is a compile error rather than a guess.
- Character-class subtraction [a-z-[aeiou]] removes the subtrahend from a
class, and nests ([a-z-[b-d-[c]]]) and composes with categories.
Categories and subtraction work both as standalone atoms and inside
character classes, so XSD pattern facets that use them now validate.
Closes#89
Add the PureXML.Regex namespace: a recursive-descent parser for the XSD
regex grammar (literals, ., character classes with ranges and negation, the
\d \w \s family and negations, single-char escapes, grouping, alternation,
and the ?, *, +, {n}, {n,}, {n,m} quantifiers) compiled to a Thompson NFA.
matches(_:) is whole-string as xs:pattern requires, and runs linear in the
input with no backtracking blow-up. \p{...} category escapes are not yet
supported.
Closes#59Closes#60Closes#30
