A \p{...}/\P{...} property name that can be no XSD charProp under any
repertoire (empty, the bare prefix Is, a non-block name carrying a non-letter,
or Is followed by a non-block character) is now a schema-validity error,
distinct from a well-formed but unrecognised category or block name, which
stays a lenient engine limitation.
XSTS invalid-schemas-accepted 203 to 200, no other bucket moved.
A `pattern` facet's value must be a valid XSD regular expression. Two
truncation cases were compiled with a non-rejecting error and the schema was
wrongly accepted:
- a trailing `\` with no character to escape (`a\`), and
- a `[` character class the expression ends before closing with `]`
(`a[`, `[a[:xyz:`).
Both are unambiguously invalid, never an engine limitation on an otherwise
valid pattern, so they now raise the dedicated `RegexError.incompleteEscape`
and `RegexError.unterminatedClass` and `rejectsPattern` treats them as
schema-validity errors. A completed escape (`\[`, `\\`), a closed class, and a
class containing an escaped `]` or a subtraction stay valid.
Caught thirteen invalid schemas: XSTS invalid-schemas-accepted 230 -> 217, no
other bucket moved.
A pattern facet with an unambiguously malformed regular expression was
compiled leniently and the schema accepted. Three such constructs are now
rejected: a reversed character-class range ([b-a]), an empty class ([]),
and a reversed bounded quantifier ({37,17}). The parser raises distinct
RegexError cases (reversedRange, emptyClass, reversedQuantifier) that the
schema-validity check treats as invalid.
Constructs the engine merely does not support (an untabulated \p{Is...}
block, the lenient {,m} form) stay non-rejecting, so a valid pattern is
never refused. XSTS invalid-schemas-accepted 262 to 244, no other bucket
moved; the 27 regex unit tests still pass.
Consolidates the in-progress #146 work; valid-instances-rejected reaches 4
on the 2006-11-06 XSTS archive (buckets now 1/262/4/147).
Regex: match Unicode scalars rather than grapheme clusters; the empty
pattern matches only the empty string; pin the category repertoire to
Unicode 3.2, the version the corpus is consistent at, so code points added
in 3.2 (Tagalog, Cyrillic Komi) are word characters while later additions
(the Tamil digit zero) stay unassigned and match \D. New XSDCategory and a
3.2 assignment table derived from DerivedAge, replacing the old Unicode31
files.
Content model: assess each child against the particle the content automaton
matched, not a by-name lookup (#180). Two same-named particles keep their
own fixed values, and an element matching a skip wildcard is no longer
validated against its global declaration. Selection is made reproducible
when an ambiguous schema slips past the UPA check.
XSTS runner: compile and merge every schemaDocument in a schemaTest, not
just the first; this also surfaced one previously-masked invalid-instance
acceptance (recorded in the baselines).
Also folds in attribute-group anyAttribute inheritance, extension wildcard
union, namespace-qualified type tables for multi-document
xsi:schemaLocation, per-container elementFormDefault, strict and lax
attribute wildcard validation, built-in type resolution, and xs:redefine
simpleType overrides.
\w was the Perl word class (alphabetic, decimal, underscore). XSD 1.0 Datatypes
F.1 defines \w as [#x0000-#x10FFFF]-[\p{P}\p{Z}\p{C}], so symbols and marks are
word characters and the connector `_` (Pc) is not; \W is its complement and had
wrongly matched symbols. \w is now computed as "not punctuation, separator, or
other" via the category matcher.
XSTS: invalid instances accepted 231 to 210. The internal regex fixtures that
assumed `_` is a word character encoded Perl semantics and were corrected to the
XSD definition (symbols are word chars, `_` is not), confirmed against the XSTS
reV cases.
Refs #147, #129.
Only about fifteen Unicode blocks were tabulated, so a pattern over any other
block (\p{IsArmenian}, \p{IsTamil}, \p{IsHangulJamo}, and many more) failed to
compile and rejected valid instances. The table now carries the Unicode 3.1
blocks the XSD regular-expression appendix fixes; surrogate blocks are omitted
since surrogates are not scalar values, and Specials uses the main FFF0..FFFD
range. Each range is verified bidirectionally by the XSTS corpus: valid
in-block instances pass and invalid out-of-block instances still fail, so no
range over-covers.
XSTS: valid instances rejected 502 to 395. Baseline updated; regression test
covers in-block matches, out-of-block rejection, and unknown block names.
Refs #147, #129.
Running the full W3C XSTS archive to completion exposed three crash and hang
classes, now fixed with regression tests, which unblocked the #129 first
measurement.
- Content-model and regex NFA builders now cap total state count (2^20),
degrading further occurrence/quantifier repetition to star, so nested
high-maxOccurs schemas no longer OOM (was 8 GB on the MS Particles set).
- Content-model validation advances one active state-set per child instead of
re-walking the prefix each time, removing the O(children^2) spin.
- Regex class subtraction [abc-[b]] no longer traps on a reversed range, and a
reversed range such as [z-a] is rejected cleanly.
XSTS first measurement baselined: 75 valid schemas rejected, 2467 invalid
schemas accepted, 604 valid instances rejected, 582 invalid instances accepted
across 14383 groups. The suite runs green in 8s, opt-in via XSTS_ROOT.
Refs #129, #122.
Close the two XSD-regex gaps from the audit:
- \p{...} and \P{...} now resolve Unicode general categories (one-letter
groups like \p{L} and two-letter codes like \p{Lu}/\p{Nd}) through
Swift's own scalar generalCategory, so no character tables are vendored.
A curated set of well-known \p{Is<Block>} blocks is supported; an
unknown category or block name is a compile error rather than a guess.
- Character-class subtraction [a-z-[aeiou]] removes the subtrahend from a
class, and nests ([a-z-[b-d-[c]]]) and composes with categories.
Categories and subtraction work both as standalone atoms and inside
character classes, so XSD pattern facets that use them now validate.
Closes#89
Use let for never-mutated bindings in DateTimeParser, HTMLEntities, and the
XSLT parser, and drop the redundant Unicode.Scalar Comparable conformance
that the standard library already provides. No behavior change.
Add the PureXML.Regex namespace: a recursive-descent parser for the XSD
regex grammar (literals, ., character classes with ranges and negation, the
\d \w \s family and negations, single-char escapes, grouping, alternation,
and the ?, *, +, {n}, {n,}, {n,m} quantifiers) compiled to a Thompson NFA.
matches(_:) is whole-string as xs:pattern requires, and runs linear in the
input with no backtracking blow-up. \p{...} category escapes are not yet
supported.
Closes#59Closes#60Closes#30
