Mihaela Mihaljevic
1919b92956
XSD 1.0 fixes the regular-expression repertoire at Unicode 3.2 (Datatypes
Appendix F), where U+1369..U+1371 (ETHIOPIC DIGIT ONE..NINE) are general
category Nd, each carrying a decimal value; a later Unicode reclassified them
to No. XSDCategory derives \d (== \p{Nd}) from the running platform's category,
pinned to 3.2 by an assigned-in-3.2 gate plus a per-code-point override table,
but the table omitted these nine, so \d used the modern No: a valid \d-constrained
Ethiopic digit was wrongly rejected (cardinal-rule false positive) and a
\D-constrained one wrongly accepted.
Pin the nine to Nd. This is the complete set of code points that were Nd at 3.2
and are not today; the reverse direction is empty (every later-Nd point was
unassigned at 3.2 and already reports Cn), verified by diffing the 3.2 and current
Unicode Nd sets. \w is unaffected: an Ethiopic numeral is category N either way,
neither punctuation, separator, nor other.
Verified over the full W3C XSTS corpus in release: invalid-instances-accepted
14 -> 12 (reT17, reT38, the \D cases, now reject), and the matching \d-valid cases
reS17/reS38/reZ004 no longer need a suspect-test exclusion and pass as ordinary
agreements, with both false-positive buckets at 0. Full suite (1830 tests) green;
an adversarial over-rejection critic re-derived the Nd delta bidirectionally and
confirmed no false-positive surface. A regression-pinning test exercises the
production regex path, including the boundary that U+1372 (No even at 3.2) is
not \d.
56 lines
3.2 KiB
Swift
56 lines
3.2 KiB
Swift
extension PureXML.Regex {
|
|
/// Unicode general categories as XSD 1.0 regular expressions see them
|
|
/// (Datatypes Appendix F). Category and block escapes (`\p{...}`) and the
|
|
/// multi-character escapes `\d`, `\w`, and friends are defined over the
|
|
/// Unicode database. The W3C conformance corpus (XSTS) is internally
|
|
/// consistent only at Unicode 3.2, so the repertoire is pinned there (see
|
|
/// ``assignedInUnicode32``): a code point unassigned in 3.2 reports `Cn` and
|
|
/// so matches `\D`/`\W`, even where the running platform now assigns it.
|
|
///
|
|
/// For code points whose 3.2-era category differs from the running platform's
|
|
/// (for example the ceiling and floor brackets, `Sm` in 3.2 but `Ps`/`Pe`
|
|
/// today), ``categoryOverrides`` pins the 3.2 classification; every other
|
|
/// assigned code point uses the platform category.
|
|
enum XSDCategory {
|
|
/// Code points whose XSD-era general category differs from the running
|
|
/// platform's. The value is the XSD-era two-letter category.
|
|
private static let categoryOverrides: [UInt32: String] = [
|
|
0x00A7: "So", 0x00B6: "So",
|
|
0x06DD: "Me",
|
|
0x0F14: "So",
|
|
0x166D: "Po", 0x17D7: "Po", 0x17DC: "Po",
|
|
0x180B: "Cf", 0x180C: "Cf", 0x180D: "Cf",
|
|
// Ethiopic digits ONE..NINE were Nd at Unicode 3.2 (each carried a
|
|
// decimal value); a later Unicode reclassified them to No. XSD `\d`
|
|
// is `\p{Nd}` over the pinned 3.2 repertoire, so they must match `\d`
|
|
// (and not `\D`). This is the complete set of code points that were
|
|
// Nd at 3.2 and are not Nd today (XSTS reS17/reS38/reT17/reT38/reZ004).
|
|
0x1369: "Nd", 0x136A: "Nd", 0x136B: "Nd", 0x136C: "Nd", 0x136D: "Nd",
|
|
0x136E: "Nd", 0x136F: "Nd", 0x1370: "Nd", 0x1371: "Nd",
|
|
0x2308: "Sm", 0x2309: "Sm", 0x230A: "Sm", 0x230B: "Sm",
|
|
]
|
|
|
|
/// The two-letter XSD category abbreviation for each Unicode general
|
|
/// category. A flat translation table, not branching logic.
|
|
private static let categoryCodes: [Unicode.GeneralCategory: String] = [
|
|
.uppercaseLetter: "Lu", .lowercaseLetter: "Ll", .titlecaseLetter: "Lt",
|
|
.modifierLetter: "Lm", .otherLetter: "Lo",
|
|
.nonspacingMark: "Mn", .spacingMark: "Mc", .enclosingMark: "Me",
|
|
.decimalNumber: "Nd", .letterNumber: "Nl", .otherNumber: "No",
|
|
.connectorPunctuation: "Pc", .dashPunctuation: "Pd", .openPunctuation: "Ps",
|
|
.closePunctuation: "Pe", .initialPunctuation: "Pi", .finalPunctuation: "Pf",
|
|
.otherPunctuation: "Po",
|
|
.mathSymbol: "Sm", .currencySymbol: "Sc", .modifierSymbol: "Sk", .otherSymbol: "So",
|
|
.spaceSeparator: "Zs", .lineSeparator: "Zl", .paragraphSeparator: "Zp",
|
|
.control: "Cc", .format: "Cf", .surrogate: "Cs", .privateUse: "Co",
|
|
.unassigned: "Cn",
|
|
]
|
|
|
|
static func generalCategoryCode(_ scalar: Unicode.Scalar) -> String {
|
|
if let override = categoryOverrides[scalar.value] { return override }
|
|
guard assignedInUnicode32(scalar.value) else { return "Cn" }
|
|
return categoryCodes[scalar.properties.generalCategory] ?? "Cn"
|
|
}
|
|
}
|
|
}
|