feat(schema): validate pattern facet regular expressions at compile time
Each pattern facet value is now compiled when the schema is, instead of only
lazily at instance time (try?) where an unparseable pattern was silently
ignored. A structurally malformed pattern (unbalanced parentheses, a quantifier
with nothing to repeat) is rejected.
The check is conservative against over-rejection: it rejects only the
unambiguous structural RegexError kinds (.unbalanced, .danglingQuantifier) and
tolerates the kinds that fire on otherwise-valid patterns the engine cannot
fully handle (.empty for the empty pattern, .unsupported for an untabulated
\p{Is...} block such as a surrogate or historic block, .badQuantifier for the
lenient {,m} form, and .badEscape/.badClass which could fire on valid character-
class subtraction). So a{2,1} and a trailing backslash remain accepted (an
under-rejection), but no valid schema is rejected.
XSTS: invalid schemas accepted 1129 to 1092 (-37); valid schemas rejected held
at 72; instance buckets unchanged.
This commit is contained in:
parent
ca8fd27877
commit
62e53b0fc1
@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- `pattern` facet regular expressions are now validated at compile time (#147). Each `pattern` value is compiled when the schema is; a structurally malformed one (unbalanced parentheses `((a)`, a quantifier with nothing to repeat `?a`) is rejected, where before it was only compiled lazily at instance time (`try?`) and so silently ignored. The check is conservative: a construct the engine merely does not support on an otherwise-valid pattern (the empty pattern, an untabulated `\p{Is...}` block such as a surrogate or historic block, the lenient `{,m}` quantifier) is tolerated, so no valid schema is rejected. XSTS: invalid schemas accepted 1129 to 1092, no other bucket raised.
|
||||
- Schema-vocabulary name and reference attribute values are now checked for lexical validity at compile time (#147). A component's `name` must be a valid NCName (`name=""`, `name="123"`, `name="a:b"` are rejected), and the reference attributes `type`/`base`/`ref`/`itemType`/`refer`/`substitutionGroup` must be lexical QNames (`type=":_"`, `type="a:b:c"` rejected), with `memberTypes` a list of QNames; these were unchecked and the schema accepted. Only the lexical form is checked (a QName's prefix is resolved elsewhere), so no valid schema is rejected. XSTS: invalid schemas accepted 1190 to 1129, no other bucket raised.
|
||||
- Schema-vocabulary attribute values are now validated at compile time (#147), the attribute-model sibling of the structural content-model check. Enumerated attributes (`form`/`elementFormDefault`/`attributeFormDefault` to `qualified`/`unqualified`, `use` to `optional`/`prohibited`/`required`, `processContents` to `skip`/`lax`/`strict`, `mixed`/`abstract`/`nillable` to a boolean) and `minOccurs`/`maxOccurs` (a `nonNegativeInteger`, `maxOccurs` also `unbounded`) are checked against their fixed value space; an out-of-space value (`use="foo"`, `mixed="yes"`, `minOccurs="-1"`) was previously ignored and the schema accepted. Only unprefixed attributes on XSD-namespace elements are checked, so a foreign attribute is never flagged. XSTS: invalid schemas accepted 1313 to 1190, no other bucket raised.
|
||||
- Schema documents are now structurally validated against the schema-for-schemas content model (#147), the general mechanism for the structural tail of the schema-validity gap. A data-driven table of the XSD 1.0 child content model rejects, at compile time, any component child the model does not admit, more than one `annotation` (or an `annotation` that is not first, except on `schema`/`redefine`), and an identity constraint missing its selector or field. The check walks only XSD-namespace elements and skips `xs:appinfo`/`xs:documentation` foreign content, so no valid schema is rejected; `restriction`/`extension` use the permissive union of their contexts (never over-rejecting). One table covers every component rather than a rule hand-coded per element. XSTS: invalid schemas accepted 1688 to 1313 (-375), no other bucket raised.
|
||||
|
||||
@ -27,12 +27,48 @@ extension PureXML.Schema {
|
||||
facets.enumeration = nil
|
||||
}
|
||||
applyFacets(restriction, into: &facets)
|
||||
for error in facetDefinitionErrors(restriction, base: baseType) {
|
||||
for error in facetDefinitionErrors(restriction, base: baseType) + patternErrors(restriction) {
|
||||
context.diagnostics.report(error)
|
||||
}
|
||||
return SimpleType(base: baseType.base, facets: facets, variety: baseType.variety)
|
||||
}
|
||||
|
||||
/// Schema-validity findings for the `pattern` facets declared on one
|
||||
/// `restriction`: each `pattern` value must be a valid XSD regular
|
||||
/// expression. The pattern was only compiled lazily at instance time
|
||||
/// (`try?`), so an unparseable one was silently ignored and the schema
|
||||
/// accepted. Only the unambiguous structural errors reject (a quantifier
|
||||
/// with nothing to repeat, `?a`; unbalanced parentheses, `((a)`); see
|
||||
/// `rejectsPattern`.
|
||||
static func patternErrors(_ restriction: XSDTree) -> [String] {
|
||||
var errors: [String] = []
|
||||
for facet in XSDNode.elementChildren(restriction) where XSDNode.localName(facet) == "pattern" {
|
||||
let value = XSDNode.attribute(facet, "value") ?? ""
|
||||
do {
|
||||
_ = try PureXML.Regex.Pattern(value)
|
||||
} catch let error as PureXML.Regex.RegexError where rejectsPattern(error) {
|
||||
errors.append("pattern '\(value)' is not a valid regular expression: \(error)")
|
||||
} catch {
|
||||
// A construct the engine does not support (`.empty`, `.unsupported`
|
||||
// for an untabulated `\p{Is...}` block, `.badQuantifier` for the
|
||||
// lenient `{,m}` form) is a valid XSD pattern, not a schema error.
|
||||
}
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
/// Whether a regex-compile failure is a genuine syntax error (so the schema
|
||||
/// is invalid), as opposed to an engine limitation on an otherwise-valid XSD
|
||||
/// pattern. Conservative: only the unambiguous structural errors reject.
|
||||
private static func rejectsPattern(_ error: PureXML.Regex.RegexError) -> Bool {
|
||||
switch error {
|
||||
case .unbalanced, .danglingQuantifier:
|
||||
true
|
||||
case .empty, .unsupported, .badQuantifier, .badEscape, .badClass:
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/// Schema-validity findings for the constraining facets declared on one
|
||||
/// `restriction` (XSD Part 2 Datatypes 4.3): the length-family rules plus
|
||||
/// the value-bound rules, both limited to the facets declared on this step
|
||||
|
||||
55
Tests/SchemaPatternFacetTests.swift
Normal file
55
Tests/SchemaPatternFacetTests.swift
Normal file
@ -0,0 +1,55 @@
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
/// A `pattern` facet value must be a valid XSD regular expression. The pattern
|
||||
/// was only compiled lazily at instance time (`try?`), so an unparseable one was
|
||||
/// silently ignored and the schema accepted. Compile-time validation rejects the
|
||||
/// unambiguous structural errors (unbalanced parentheses, a quantifier with
|
||||
/// nothing to repeat) while tolerating constructs the engine merely does not
|
||||
/// support on an otherwise-valid pattern (an untabulated `\p{Is...}` block, the
|
||||
/// empty pattern, the lenient `{,m}` quantifier), so no valid schema is rejected.
|
||||
@Suite("Pattern facet regex validity")
|
||||
struct SchemaPatternFacetTests {
|
||||
private func compile(_ pattern: String) throws {
|
||||
_ = try PureXML.Schema.Document("""
|
||||
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
|
||||
<xs:element name="v">
|
||||
<xs:simpleType>
|
||||
<xs:restriction base="xs:string"><xs:pattern value="\(pattern)"/></xs:restriction>
|
||||
</xs:simpleType>
|
||||
</xs:element>
|
||||
</xs:schema>
|
||||
""")
|
||||
}
|
||||
|
||||
private func rejects(_ pattern: String) -> Bool {
|
||||
do { try compile(pattern)
|
||||
return false
|
||||
} catch { return true }
|
||||
}
|
||||
|
||||
@Test("a structurally malformed pattern is rejected")
|
||||
func test_malformed() {
|
||||
#expect(rejects("((a)"))
|
||||
#expect(rejects("(a))"))
|
||||
#expect(rejects("?a"))
|
||||
#expect(rejects("*a"))
|
||||
#expect(rejects("+a"))
|
||||
}
|
||||
|
||||
@Test("a valid pattern compiles")
|
||||
func test_valid() throws {
|
||||
try compile("[a-z]+")
|
||||
try compile("[0-9]{2,4}")
|
||||
try compile("(ab)*c?")
|
||||
}
|
||||
|
||||
@Test("a construct the engine does not support is not treated as a schema error")
|
||||
func test_engineGapTolerated() throws {
|
||||
// Empty pattern (valid: matches the empty string), an untabulated Unicode
|
||||
// block escape, and the lenient `{,m}` quantifier are all accepted.
|
||||
try compile("")
|
||||
try compile(#"\p{IsHighSurrogates}"#)
|
||||
try compile("[0-9]{,5}")
|
||||
}
|
||||
}
|
||||
@ -29,7 +29,7 @@ struct XSTSSuiteTests {
|
||||
/// XSTS` (debug is far slower and the corpus is large). Per-case deviations
|
||||
/// are written to /tmp/xsts-failures.txt for the burn-down.
|
||||
private let knownSchemaValidRejected = 72
|
||||
private let knownSchemaInvalidAccepted = 1129
|
||||
private let knownSchemaInvalidAccepted = 1092
|
||||
private let knownInstanceValidRejected = 214
|
||||
private let knownInstanceInvalidAccepted = 165
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open.
|
||||
| Bucket | Count | Reading |
|
||||
|---|---|---|
|
||||
| valid-schemas-rejected | 72 | we rarely reject a good schema |
|
||||
| **invalid-schemas-accepted** | **2461 -> 1129** | we rarely catch a bad one (facet, id, and structural content-model validity landed) |
|
||||
| **invalid-schemas-accepted** | **2461 -> 1092** | we rarely catch a bad one (facet, id, and structural content-model validity landed) |
|
||||
| valid-instances-rejected | 233 | (instance side, separately tracked) |
|
||||
| invalid-instances-accepted | 165 | |
|
||||
|
||||
@ -73,7 +73,7 @@ Each iteration targets one self-contained rule family with a clean root cause.
|
||||
(1688 -> 1313, -375)._ A data-driven `allowedChildren` table (the schema-for-
|
||||
schemas child content model) rejects disallowed children, multi/misplaced
|
||||
`annotation`, and identity constraints missing selector/field, via one general
|
||||
mechanism (`SchemaStructure.swift`). Attribute **value** validity also landed (enumerated attributes, `minOccurs`/`maxOccurs`; 1313 -> 1190), plus **name/reference lexical validity** (`name` an NCName; `type`/`base`/`ref`/`itemType`/`refer`/`substitutionGroup` QNames; 1190 -> 1129). **Still open:** attribute **applicability** (which attributes each component admits, e.g. `form` is prohibited on a global declaration, `ref` excludes `name`); child *order* beyond the leading annotation; and particle / model-group semantic rules (Unique Particle Attribution, Particle-Valid-Restriction completeness).
|
||||
mechanism (`SchemaStructure.swift`). Attribute **value** validity also landed (enumerated attributes, `minOccurs`/`maxOccurs`; 1313 -> 1190), plus **name/reference lexical validity** (`name` an NCName; `type`/`base`/`ref`/`itemType`/`refer`/`substitutionGroup` QNames; 1190 -> 1129), plus **`pattern` regex validity** (compile each pattern; reject only unambiguous syntax errors, tolerating engine gaps; 1129 -> 1092). **Still open:** attribute **applicability** (which attributes each component admits, e.g. `form` is prohibited on a global declaration, `ref` excludes `name`); child *order* beyond the leading annotation; and particle / model-group semantic rules (Unique Particle Attribution, Particle-Valid-Restriction completeness).
|
||||
|
||||
## Per-iteration protocol (the PR-critic-loop)
|
||||
|
||||
|
||||
@ -47,7 +47,7 @@ and to document the remainder as spec-justified, deliberate exclusions.
|
||||
|
||||
## Category A: genuine correctness bugs
|
||||
|
||||
### Fixed (twenty root causes, ~2006 deviations cleared)
|
||||
### Fixed (twenty-one root causes, ~2043 deviations cleared)
|
||||
|
||||
list-datatype length facets (#146) · union pattern/enumeration facets · duration
|
||||
partial-order facets · QName length non-constraining · the full XSD `\p{Is...}`
|
||||
@ -65,7 +65,7 @@ validity (`min`/`maxInclusive`, `min`/`maxExclusive`, `enumeration` values valid
|
||||
in the base space; inclusive/exclusive exclusions; bound ordering) · `gMonth`
|
||||
`--MM--` lexical form · schema `id` attribute validity (`xs:ID`: NCName and
|
||||
unique within the document). The schema-validity work brought invalid schemas
|
||||
accepted 2461 to 1129 (also: schema-document structural validity, content model + attribute values + name/reference lexical validity, against the schema-for-schemas); see `schema-validity-burndown.md`.
|
||||
accepted 2461 to 1092 (also: schema-document structural validity, content model + attribute values + name/reference + pattern-regex lexical validity, against the schema-for-schemas); see `schema-validity-burndown.md`.
|
||||
Plus the measurement fix (exclude W3C-disputed `status="queried"` entries).
|
||||
|
||||
### Remaining, root-caused (the hard tail)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user