From 62e53b0fc137cb6dd3568926b00acaf69cb66798 Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Sat, 13 Jun 2026 23:52:11 +0200 Subject: [PATCH] feat(schema): validate pattern facet regular expressions at compile time Each pattern facet value is now compiled when the schema is, instead of only lazily at instance time (try?) where an unparseable pattern was silently ignored. A structurally malformed pattern (unbalanced parentheses, a quantifier with nothing to repeat) is rejected. The check is conservative against over-rejection: it rejects only the unambiguous structural RegexError kinds (.unbalanced, .danglingQuantifier) and tolerates the kinds that fire on otherwise-valid patterns the engine cannot fully handle (.empty for the empty pattern, .unsupported for an untabulated \p{Is...} block such as a surrogate or historic block, .badQuantifier for the lenient {,m} form, and .badEscape/.badClass which could fire on valid character- class subtraction). So a{2,1} and a trailing backslash remain accepted (an under-rejection), but no valid schema is rejected. XSTS: invalid schemas accepted 1129 to 1092 (-37); valid schemas rejected held at 72; instance buckets unchanged. --- CHANGELOG.md | 1 + Sources/Schema/XSDSimpleParser.swift | 38 ++++++++++++++++++- Tests/SchemaPatternFacetTests.swift | 55 ++++++++++++++++++++++++++++ Tests/XSTSSuiteTests.swift | 2 +- docs/schema-validity-burndown.md | 4 +- docs/xsts-deviations.md | 4 +- 6 files changed, 98 insertions(+), 6 deletions(-) create mode 100644 Tests/SchemaPatternFacetTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a8fc4e..f6ca59f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- `pattern` facet regular expressions are now validated at compile time (#147). Each `pattern` value is compiled when the schema is; a structurally malformed one (unbalanced parentheses `((a)`, a quantifier with nothing to repeat `?a`) is rejected, where before it was only compiled lazily at instance time (`try?`) and so silently ignored. The check is conservative: a construct the engine merely does not support on an otherwise-valid pattern (the empty pattern, an untabulated `\p{Is...}` block such as a surrogate or historic block, the lenient `{,m}` quantifier) is tolerated, so no valid schema is rejected. XSTS: invalid schemas accepted 1129 to 1092, no other bucket raised. - Schema-vocabulary name and reference attribute values are now checked for lexical validity at compile time (#147). A component's `name` must be a valid NCName (`name=""`, `name="123"`, `name="a:b"` are rejected), and the reference attributes `type`/`base`/`ref`/`itemType`/`refer`/`substitutionGroup` must be lexical QNames (`type=":_"`, `type="a:b:c"` rejected), with `memberTypes` a list of QNames; these were unchecked and the schema accepted. Only the lexical form is checked (a QName's prefix is resolved elsewhere), so no valid schema is rejected. XSTS: invalid schemas accepted 1190 to 1129, no other bucket raised. - Schema-vocabulary attribute values are now validated at compile time (#147), the attribute-model sibling of the structural content-model check. Enumerated attributes (`form`/`elementFormDefault`/`attributeFormDefault` to `qualified`/`unqualified`, `use` to `optional`/`prohibited`/`required`, `processContents` to `skip`/`lax`/`strict`, `mixed`/`abstract`/`nillable` to a boolean) and `minOccurs`/`maxOccurs` (a `nonNegativeInteger`, `maxOccurs` also `unbounded`) are checked against their fixed value space; an out-of-space value (`use="foo"`, `mixed="yes"`, `minOccurs="-1"`) was previously ignored and the schema accepted. Only unprefixed attributes on XSD-namespace elements are checked, so a foreign attribute is never flagged. XSTS: invalid schemas accepted 1313 to 1190, no other bucket raised. - Schema documents are now structurally validated against the schema-for-schemas content model (#147), the general mechanism for the structural tail of the schema-validity gap. A data-driven table of the XSD 1.0 child content model rejects, at compile time, any component child the model does not admit, more than one `annotation` (or an `annotation` that is not first, except on `schema`/`redefine`), and an identity constraint missing its selector or field. The check walks only XSD-namespace elements and skips `xs:appinfo`/`xs:documentation` foreign content, so no valid schema is rejected; `restriction`/`extension` use the permissive union of their contexts (never over-rejecting). One table covers every component rather than a rule hand-coded per element. XSTS: invalid schemas accepted 1688 to 1313 (-375), no other bucket raised. diff --git a/Sources/Schema/XSDSimpleParser.swift b/Sources/Schema/XSDSimpleParser.swift index 98abc35..4e1af9c 100644 --- a/Sources/Schema/XSDSimpleParser.swift +++ b/Sources/Schema/XSDSimpleParser.swift @@ -27,12 +27,48 @@ extension PureXML.Schema { facets.enumeration = nil } applyFacets(restriction, into: &facets) - for error in facetDefinitionErrors(restriction, base: baseType) { + for error in facetDefinitionErrors(restriction, base: baseType) + patternErrors(restriction) { context.diagnostics.report(error) } return SimpleType(base: baseType.base, facets: facets, variety: baseType.variety) } + /// Schema-validity findings for the `pattern` facets declared on one + /// `restriction`: each `pattern` value must be a valid XSD regular + /// expression. The pattern was only compiled lazily at instance time + /// (`try?`), so an unparseable one was silently ignored and the schema + /// accepted. Only the unambiguous structural errors reject (a quantifier + /// with nothing to repeat, `?a`; unbalanced parentheses, `((a)`); see + /// `rejectsPattern`. + static func patternErrors(_ restriction: XSDTree) -> [String] { + var errors: [String] = [] + for facet in XSDNode.elementChildren(restriction) where XSDNode.localName(facet) == "pattern" { + let value = XSDNode.attribute(facet, "value") ?? "" + do { + _ = try PureXML.Regex.Pattern(value) + } catch let error as PureXML.Regex.RegexError where rejectsPattern(error) { + errors.append("pattern '\(value)' is not a valid regular expression: \(error)") + } catch { + // A construct the engine does not support (`.empty`, `.unsupported` + // for an untabulated `\p{Is...}` block, `.badQuantifier` for the + // lenient `{,m}` form) is a valid XSD pattern, not a schema error. + } + } + return errors + } + + /// Whether a regex-compile failure is a genuine syntax error (so the schema + /// is invalid), as opposed to an engine limitation on an otherwise-valid XSD + /// pattern. Conservative: only the unambiguous structural errors reject. + private static func rejectsPattern(_ error: PureXML.Regex.RegexError) -> Bool { + switch error { + case .unbalanced, .danglingQuantifier: + true + case .empty, .unsupported, .badQuantifier, .badEscape, .badClass: + false + } + } + /// Schema-validity findings for the constraining facets declared on one /// `restriction` (XSD Part 2 Datatypes 4.3): the length-family rules plus /// the value-bound rules, both limited to the facets declared on this step diff --git a/Tests/SchemaPatternFacetTests.swift b/Tests/SchemaPatternFacetTests.swift new file mode 100644 index 0000000..17fd209 --- /dev/null +++ b/Tests/SchemaPatternFacetTests.swift @@ -0,0 +1,55 @@ +@testable import PureXML +import Testing + +/// A `pattern` facet value must be a valid XSD regular expression. The pattern +/// was only compiled lazily at instance time (`try?`), so an unparseable one was +/// silently ignored and the schema accepted. Compile-time validation rejects the +/// unambiguous structural errors (unbalanced parentheses, a quantifier with +/// nothing to repeat) while tolerating constructs the engine merely does not +/// support on an otherwise-valid pattern (an untabulated `\p{Is...}` block, the +/// empty pattern, the lenient `{,m}` quantifier), so no valid schema is rejected. +@Suite("Pattern facet regex validity") +struct SchemaPatternFacetTests { + private func compile(_ pattern: String) throws { + _ = try PureXML.Schema.Document(""" + + + + + + + + """) + } + + private func rejects(_ pattern: String) -> Bool { + do { try compile(pattern) + return false + } catch { return true } + } + + @Test("a structurally malformed pattern is rejected") + func test_malformed() { + #expect(rejects("((a)")) + #expect(rejects("(a))")) + #expect(rejects("?a")) + #expect(rejects("*a")) + #expect(rejects("+a")) + } + + @Test("a valid pattern compiles") + func test_valid() throws { + try compile("[a-z]+") + try compile("[0-9]{2,4}") + try compile("(ab)*c?") + } + + @Test("a construct the engine does not support is not treated as a schema error") + func test_engineGapTolerated() throws { + // Empty pattern (valid: matches the empty string), an untabulated Unicode + // block escape, and the lenient `{,m}` quantifier are all accepted. + try compile("") + try compile(#"\p{IsHighSurrogates}"#) + try compile("[0-9]{,5}") + } +} diff --git a/Tests/XSTSSuiteTests.swift b/Tests/XSTSSuiteTests.swift index 317a9c9..1688a93 100644 --- a/Tests/XSTSSuiteTests.swift +++ b/Tests/XSTSSuiteTests.swift @@ -29,7 +29,7 @@ struct XSTSSuiteTests { /// XSTS` (debug is far slower and the corpus is large). Per-case deviations /// are written to /tmp/xsts-failures.txt for the burn-down. private let knownSchemaValidRejected = 72 - private let knownSchemaInvalidAccepted = 1129 + private let knownSchemaInvalidAccepted = 1092 private let knownInstanceValidRejected = 214 private let knownInstanceInvalidAccepted = 165 diff --git a/docs/schema-validity-burndown.md b/docs/schema-validity-burndown.md index 1e788a8..7ca31d3 100644 --- a/docs/schema-validity-burndown.md +++ b/docs/schema-validity-burndown.md @@ -10,7 +10,7 @@ irreducible tail. Do not deviate to new features while this is open. | Bucket | Count | Reading | |---|---|---| | valid-schemas-rejected | 72 | we rarely reject a good schema | -| **invalid-schemas-accepted** | **2461 -> 1129** | we rarely catch a bad one (facet, id, and structural content-model validity landed) | +| **invalid-schemas-accepted** | **2461 -> 1092** | we rarely catch a bad one (facet, id, and structural content-model validity landed) | | valid-instances-rejected | 233 | (instance side, separately tracked) | | invalid-instances-accepted | 165 | | @@ -73,7 +73,7 @@ Each iteration targets one self-contained rule family with a clean root cause. (1688 -> 1313, -375)._ A data-driven `allowedChildren` table (the schema-for- schemas child content model) rejects disallowed children, multi/misplaced `annotation`, and identity constraints missing selector/field, via one general - mechanism (`SchemaStructure.swift`). Attribute **value** validity also landed (enumerated attributes, `minOccurs`/`maxOccurs`; 1313 -> 1190), plus **name/reference lexical validity** (`name` an NCName; `type`/`base`/`ref`/`itemType`/`refer`/`substitutionGroup` QNames; 1190 -> 1129). **Still open:** attribute **applicability** (which attributes each component admits, e.g. `form` is prohibited on a global declaration, `ref` excludes `name`); child *order* beyond the leading annotation; and particle / model-group semantic rules (Unique Particle Attribution, Particle-Valid-Restriction completeness). + mechanism (`SchemaStructure.swift`). Attribute **value** validity also landed (enumerated attributes, `minOccurs`/`maxOccurs`; 1313 -> 1190), plus **name/reference lexical validity** (`name` an NCName; `type`/`base`/`ref`/`itemType`/`refer`/`substitutionGroup` QNames; 1190 -> 1129), plus **`pattern` regex validity** (compile each pattern; reject only unambiguous syntax errors, tolerating engine gaps; 1129 -> 1092). **Still open:** attribute **applicability** (which attributes each component admits, e.g. `form` is prohibited on a global declaration, `ref` excludes `name`); child *order* beyond the leading annotation; and particle / model-group semantic rules (Unique Particle Attribution, Particle-Valid-Restriction completeness). ## Per-iteration protocol (the PR-critic-loop) diff --git a/docs/xsts-deviations.md b/docs/xsts-deviations.md index 3965400..e6f92cd 100644 --- a/docs/xsts-deviations.md +++ b/docs/xsts-deviations.md @@ -47,7 +47,7 @@ and to document the remainder as spec-justified, deliberate exclusions. ## Category A: genuine correctness bugs -### Fixed (twenty root causes, ~2006 deviations cleared) +### Fixed (twenty-one root causes, ~2043 deviations cleared) list-datatype length facets (#146) · union pattern/enumeration facets · duration partial-order facets · QName length non-constraining · the full XSD `\p{Is...}` @@ -65,7 +65,7 @@ validity (`min`/`maxInclusive`, `min`/`maxExclusive`, `enumeration` values valid in the base space; inclusive/exclusive exclusions; bound ordering) · `gMonth` `--MM--` lexical form · schema `id` attribute validity (`xs:ID`: NCName and unique within the document). The schema-validity work brought invalid schemas -accepted 2461 to 1129 (also: schema-document structural validity, content model + attribute values + name/reference lexical validity, against the schema-for-schemas); see `schema-validity-burndown.md`. +accepted 2461 to 1092 (also: schema-document structural validity, content model + attribute values + name/reference + pattern-regex lexical validity, against the schema-for-schemas); see `schema-validity-burndown.md`. Plus the measurement fix (exclude W3C-disputed `status="queried"` entries). ### Remaining, root-caused (the hard tail)