From 848c7002c7a3dd69aece36ea0372b1e1ebcd5b53 Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Wed, 10 Jun 2026 05:16:05 +0200 Subject: [PATCH] feat(parsing): second W3C-suite strictness pass Character references now require lowercase 'x' for hex and a referenced code point inside the XML Char production (rejecting �, , ￿, surrogates, and X). The reserved 'xml' processing-instruction target (any case) is legal only as the declaration at the document's very first bytes: whitespace or a comment before , a declaration inside content or after the root, and case variants like are all rejected. Comments and PI data are checked for the XML Char production like content. not-wf/sa: 109/186 rejected (from 93); valid/sa unchanged at 118/119; all 846 suite tests green, baseline rebased to the exact 77 remaining (the internal-subset grammar and entity replacement-text classes on #120). Part of #120, #105 --- CHANGELOG.md | 2 +- Sources/Parsing/Diagnostic.swift | 2 +- Sources/Parsing/EntityDecoder.swift | 11 +++++++++-- Sources/Parsing/EventReader.swift | 12 ++++++++++++ Sources/Parsing/EventReaderRecovery.swift | 3 +++ Sources/Parsing/ParseError.swift | 5 +++++ Tests/W3CXMLSuiteTests.swift | 20 +++++++++----------- 7 files changed, 40 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0b991c7..2d72b20 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,7 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added -- Parser strictness, first W3C-suite pass (#105 Tier 2, #120): the strict parser now rejects `--` inside comments (including the `--->` ending), a raw `<` in attribute values, the literal `]]>` sequence in character data, attributes not separated by whitespace, and characters outside the XML `Char` production in content, five new located `ParseError` cases. Driven by a new opt-in runner for the W3C XML Conformance Test Suite's xmltest section (`XMLTS_ROOT`-gated; the suite is never vendored because its license permits redistribution only as the unmodified archive): valid/sa 118/119, not-wf/sa rejection improved from 80 to 93 of 186, with the exact remaining baseline carried in the test and the two remaining leniency classes (internal-subset DTD grammar, entity replacement-text well-formedness) tracked as #120. +- Parser strictness, first W3C-suite pass (#105 Tier 2, #120): the strict parser now rejects `--` inside comments (including the `--->` ending), a raw `<` in attribute values, the literal `]]>` sequence in character data, attributes not separated by whitespace, and characters outside the XML `Char` production in content, five new located `ParseError` cases. Driven by a new opt-in runner for the W3C XML Conformance Test Suite's xmltest section (`XMLTS_ROOT`-gated; the suite is never vendored because its license permits redistribution only as the unmodified archive): valid/sa 118/119, not-wf/sa rejection improved from 80 to 109 of 186 across the first two strictness passes (the second adds character-reference validation, lowercase-x-only hex references and the referenced code point must be a valid XML Char; the reserved 'xml' PI target, legal only as the declaration at the document's very first bytes; and XML Char validation in comments and PI data), with the exact remaining baseline carried in the test and the two remaining leniency classes (internal-subset DTD grammar, entity replacement-text well-formedness) tracked as #120. - XSD Particle Valid (Restriction) (#117): a complex type derived by restriction is now checked at schema compile to accept a structural subset of its base, covering element name matching, element-vs-wildcard admission, wildcard narrowing, occurrence-range subsumption, and the group recursions (sequence/all order-preserving with emptiable skips, choice mapping, sequence-into-choice, sequence-into-all, RecurseAsIfGroup). A structurally unfaithful restriction (widened occurrence, new element name, reordered sequence, content added to an EMPTY or required-into-empty base, an element outside a base wildcard's namespace) throws `SchemaError.invalidRestriction` with the type, base, and reason. The spec's effective-total-range arithmetic for MapAndSum is approximated by per-particle checks, a documented simplification. - Conformance corpus harness (#105, Tier 2): the start of conformance-suite checking, expressed through the validation framework (the OpenAPIKit idiom). A `PureXML.Validation.ConformanceCase` is a `Validatable` named expectation (actual vs spec-authoritative expected output), and `Conformance.matchesExpected` is a `Validation` rule that emits one located `ValidationError` per diverging case, so a whole suite reports every failure at once with the case name as its coding path. Seeded with a Canonical XML 1.0 corpus (attribute ordering, empty-element expansion, comment removal, CDATA-to-escaped-text, processing instructions, namespace rendering) an XPath 1.0 core-function corpus, and an XSD datatype/facet corpus (length, range, pattern, enumeration, lexical space), a RELAX NG pattern corpus (text, optional, repetition, choice, ordered group, attribute, interleave, empty), an XSLT 1.0 transformation corpus (value-of, for-each, if, choose, sort, variable, format-number, apply-templates, call-template), an HTML5 tree-construction corpus (void elements, implied tag closing, case normalization, the adoption agency, entities), a Schematron corpus (assert, report, rule context), a C14N exclusive/2.0 corpus (unused-namespace dropping, render-at-use, whitespace trimming), an XPath axes/predicates corpus (descendant, ancestor, sibling axes, positional and attribute predicates, union), a DTD content-model corpus (sequence, choice, occurrence indicators, mixed, EMPTY, ANY), an XInclude/XPointer corpus (substitution, text inclusion with escaping, fallback, relative href resolution; shorthand ids, element() navigation, xpointer() including the per-step predicate semantic), an XSD regex corpus (implicit whole-string anchoring, the character-class escapes, Unicode categories, quantifier ranges, alternation, class negation and subtraction), and an XML Catalog corpus (system/public/uri entries, rewrite prefixes, unmatched identifiers). The HTML corpus surfaced its first divergence (the fragment parser does not reconstruct active formatting elements like the document parser, tracked as #109). Growing the corpora against the official W3C/OASIS suites is the ongoing Tier-2 work. - EXSLT extension functions (#105, Tier 3): the `common`, `math`, and `sets` EXSLT modules, dispatched by resolving a function's prefix to its namespace (so any prefix bound to an EXSLT namespace works). `math:min`, `math:max`, `math:highest`, `math:lowest`, `math:abs`, `math:sqrt` (with the EXSLT NaN rule for non-numeric nodes); `set:distinct`, `set:difference`, `set:intersection`, `set:has-same-node`, `set:leading`, `set:trailing` (by node identity and document order); `exsl:object-type`. `math:power` and the other transcendentals are omitted because they need a C math library, which the pure-Swift, Foundation-free target excludes; `exsl:node-set` awaits result-tree-fragment support. diff --git a/Sources/Parsing/Diagnostic.swift b/Sources/Parsing/Diagnostic.swift index 903dc6d..924fbb8 100644 --- a/Sources/Parsing/Diagnostic.swift +++ b/Sources/Parsing/Diagnostic.swift @@ -40,7 +40,7 @@ public extension PureXML.Parsing.ParseError { let .junkAfterDocumentElement(mark), let .unsupportedDoctype(mark), let .malformedDeclaration(mark), let .doubleHyphenInComment(mark), let .rawLessThanInAttribute(mark), let .cdataCloseInContent(mark), let .missingSpaceBeforeAttribute(mark), - let .invalidCharacter(mark): + let .invalidCharacter(mark), let .reservedProcessingInstructionTarget(mark): mark case let .unexpectedCharacter(_, mark), let .mismatchedEndTag(_, _, mark), let .duplicateAttribute(_, mark), let .invalidReference(_, mark), diff --git a/Sources/Parsing/EntityDecoder.swift b/Sources/Parsing/EntityDecoder.swift index cae8f35..5ae5a4f 100644 --- a/Sources/Parsing/EntityDecoder.swift +++ b/Sources/Parsing/EntityDecoder.swift @@ -101,12 +101,19 @@ private struct EntityExpander { private static func characterReference(_ body: String, at mark: PureXML.Parsing.Mark) throws -> Character { let digits = String(body.dropFirst()) - let scalarValue: UInt32? = if digits.hasPrefix("x") || digits.hasPrefix("X") { + // XML 1.0: only lowercase 'x' introduces a hex reference, and the + // referenced code point must be a valid XML Char (rejecting NUL, bare + // control characters, surrogates, and U+FFFE/U+FFFF). + let scalarValue: UInt32? = if digits.hasPrefix("x") { UInt32(digits.dropFirst(), radix: 16) + } else if digits.hasPrefix("X") { + nil } else { UInt32(digits, radix: 10) } - guard let value = scalarValue, let scalar = Unicode.Scalar(value) else { + guard let value = scalarValue, let scalar = Unicode.Scalar(value), + PureXML.Parsing.XMLCharacter.isChar(scalar) + else { throw PureXML.Parsing.ParseError.invalidReference("&\(body);", mark) } return Character(scalar) diff --git a/Sources/Parsing/EventReader.swift b/Sources/Parsing/EventReader.swift index 12cf3db..f824e6a 100644 --- a/Sources/Parsing/EventReader.swift +++ b/Sources/Parsing/EventReader.swift @@ -105,6 +105,11 @@ public extension PureXML.Parsing { let mark = reader.mark let instruction = try scanProcessingInstruction() if instruction.target.lowercased() == "xml" { + // The reserved target is only legal as the XML declaration, + // which must be the very first bytes of the document. + guard mark == .start, instruction.target == "xml" else { + throw ParseError.reservedProcessingInstructionTarget(mark) + } try recordDeclaration(instruction.data, at: mark) continue } @@ -138,7 +143,11 @@ public extension PureXML.Parsing { if reader.matches("") diff --git a/Sources/Parsing/ParseError.swift b/Sources/Parsing/ParseError.swift index ea2948b..775fd96 100644 --- a/Sources/Parsing/ParseError.swift +++ b/Sources/Parsing/ParseError.swift @@ -45,6 +45,9 @@ public extension PureXML.Parsing { case missingSpaceBeforeAttribute(Mark) /// A character outside the XML `Char` production appears in content. case invalidCharacter(Mark) + /// A processing instruction uses the reserved target `xml` (any case), + /// which is only legal as the document's XML declaration. + case reservedProcessingInstructionTarget(Mark) public var description: String { switch self { @@ -106,6 +109,8 @@ public extension PureXML.Parsing { "attributes must be separated by whitespace at \(mark)" case let .invalidCharacter(mark): "character is not allowed in XML content at \(mark)" + case let .reservedProcessingInstructionTarget(mark): + "the processing-instruction target 'xml' is reserved at \(mark)" case let .malformedDeclaration(mark): "malformed XML declaration at \(mark)" } diff --git a/Tests/W3CXMLSuiteTests.swift b/Tests/W3CXMLSuiteTests.swift index 32cca59..9a4fc4c 100644 --- a/Tests/W3CXMLSuiteTests.swift +++ b/Tests/W3CXMLSuiteTests.swift @@ -24,17 +24,15 @@ struct W3CXMLSuiteTests { /// markup in values). The baseline below is exact, so any regression in /// the cases that DO pass is still caught. private let knownNotWFDeviations: Set = [ - "031.xml", "032.xml", "054.xml", "057.xml", "058.xml", "059.xml", "060.xml", "061.xml", - "062.xml", "063.xml", "064.xml", "065.xml", "066.xml", "067.xml", "068.xml", "069.xml", - "074.xml", "078.xml", "079.xml", "080.xml", "082.xml", "084.xml", "085.xml", "086.xml", - "087.xml", "089.xml", "090.xml", "091.xml", "092.xml", "093.xml", "096.xml", "101.xml", - "102.xml", "103.xml", "107.xml", "113.xml", "114.xml", "115.xml", "116.xml", "117.xml", - "119.xml", "120.xml", "121.xml", "122.xml", "123.xml", "124.xml", "125.xml", "126.xml", - "127.xml", "128.xml", "129.xml", "130.xml", "131.xml", "132.xml", "133.xml", "134.xml", - "135.xml", "136.xml", "137.xml", "138.xml", "139.xml", "140.xml", "141.xml", "142.xml", - "143.xml", "144.xml", "147.xml", "148.xml", "149.xml", "150.xml", "151.xml", "153.xml", - "154.xml", "155.xml", "156.xml", "157.xml", "158.xml", "160.xml", "161.xml", "162.xml", - "165.xml", "168.xml", "169.xml", "170.xml", "171.xml", "172.xml", "173.xml", "174.xml", + "054.xml", "057.xml", "058.xml", "059.xml", "060.xml", "061.xml", "062.xml", "063.xml", + "064.xml", "065.xml", "066.xml", "067.xml", "068.xml", "069.xml", "074.xml", "078.xml", + "079.xml", "080.xml", "082.xml", "084.xml", "085.xml", "086.xml", "087.xml", "089.xml", + "090.xml", "091.xml", "092.xml", "096.xml", "101.xml", "102.xml", "103.xml", "107.xml", + "113.xml", "114.xml", "115.xml", "116.xml", "117.xml", "119.xml", "120.xml", "121.xml", + "122.xml", "123.xml", "124.xml", "125.xml", "126.xml", "127.xml", "128.xml", "129.xml", + "130.xml", "131.xml", "132.xml", "133.xml", "134.xml", "135.xml", "136.xml", "137.xml", + "138.xml", "139.xml", "140.xml", "141.xml", "149.xml", "153.xml", "158.xml", "160.xml", + "161.xml", "162.xml", "165.xml", "168.xml", "169.xml", "170.xml", "173.xml", "174.xml", "175.xml", "180.xml", "182.xml", "183.xml", "184.xml", ]