diff --git a/CHANGELOG.md b/CHANGELOG.md index e4d0f93..5390acd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- Namespaces 1.0 constraints (#136): the namespace layer now enforces the binding rules as located parse errors: malformed qualified names (`foo:`, `:foo`, `a:b:c`, `xmlns:`) are rejected; the `xmlns` prefix may never be declared; `xml` may be bound only to its own namespace name, which no other prefix (nor the default) may take, likewise the xmlns namespace name; a prefix may not be undeclared (Namespaces 1.0); attributes must be distinct by expanded name (the same URI+local name through two prefixes is a duplicate); PI targets and entity/notation names are NCNames; and a namespace declaration whose attribute is DTD-declared with a tokenized type binds its normalized value (so ` urn:x ` and `urn:x` are one namespace name, eduni ns 012). Clears all 18 namespace cases from the eduni baseline, leaving only the four #137 encoding-mismatch cases; xmltest valid/sa/012 (an attribute literally named `:`) joins the documented deviations per the suite's own NAMESPACE='no' flag, mirrored by the eduni runner's new NAMESPACE filter. - W3C xmlconf eduni sections (#121, #127): the Edinburgh errata suites (2e, 3e, and the 5th-edition 4e), Namespaces 1.0 with its errata, and the misc cases run via the XMLCONF_ROOT runner; XML 1.1 and Namespaces 1.1 are explicitly out of scope and EDITION-gated cases skip when they do not apply to the Fifth Edition. The burn-down fixed a real spec bug and several errata behaviors: attribute values now get the 3.3.3 CDATA normalization (literal whitespace becomes a space before reference decoding, so character-referenced whitespace survives, fixing errata E36 and E20 semantics) and tokenized normalization collapses only SPACE characters; an undeclared general entity in content or an attribute value is a deferred validity finding instead of a fatal error when unread external declarations might supply it (production 68; the reference stays literal, nothing is fetched), matching the scanner-side split; the errata E15 family is enforced (any reference, comment, PI, or whitespace inside an element declared EMPTY; direct or double-escaped character-reference whitespace as character data in element content, while entity-supplied whitespace stays legal); errata E2 reports duplicate tokens in enumerated and NOTATION declarations; errata E14's declaration-completed-inside-a-PE-replacement parses and reports VC: Proper Declaration/PE Nesting; the ID family must be NCNames under namespace validation; and DTD-strict mode requires namespace declarations to be declared like any attribute. Remaining deviations are exact and classified: 24+1 grapheme-cluster lexing cases (a combining mark after an ASCII delimiter merges into one Swift Character), 18 namespace-constraint cases, 4 encoding-declaration mismatches, and E18 (per-entity base-URI tracking), each filed as its own issue. - W3C xmlconf japanese section, fully clean (#121, #126): all 13 cases (the same prose and parameter-entity DTDs encoded as UTF-8, UTF-16 in both byte orders, Shift_JIS, EUC-JP, and ISO-2022-JP) decode through PureXML's own byte decoder, parse, and DTD-validate cleanly with zero deviations; the optional encodings the suite classifies as TYPE='error' are all supported, so they are held to the must-parse bar. - W3C xmlconf IBM section, clean to two documented classes (#121, #125): the deepest block, 1094 cases. Valid 220/220 (parse + strict DTD validation), invalid 64/64 reporting, not-wf 529/810 rejected with the 281 accepted cases being exactly two documented classes: 279 cases from productions 85-89, the 1998 character-class appendices deleted by XML 1.0 Fifth Edition (whose count is asserted exactly so drift in either direction is caught), and 2 parameter-entity-in-internal-subset feature cases (the xmltest 160-162 class, headed for #128's strict profile). Closing the section's real gaps: a document without a root element is rejected (production 1); an entity reference in an entity literal must be a lexical Name or character reference (production 68); WFC: No Recursion is enforced at declaration over the combined general/parameter reference graph (CDATA-shielded), so a recursive pair is rejected without being referenced; an undeclared parameter-entity reference is a well-formedness error when the document is standalone or has neither an external subset nor parameter entities, and a deferred validity finding otherwise (the production-68 WFC/VC split, also applied to entity references in attribute defaults); a declared external PE the resolver refused is neither; and a content-model group opening in one PE replacement and closing in another is reported (VC: Proper Group/PE Nesting). diff --git a/README.md b/README.md index 029c248..8dbe794 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,7 @@ flowchart TB Eduni["#127 eduni: runner + exact baselines, four classified deviations"]:::done StrictSubset["#128 strict internal-subset profile: xmltest 185/186"]:::todo ScalarLex["#135 scalar-level lexing (grapheme clusters)"]:::todo - NSConstraints["#136 namespace constraints"]:::todo + NSConstraints["#136 namespace constraints: complete"]:::done EncodingMismatch["#137 encoding-declaration mismatches"]:::todo BaseURI["#138 per-entity base URIs"]:::todo Epic122["#122 official spec suites"]:::epic @@ -125,7 +125,7 @@ flowchart TB PureXML is a working, dependency-free XML library today: parse, emit, validate, query, and stream documents on macOS, Linux, Windows, and WASM. The test suite -currently runs **908 tests in 153 suites** (`swift test`). +currently runs **914 tests in 154 suites** (`swift test`). ### Shipped (libxml2-aligned surface) diff --git a/Sources/Parsing/Diagnostic.swift b/Sources/Parsing/Diagnostic.swift index 633d4ec..6615fc6 100644 --- a/Sources/Parsing/Diagnostic.swift +++ b/Sources/Parsing/Diagnostic.swift @@ -45,6 +45,7 @@ public extension PureXML.Parsing.ParseError { let .invalidPublicIdentifier(mark): mark case let .unexpectedCharacter(_, mark), let .mismatchedEndTag(_, _, mark), + let .namespaceConstraint(_, mark), let .duplicateAttribute(_, mark), let .invalidReference(_, mark), let .undefinedEntity(_, mark), let .recursiveEntity(_, mark), let .unexpectedEndTag(_, mark), let .undefinedNamespacePrefix(_, mark), diff --git a/Sources/Parsing/DoctypeScanner.swift b/Sources/Parsing/DoctypeScanner.swift index 7b68298..2c721e9 100644 --- a/Sources/Parsing/DoctypeScanner.swift +++ b/Sources/Parsing/DoctypeScanner.swift @@ -158,6 +158,9 @@ struct DTDScanner { if target.lowercased() == "xml" { throw ParseError.reservedProcessingInstructionTarget(mark) } + if target.contains(":") { + throw ParseError.namespaceConstraint(reason: "PI target '\(target)' is not a legal NCName", mark) + } if let next = reader.peek(), !next.isWhitespace, !reader.matches("?>") { throw ParseError.unexpectedCharacter(next, reader.mark) } diff --git a/Sources/Parsing/DoctypeScannerStrict.swift b/Sources/Parsing/DoctypeScannerStrict.swift index 2b417c7..dd729cc 100644 --- a/Sources/Parsing/DoctypeScannerStrict.swift +++ b/Sources/Parsing/DoctypeScannerStrict.swift @@ -119,12 +119,17 @@ extension DTDScanner { /// Scans a Name strictly: the first character must satisfy NameStartChar /// and the rest NameChar, so `-ge` or `.pe` is rejected where the lenient - /// continuation-character scan would accept it. + /// continuation-character scan would accept it. Entity and notation names + /// are NCNames under namespaces, so a colon is also rejected. mutating func scanStrictName(_ reader: inout Reader, at mark: Mark) throws -> String { guard let first = reader.peek(), first.unicodeScalars.allSatisfy(PureXML.Parsing.XMLCharacter.isNameStart) else { throw ParseError.expectedName(mark) } - return scanName(&reader) + let name = scanName(&reader) + guard !name.contains(":") else { + throw ParseError.namespaceConstraint(reason: "'\(name)' is not a legal NCName", mark) + } + return name } /// Removes a leading text declaration from an external entity's resolved diff --git a/Sources/Parsing/EventReader.swift b/Sources/Parsing/EventReader.swift index 01bfabd..6d18ccc 100644 --- a/Sources/Parsing/EventReader.swift +++ b/Sources/Parsing/EventReader.swift @@ -173,7 +173,7 @@ public extension PureXML.Parsing { let mark = reader.mark reader.consume("<") let rawName = try scanName() - let rawAttributes = try scanAttributes() + let rawAttributes = try normalizedBindings(element: rawName, attributes: scanAttributes()) let resolved = try namespaces.enterElement(name: rawName, attributes: rawAttributes, at: mark) if reader.consume("/>") { namespaces.leaveElement() @@ -255,6 +255,10 @@ public extension PureXML.Parsing { let mark = reader.mark reader.consume("` is not well-formed. if let next = reader.peek(), !next.isWhitespace, !reader.matches("?>") { diff --git a/Sources/Parsing/EventReaderValidity.swift b/Sources/Parsing/EventReaderValidity.swift index 550e316..195d681 100644 --- a/Sources/Parsing/EventReaderValidity.swift +++ b/Sources/Parsing/EventReaderValidity.swift @@ -103,4 +103,28 @@ extension PureXML.Parsing.EventReader { documentType.validityFindings.append("character-reference whitespace in the element content of <\(current.description)>") } } + + /// Applies the DTD-declared tokenized normalization to namespace + /// declaration values before they bind: an `xmlns:b` declared NMTOKEN in + /// the DTD binds its collapsed value, so ' urn:x ' and 'urn:x' are the + /// same namespace name (eduni ns 1.0/012). + mutating func normalizedBindings( + element: PureXML.Model.QualifiedName, + attributes: [PureXML.Model.Attribute], + ) -> [PureXML.Model.Attribute] { + let hasBinding = attributes.contains { $0.name.prefix == "xmlns" || ($0.name.prefix == nil && $0.name.localName == "xmlns") } + guard hasBinding, + let body = documentType.attributeLists[element.description] ?? documentType.attributeLists[element.localName] + else { return attributes } + let declarations = PureXML.Validation.AttributeListParser.parse(body) + return attributes.map { attribute in + let isBinding = attribute.name.prefix == "xmlns" || (attribute.name.prefix == nil && attribute.name.localName == "xmlns") + guard isBinding, + let declaration = declarations.first(where: { $0.name == attribute.name.description }) + else { return attribute } + if case .cdata = declaration.type { return attribute } + let collapsed = attribute.value.split(separator: " ").joined(separator: " ") + return PureXML.Model.Attribute(name: attribute.name, value: collapsed) + } + } } diff --git a/Sources/Parsing/NamespaceContext.swift b/Sources/Parsing/NamespaceContext.swift index 09b5d14..b3c69b8 100644 --- a/Sources/Parsing/NamespaceContext.swift +++ b/Sources/Parsing/NamespaceContext.swift @@ -9,6 +9,9 @@ extension PureXML.Parsing { struct NamespaceContext { /// The reserved binding for the `xml` prefix. static let xmlNamespaceURI = "http://www.w3.org/XML/1998/namespace" + /// The namespace name reserved for `xmlns` itself; it may never be + /// bound by a declaration. + static let xmlnsNamespaceURI = "http://www.w3.org/2000/xmlns/" /// A stack of scopes; each maps a prefix ("" = default namespace) to a URI. private var scopes: [[String: String]] = [] @@ -21,11 +24,15 @@ extension PureXML.Parsing { attributes: [PureXML.Model.Attribute], at mark: Mark, ) throws -> (name: PureXML.Model.QualifiedName, attributes: [PureXML.Model.Attribute]) { + try checkNameShape(name, at: mark) var scope: [String: String] = [:] for attribute in attributes { + try checkNameShape(attribute.name, at: mark) if attribute.name.prefix == nil, attribute.name.localName == "xmlns" { + try checkDefaultBinding(attribute.value, at: mark) scope[""] = attribute.value } else if attribute.name.prefix == "xmlns" { + try checkPrefixBinding(attribute.name.localName, uri: attribute.value, at: mark) scope[attribute.name.localName] = attribute.value } } @@ -33,6 +40,7 @@ extension PureXML.Parsing { let resolvedName = try resolveElement(name, at: mark) let resolvedAttributes = try attributes.map { try resolveAttribute($0, at: mark) } + try checkExpandedNames(resolvedAttributes, at: mark) return (resolvedName, resolvedAttributes) } @@ -75,6 +83,55 @@ extension PureXML.Parsing { throw ParseError.undefinedNamespacePrefix(prefix: prefix ?? "", mark) } + /// A qualified name has exactly one colon separating two non-empty + /// NCNames; `QualifiedName` keeps a malformed raw form (`foo:`, + /// `:foo`, `a:b:c`, `xmlns:`) in its local name, where the colon + /// betrays it. + private func checkNameShape(_ name: PureXML.Model.QualifiedName, at mark: Mark) throws { + guard name.localName.contains(":") else { return } + throw ParseError.namespaceConstraint(reason: "'\(name.description)' is not a legal qualified name", mark) + } + + /// The reserved-binding rules for `xmlns:prefix` declarations: `xmlns` + /// itself may never be declared, `xml` only with its own namespace + /// name, no other prefix may take either reserved namespace name, and + /// a prefix may not be undeclared in Namespaces 1.0. + private func checkPrefixBinding(_ prefix: String, uri: String, at mark: Mark) throws { + if prefix == "xmlns" { + throw ParseError.namespaceConstraint(reason: "the 'xmlns' prefix must not be declared", mark) + } + if prefix == "xml" { + guard uri == Self.xmlNamespaceURI else { + throw ParseError.namespaceConstraint(reason: "the 'xml' prefix is bound to '\(Self.xmlNamespaceURI)' and cannot be rebound", mark) + } + return + } + if uri == Self.xmlNamespaceURI || uri == Self.xmlnsNamespaceURI { + throw ParseError.namespaceConstraint(reason: "the reserved namespace name '\(uri)' must not be bound to prefix '\(prefix)'", mark) + } + if uri.isEmpty { + throw ParseError.namespaceConstraint(reason: "prefix '\(prefix)' may not be undeclared in Namespaces 1.0", mark) + } + } + + /// Neither reserved namespace name may become the default namespace. + private func checkDefaultBinding(_ uri: String, at mark: Mark) throws { + guard uri == Self.xmlNamespaceURI || uri == Self.xmlnsNamespaceURI else { return } + throw ParseError.namespaceConstraint(reason: "the reserved namespace name '\(uri)' must not be the default namespace", mark) + } + + /// Attributes must be distinct by expanded name: two prefixes bound to + /// the same URI cannot carry the same local name on one element. + private func checkExpandedNames(_ attributes: [PureXML.Model.Attribute], at mark: Mark) throws { + var seen: Set = [] + for attribute in attributes where attribute.name.namespaceURI != nil { + let key = (attribute.name.namespaceURI ?? "") + "\u{0}" + attribute.name.localName + guard seen.insert(key).inserted else { + throw ParseError.duplicateAttribute(name: attribute.name.description, mark) + } + } + } + private func lookup(_ key: String) -> String? { for scope in scopes.reversed() { if let uri = scope[key] { diff --git a/Sources/Parsing/ParseError.swift b/Sources/Parsing/ParseError.swift index 275c1b5..995faa3 100644 --- a/Sources/Parsing/ParseError.swift +++ b/Sources/Parsing/ParseError.swift @@ -56,6 +56,9 @@ public extension PureXML.Parsing { case invalidAttributeListDeclaration(Mark) /// A public identifier literal contains a character outside `PubidChar`. case invalidPublicIdentifier(Mark) + /// A namespace constraint is violated: a malformed qualified name, a + /// reserved prefix or namespace name misused, or an illegal binding. + case namespaceConstraint(reason: String, Mark) public var description: String { switch self { @@ -125,6 +128,8 @@ public extension PureXML.Parsing { "malformed declaration at \(mark)" case let .invalidAttributeListDeclaration(mark): "malformed declaration at \(mark)" + case let .namespaceConstraint(reason, mark): + "\(reason) at \(mark)" case let .invalidPublicIdentifier(mark): "public identifier contains an illegal character at \(mark)" case let .malformedDeclaration(mark): diff --git a/Tests/NamespaceConstraintTests.swift b/Tests/NamespaceConstraintTests.swift new file mode 100644 index 0000000..37a0364 --- /dev/null +++ b/Tests/NamespaceConstraintTests.swift @@ -0,0 +1,70 @@ +@testable import PureXML +import Testing + +/// The Namespaces 1.0 constraints (#136): reserved prefixes and namespace +/// names, qualified-name shape, duplicate expanded-name attributes, NCName +/// rules for PI targets and entity/notation names, and DTD-typed +/// normalization of binding values. +@Suite("Namespace constraints") +struct NamespaceConstraintTests { + private func rejects(_ xml: String) -> Bool { + (try? PureXML.parse(xml, limits: .init(allowDoctype: true))) == nil + } + + @Test("Malformed qualified names are rejected") + func test_qualifiedNameShape() { + #expect(rejects("")) + #expect(rejects("<:foo />")) + #expect(rejects("")) + #expect(rejects("")) + #expect(!rejects("")) + } + + @Test("Reserved prefixes and namespace names are protected") + func test_reservedBindings() { + #expect(rejects("")) + #expect(!rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + #expect(rejects("")) + } + + @Test("A prefix may not be undeclared in Namespaces 1.0") + func test_noPrefixUndeclaration() { + #expect(rejects("")) + #expect(!rejects("")) + } + + @Test("Attributes must be distinct by expanded name") + func test_duplicateExpandedNames() { + let xml = "" + #expect(rejects(xml)) + let distinct = "" + #expect(!rejects(distinct)) + } + + @Test("PI targets and entity/notation names are NCNames") + func test_ncNames() { + #expect(rejects("\n")) + #expect(rejects("]>\n")) + #expect(rejects("]>\n")) + } + + @Test("A DTD-typed xmlns value binds its normalized form") + func test_typedBindingNormalization() { + let xml = """ + + + + ]> + + + + """ + #expect(rejects(xml)) + } +} diff --git a/Tests/W3CEduniSuiteTests.swift b/Tests/W3CEduniSuiteTests.swift index 9ad499c..234eb2f 100644 --- a/Tests/W3CEduniSuiteTests.swift +++ b/Tests/W3CEduniSuiteTests.swift @@ -26,21 +26,11 @@ struct W3CEduniSuiteTests { ("misc/ht-bh.xml", "misc"), ] - /// not-wf cases this implementation knowingly accepts, two classes: - /// the namespace constraints not yet enforced (reserved prefixes and - /// namespace names, duplicate expanded-name attributes, NCName rules) - /// and encoding-declaration mismatches the byte decoder does not yet - /// reject (a declared encoding contradicting the actual bytes or BOM). - /// Both classes are tracked as their own issues under #121. + /// not-wf cases this implementation knowingly accepts: the encoding- + /// declaration mismatches the byte decoder does not yet reject (a + /// declared encoding contradicting the actual bytes or BOM), tracked + /// as #137. private let knownNotWFAccepted: Set = [ - // Namespace constraints. - "namespaces/1.0/009.xml", "namespaces/1.0/010.xml", "namespaces/1.0/011.xml", - "namespaces/1.0/012.xml", "namespaces/1.0/014.xml", "namespaces/1.0/015.xml", - "namespaces/1.0/016.xml", "namespaces/1.0/029.xml", "namespaces/1.0/030.xml", - "namespaces/1.0/031.xml", "namespaces/1.0/032.xml", "namespaces/1.0/033.xml", - "namespaces/1.0/036.xml", "namespaces/1.0/042.xml", "namespaces/1.0/043.xml", - "namespaces/1.0/044.xml", "namespaces/errata-1e/NE13a.xml", "namespaces/errata-1e/NE13b.xml", - // Encoding-declaration mismatches. "errata-2e/E38.xml", "errata-2e/E61.xml", "misc/007.xml", "misc/008.xml", ] @@ -73,6 +63,7 @@ struct W3CEduniSuiteTests { let type: String let uri: String let editions: String? + let namespaceAware: Bool } private func manifest(_ file: String, in eduni: String) throws -> [ManifestCase] { @@ -89,12 +80,21 @@ struct W3CEduniSuiteTests { guard let id = attribute("ID"), let type = attribute("TYPE"), let uri = attribute("URI") else { return nil } - return ManifestCase(id: id, type: type, uri: uri, editions: attribute("EDITION")) + return ManifestCase( + id: id, + type: type, + uri: uri, + editions: attribute("EDITION"), + namespaceAware: attribute("NAMESPACE") != "no", + ) } } - /// Whether a case applies to the Fifth Edition this package implements. + /// Whether a case applies to this implementation: the Fifth Edition, and + /// namespace-aware processing (NAMESPACE='no' cases use pre-namespace + /// colons in names, which a namespace-aware parser correctly refuses). private func appliesToFifthEdition(_ testCase: ManifestCase) -> Bool { + guard testCase.namespaceAware else { return false } guard let editions = testCase.editions else { return true } return editions.split(separator: " ").contains("5") } diff --git a/Tests/W3CXMLSuiteTests.swift b/Tests/W3CXMLSuiteTests.swift index 99df02a..2e569b1 100644 --- a/Tests/W3CXMLSuiteTests.swift +++ b/Tests/W3CXMLSuiteTests.swift @@ -27,10 +27,10 @@ struct W3CXMLSuiteTests { "063.xml", "141.xml", "160.xml", "161.xml", "162.xml", ] - /// valid/sa/114.xml: a CDATA section inside an entity value must protect - /// the &foo; reference from expansion; the entity decoder expands it - /// before the CDATA section is recognized. Same hardening track. - private let knownValidDeviations: Set = [] + /// valid/sa/012.xml is flagged NAMESPACE='no' by the suite's own + /// manifest: it declares an attribute named ':', a pre-namespace XML 1.0 + /// Name that a namespace-aware parser correctly refuses. + private let knownValidDeviations: Set = ["012.xml"] private func files(in directory: String) throws -> [String] { try FileManager.default.contentsOfDirectory(atPath: directory)