feat(parsing): per-entity base-URI tracking for nested external entities
A relative system identifier declared inside an external entity resolves against that entity's own URI (RFC 3986, the C14N resolver): ExternalID gains base and resolvedSystemID, the scanner threads the current entity's URI through external-subset and PE scans, and resolvers receive the merged identifier. Clears eduni E18; the eduni baseline now holds only the grapheme-lexing class (#135). Closes #138
This commit is contained in:
parent
2c375ac74b
commit
3d56d19cbd
@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Added
|
||||
|
||||
- Per-entity base-URI tracking (#138): a relative system identifier declared inside an external entity now resolves against that entity's own URI (RFC 3986, reusing the C14N resolver), so nested external entities find their siblings; `ExternalID` gains an optional `base` and a `resolvedSystemID`, the DTD scanner threads the current entity's URI through external-subset and parameter-entity scans, and resolvers receive the already-merged identifier (raw identifiers unchanged when no base applies). Clears eduni errata-2e/E18; the eduni baseline now holds only the #135 grapheme-lexing class.
|
||||
- Encoding-declaration contradictions are fatal (#137, 4.3.3): a document whose XML declaration names a 16- or 32-bit encoding over BOM-less 8-bit bytes is rejected; a BOM followed by a declaration outside its family (a UTF-8 BOM with `encoding='iso-8859-1'`, a UTF-16 BOM with an 8-bit name) is rejected, with the declaration read through the BOM's own byte pattern; and an external entity whose text declaration names a higher XML version than the document's is refused (errata E38), in both the external-subset and external-entity paths. The eduni not-wf baseline is now completely empty.
|
||||
- Namespaces 1.0 constraints (#136): the namespace layer now enforces the binding rules as located parse errors: malformed qualified names (`foo:`, `:foo`, `a:b:c`, `xmlns:`) are rejected; the `xmlns` prefix may never be declared; `xml` may be bound only to its own namespace name, which no other prefix (nor the default) may take, likewise the xmlns namespace name; a prefix may not be undeclared (Namespaces 1.0); attributes must be distinct by expanded name (the same URI+local name through two prefixes is a duplicate); PI targets and entity/notation names are NCNames; and a namespace declaration whose attribute is DTD-declared with a tokenized type binds its normalized value (so ` urn:x ` and `urn:x` are one namespace name, eduni ns 012). Clears all 18 namespace cases from the eduni baseline, leaving only the four #137 encoding-mismatch cases; xmltest valid/sa/012 (an attribute literally named `:`) joins the documented deviations per the suite's own NAMESPACE='no' flag, mirrored by the eduni runner's new NAMESPACE filter.
|
||||
- W3C xmlconf eduni sections (#121, #127): the Edinburgh errata suites (2e, 3e, and the 5th-edition 4e), Namespaces 1.0 with its errata, and the misc cases run via the XMLCONF_ROOT runner; XML 1.1 and Namespaces 1.1 are explicitly out of scope and EDITION-gated cases skip when they do not apply to the Fifth Edition. The burn-down fixed a real spec bug and several errata behaviors: attribute values now get the 3.3.3 CDATA normalization (literal whitespace becomes a space before reference decoding, so character-referenced whitespace survives, fixing errata E36 and E20 semantics) and tokenized normalization collapses only SPACE characters; an undeclared general entity in content or an attribute value is a deferred validity finding instead of a fatal error when unread external declarations might supply it (production 68; the reference stays literal, nothing is fetched), matching the scanner-side split; the errata E15 family is enforced (any reference, comment, PI, or whitespace inside an element declared EMPTY; direct or double-escaped character-reference whitespace as character data in element content, while entity-supplied whitespace stays legal); errata E2 reports duplicate tokens in enumerated and NOTATION declarations; errata E14's declaration-completed-inside-a-PE-replacement parses and reports VC: Proper Declaration/PE Nesting; the ID family must be NCNames under namespace validation; and DTD-strict mode requires namespace declarations to be declared like any attribute. Remaining deviations are exact and classified: 24+1 grapheme-cluster lexing cases (a combining mark after an ASCII delimiter merges into one Swift Character), 18 namespace-constraint cases, 4 encoding-declaration mismatches, and E18 (per-entity base-URI tracking), each filed as its own issue.
|
||||
|
||||
@ -97,7 +97,7 @@ flowchart TB
|
||||
ScalarLex["#135 scalar-level lexing (grapheme clusters)"]:::todo
|
||||
NSConstraints["#136 namespace constraints: complete"]:::done
|
||||
EncodingMismatch["#137 encoding-declaration mismatches: complete"]:::done
|
||||
BaseURI["#138 per-entity base URIs"]:::todo
|
||||
BaseURI["#138 per-entity base URIs: complete"]:::done
|
||||
Epic122["#122 official spec suites"]:::epic
|
||||
XSTS["#129 W3C XSD test suite (XSTS)"]:::todo
|
||||
XSLTTS["#130 OASIS XSLT 1.0 suite (+XPath corpus)"]:::todo
|
||||
@ -125,7 +125,7 @@ flowchart TB
|
||||
|
||||
PureXML is a working, dependency-free XML library today: parse, emit, validate,
|
||||
query, and stream documents on macOS, Linux, Windows, and WASM. The test suite
|
||||
currently runs **917 tests in 155 suites** (`swift test`).
|
||||
currently runs **919 tests in 156 suites** (`swift test`).
|
||||
|
||||
### Shipped (libxml2-aligned surface)
|
||||
|
||||
|
||||
@ -54,6 +54,12 @@ struct DTDScanner {
|
||||
/// resolver refused them): a reference to one is not an undeclared-entity
|
||||
/// error, the processor simply did not read it.
|
||||
var unresolvedParameterEntities: Set<String> = []
|
||||
/// The URI of the entity whose text is being scanned (nil for the
|
||||
/// document): relative identifiers declared here resolve against it.
|
||||
var currentBase: String?
|
||||
/// Each external parameter entity's own resolved URI, the base for the
|
||||
/// identifiers its replacement declares.
|
||||
var parameterEntityBases: [String: String] = [:]
|
||||
var parameterBudget: Int
|
||||
/// Bounds parameter-entity injection recursion (modularized DTDs nest, but
|
||||
/// only a little); deeper references are ignored rather than trapping.
|
||||
@ -112,7 +118,11 @@ struct DTDScanner {
|
||||
}
|
||||
var sub = Reader(text)
|
||||
inExternalContext = true
|
||||
defer { inExternalContext = false }
|
||||
currentBase = id.resolvedSystemID
|
||||
defer {
|
||||
inExternalContext = false
|
||||
currentBase = nil
|
||||
}
|
||||
try scanTextDeclaration(&sub, at: mark)
|
||||
try scanDeclarations(&sub, depth: 1, terminatedByBracket: false, at: mark)
|
||||
}
|
||||
|
||||
@ -9,7 +9,7 @@ extension DTDScanner {
|
||||
/// resolver keeps XXE closed.
|
||||
mutating func resolveExternalEntities() {
|
||||
for (name, id) in doctype.externalEntities where doctype.entities[name] == nil {
|
||||
if let text = resolver.resolveEntity(name, id) {
|
||||
if let text = resolver.resolveEntity(name, baseResolved(id)) {
|
||||
// An external parsed entity may begin with a text declaration,
|
||||
// which is not part of its replacement text (4.3.1).
|
||||
doctype.entities[name] = try? strippingTextDeclaration(text)
|
||||
@ -17,6 +17,15 @@ extension DTDScanner {
|
||||
}
|
||||
}
|
||||
|
||||
/// The identifier with its base applied into the system ID, so a resolver
|
||||
/// that only reads `systemID` still sees the resolved path.
|
||||
func baseResolved(_ id: ExternalID) -> ExternalID {
|
||||
var resolved = id
|
||||
resolved.systemID = id.resolvedSystemID
|
||||
resolved.base = nil
|
||||
return resolved
|
||||
}
|
||||
|
||||
/// Handles a bare `%name;` between declarations by injecting the parameter
|
||||
/// entity's replacement text and scanning its declarations. Bounded by depth
|
||||
/// and the expansion budget.
|
||||
@ -49,6 +58,13 @@ extension DTDScanner {
|
||||
}
|
||||
parameterBudget -= replacement.count
|
||||
var sub = Reader(replacement)
|
||||
// Identifiers declared inside the replacement resolve against the
|
||||
// parameter entity's own URI (per-entity base, RFC 3986).
|
||||
let outerBase = currentBase
|
||||
if let base = parameterEntityBases[name] {
|
||||
currentBase = base
|
||||
}
|
||||
defer { currentBase = outerBase }
|
||||
try scanDeclarations(&sub, depth: depth + 1, terminatedByBracket: false, at: mark)
|
||||
}
|
||||
|
||||
@ -64,9 +80,10 @@ extension DTDScanner {
|
||||
}
|
||||
} else if isParameter {
|
||||
if doctype.parameterEntities[name] == nil {
|
||||
if let text = resolver.resolveExternalSubset(id) {
|
||||
if let text = resolver.resolveExternalSubset(baseResolved(id)) {
|
||||
if let stripped = try? strippingTextDeclaration(text) {
|
||||
doctype.parameterEntities[name] = expandParameterReferences(stripped)
|
||||
parameterEntityBases[name] = id.resolvedSystemID
|
||||
} else {
|
||||
unresolvedParameterEntities.insert(name)
|
||||
}
|
||||
|
||||
@ -203,7 +203,7 @@ extension DTDScanner {
|
||||
guard let systemID = scanLiteral(&reader) else {
|
||||
throw ParseError.invalidEntityDeclaration(mark)
|
||||
}
|
||||
return ExternalID(systemID: systemID)
|
||||
return ExternalID(systemID: systemID, base: currentBase)
|
||||
}
|
||||
if reader.consume("PUBLIC") {
|
||||
return try parsePublicID(&reader, requireSystem: requireSystem, at: mark)
|
||||
@ -230,7 +230,7 @@ extension DTDScanner {
|
||||
guard let systemID = scanLiteral(&reader) else {
|
||||
throw ParseError.invalidEntityDeclaration(mark)
|
||||
}
|
||||
return ExternalID(publicID: publicID, systemID: systemID)
|
||||
return ExternalID(publicID: publicID, systemID: systemID, base: currentBase)
|
||||
}
|
||||
// Notations may carry a public identifier alone.
|
||||
var systemID = ""
|
||||
@ -238,7 +238,7 @@ extension DTDScanner {
|
||||
reader.skipSpace()
|
||||
if let literal = scanLiteral(&reader) { systemID = literal }
|
||||
}
|
||||
return ExternalID(publicID: publicID, systemID: systemID)
|
||||
return ExternalID(publicID: publicID, systemID: systemID, base: currentBase)
|
||||
}
|
||||
|
||||
/// `PubidChar ::= #x20 | #xD | #xA | [a-zA-Z0-9] | [-'()+,./:=?;!*#@$_%]`
|
||||
|
||||
@ -13,10 +13,28 @@ public extension PureXML.Parsing {
|
||||
public var publicID: String?
|
||||
/// The system identifier (the URI or path literal).
|
||||
public var systemID: String
|
||||
/// The URI of the entity whose text declared this identifier, when
|
||||
/// known: a relative system identifier resolves against it (RFC 3986),
|
||||
/// so nested external entities find their siblings.
|
||||
public var base: String?
|
||||
|
||||
public init(publicID: String? = nil, systemID: String) {
|
||||
public init(publicID: String? = nil, systemID: String, base: String? = nil) {
|
||||
self.publicID = publicID
|
||||
self.systemID = systemID
|
||||
self.base = base
|
||||
}
|
||||
|
||||
/// The system identifier with any base applied, ready for a resolver.
|
||||
public var resolvedSystemID: String {
|
||||
guard let base else { return systemID }
|
||||
let merged = PureXML.Canonical.Canonicalizer.resolveURI(systemID, against: base)
|
||||
// The RFC 3986 merge assumes an absolute base; when both base and
|
||||
// reference are relative paths, keep the merged path relative.
|
||||
let baseHasScheme = base.split(separator: "/", maxSplits: 1)[0].hasSuffix(":")
|
||||
if !base.hasPrefix("/"), !baseHasScheme, !systemID.hasPrefix("/"), merged.hasPrefix("/") {
|
||||
return String(merged.dropFirst())
|
||||
}
|
||||
return merged
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
49
Tests/EntityBaseURITests.swift
Normal file
49
Tests/EntityBaseURITests.swift
Normal file
@ -0,0 +1,49 @@
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
/// Per-entity base-URI tracking (#138): a relative system identifier inside
|
||||
/// an external entity resolves against that entity's own URI, not the
|
||||
/// document's, so nested external entities find their siblings (RFC 3986).
|
||||
@Suite("Entity base URIs")
|
||||
struct EntityBaseURITests {
|
||||
@Test("Nested relative identifiers resolve against the declaring entity's URI")
|
||||
func test_nestedRelativeResolution() throws {
|
||||
// The eduni E18 shape: subdir1/pe declares ../subdir2/extpe, which
|
||||
// declares the general entity the document references.
|
||||
let files: [String: String] = [
|
||||
"subdir1/pe": "<!ENTITY % extpe SYSTEM \"../subdir2/extpe\">\n<!ENTITY % intpe \"%extpe;\">",
|
||||
"subdir2/extpe": "<!ENTITY ent \"resolved\">",
|
||||
]
|
||||
let resolver = PureXML.Parsing.EntityResolver(
|
||||
resolveEntity: { _, id in files[id.systemID] },
|
||||
resolveExternalSubset: { id in files[id.systemID] },
|
||||
)
|
||||
let xml = """
|
||||
<!DOCTYPE foo [
|
||||
<!ELEMENT foo ANY>
|
||||
<!ENTITY % pe SYSTEM "subdir1/pe">
|
||||
%pe;
|
||||
%intpe;
|
||||
]>
|
||||
<foo>&ent;</foo>
|
||||
"""
|
||||
let node = try PureXML.parse(xml, limits: .init(allowDoctype: true), resolver: resolver)
|
||||
guard case let .document(children) = node, let foo = children.compactMap(\.element).first,
|
||||
case let .text(value)? = foo.children.first
|
||||
else {
|
||||
Issue.record("no text")
|
||||
return
|
||||
}
|
||||
#expect(value == "resolved")
|
||||
}
|
||||
|
||||
@Test("The resolver still sees raw identifiers when no base applies")
|
||||
func test_documentLevelUnchanged() {
|
||||
let resolver = PureXML.Parsing.EntityResolver(
|
||||
resolveEntity: { _, id in id.systemID == "plain.ent" ? "ok" : nil },
|
||||
resolveExternalSubset: { _ in nil },
|
||||
)
|
||||
let xml = "<!DOCTYPE f [<!ELEMENT f ANY><!ENTITY e SYSTEM \"plain.ent\">]>\n<f>&e;</f>"
|
||||
#expect((try? PureXML.parse(xml, limits: .init(allowDoctype: true), resolver: resolver)) != nil)
|
||||
}
|
||||
}
|
||||
@ -29,13 +29,11 @@ struct W3CEduniSuiteTests {
|
||||
/// not-wf cases this implementation knowingly accepts (none).
|
||||
private let knownNotWFAccepted: Set<String> = []
|
||||
|
||||
/// valid cases this implementation knowingly fails, two classes: the
|
||||
/// valid cases this implementation knowingly fails, one class: the
|
||||
/// grapheme-cluster lexing limitation (a combining mark directly after an
|
||||
/// ASCII delimiter merges into one Swift Character, so scalar-level 5e
|
||||
/// name characters are mis-lexed; its own issue under #121) and E18,
|
||||
/// which needs per-entity base-URI tracking for nested external entities.
|
||||
/// name characters are mis-lexed; #135).
|
||||
private let knownValidFailures: Set<String> = [
|
||||
"errata-2e/E18.xml",
|
||||
"errata-4e/ibm04v01.xml", "errata-4e/ibm05v03.xml", "errata-4e/ibm07v01.xml",
|
||||
"errata-4e/ibm85n107.xml", "errata-4e/ibm85n114.xml", "errata-4e/ibm85n119.xml",
|
||||
"errata-4e/ibm85n121.xml", "errata-4e/ibm85n122.xml", "errata-4e/ibm85n136.xml",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user