feat(parsing): strict DTD subset grammar, oasis section of xmlconf fully clean
The OASIS/NIST section of the W3C conformance suite runs via a new XMLCONF_ROOT-gated runner driven by the section's own manifest (parsed with PureXML): 247/247 not-wf rejected, all 100 well-formed cases parse, zero deviations. Closing its 24 gaps makes the DTD subset grammar strict: exact PEReference shape, strict declaration names and keywords, NDATA and notation tails, no bare '%' in entity values, case-sensitive balanced conditional sections, junk between declarations rejected, external-subset violations now propagate (with text-declaration support, production 77), and a PI target must be separated from its data. The xmltest baseline tightens to 181/186: CDATA in the internal subset (107) is now rejected, five documented deviations remain. Closes #123
This commit is contained in:
parent
676b719003
commit
5671fb3ce1
@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- W3C xmlconf OASIS/NIST section, fully clean (#121, #123): a second opt-in runner (XMLCONF_ROOT-gated, suite never vendored) drives the oasis section from its own manifest, parsed with PureXML itself; all 247 not-wf cases are rejected and all 100 well-formed (valid + invalid) cases parse, with zero deviations (the one excluded case is flagged NAMESPACE='no' by the manifest itself: pre-namespace colons in names, which a namespace-aware parser correctly refuses). Closing the section's last 24 gaps made the DTD subset grammar strict: unknown, lowercase, or space-separated markup-declaration keywords are rejected; a parameter-entity reference must be '%' Name ';' exactly; entity, parameter-entity, and notation names must start with a NameStartChar; NDATA requires whitespace and a strict name; a notation declaration requires its identifier and a clean tail; entity value literals admit no bare '%' (production 9); conditional-section keywords are case-sensitive INCLUDE/IGNORE and sections must balance to their ']]>'; junk between declarations is not well-formed; a grammar violation in the external subset now rejects the document (previously swallowed); the external subset may open with a text declaration (version optional, encoding required, standalone forbidden, production 77); and a processing-instruction target must be separated from its data by whitespace (production 16). The new strictness also tightened the xmltest baseline from 180/186 to 181/186: a CDATA section in the internal subset (case 107) is now correctly rejected by the conditional-keyword check, leaving five documented deviations.
|
||||
|
||||
### Changed
|
||||
|
||||
- Schema-consistency checks now run through the validation framework (#117 follow-up): the derivation `final` check and Particle Valid (Restriction) are composable `Validation<SchemaTypeFact, CompiledSchemaFacts>` rules (`finalRespected`, `restrictionsAreSubsets`) applied per named type, so a schema with several problems reports them ALL at once. `SchemaError.finalViolation` and `.invalidRestriction` are replaced by `.inconsistent([String])` carrying every finding with its type's coding path; the finding texts are unchanged. `checkRedefine`, `checkAllGroups`, and `notASchema` stay fail-fast throws because they examine the raw schema source, the compilation analog of well-formedness.
|
||||
|
||||
@ -88,7 +88,7 @@ flowchart TB
|
||||
Epic74 --> Done
|
||||
Epic105["#105 parity frontier: complete (#106 #107 #108 output encodings, streaming validation, last encodings; findings #109-#119; first W3C pass #120)"]:::epic
|
||||
Epic121["#121 xmlconf full archive"]:::epic
|
||||
Oasis["#123 OASIS/NIST section"]:::todo
|
||||
Oasis["#123 OASIS/NIST section: complete, zero deviations"]:::done
|
||||
Sun["#124 Sun section: DTD-validity baselines"]:::todo
|
||||
IBM["#125 IBM section: per-production validity depth"]:::todo
|
||||
Japanese["#126 japanese section: encoding declarations"]:::todo
|
||||
@ -117,7 +117,7 @@ flowchart TB
|
||||
|
||||
PureXML is a working, dependency-free XML library today: parse, emit, validate,
|
||||
query, and stream documents on macOS, Linux, Windows, and WASM. The test suite
|
||||
currently runs **850 tests in 142 suites** (`swift test`).
|
||||
currently runs **863 tests in 144 suites** (`swift test`).
|
||||
|
||||
### Shipped (libxml2-aligned surface)
|
||||
|
||||
|
||||
@ -41,7 +41,7 @@ struct DTDScanner {
|
||||
var parameterBudget: Int
|
||||
/// Bounds parameter-entity injection recursion (modularized DTDs nest, but
|
||||
/// only a little); deeper references are ignored rather than trapping.
|
||||
private let maxDepth = 40
|
||||
let maxDepth = 40
|
||||
|
||||
init(limits: PureXML.Parsing.Limits, resolver: PureXML.Parsing.EntityResolver) {
|
||||
self.limits = limits
|
||||
@ -63,7 +63,7 @@ struct DTDScanner {
|
||||
guard reader.consume(">") else {
|
||||
throw ParseError.unterminatedTag(mark)
|
||||
}
|
||||
loadExternalSubset(at: mark)
|
||||
try loadExternalSubset(at: mark)
|
||||
resolveExternalEntities()
|
||||
return doctype
|
||||
}
|
||||
@ -82,16 +82,19 @@ struct DTDScanner {
|
||||
|
||||
/// Loads the external subset through the resolver, if one is configured and
|
||||
/// returns text. External declarations never override internal ones (the
|
||||
/// internal subset is scanned first and wins).
|
||||
private mutating func loadExternalSubset(at mark: Mark) {
|
||||
/// internal subset is scanned first and wins). A grammar violation in the
|
||||
/// external subset is a well-formedness error of the document, so errors
|
||||
/// propagate. The subset may open with a text declaration (production 77).
|
||||
private mutating func loadExternalSubset(at mark: Mark) throws {
|
||||
guard let id = doctype.externalSubset, let text = resolver.resolveExternalSubset(id) else {
|
||||
return
|
||||
}
|
||||
var sub = Reader(text)
|
||||
try? scanDeclarations(&sub, depth: 1, terminatedByBracket: false, at: mark)
|
||||
try scanTextDeclaration(&sub, at: mark)
|
||||
try scanDeclarations(&sub, depth: 1, terminatedByBracket: false, at: mark)
|
||||
}
|
||||
|
||||
private mutating func scanDeclarations(
|
||||
mutating func scanDeclarations(
|
||||
_ reader: inout Reader,
|
||||
depth: Int,
|
||||
terminatedByBracket: Bool,
|
||||
@ -112,54 +115,16 @@ struct DTDScanner {
|
||||
try scanParameterReference(&reader, depth: depth, at: mark)
|
||||
} else if reader.matches("<![") {
|
||||
try scanConditionalSection(&reader, depth: depth, at: mark)
|
||||
} else {
|
||||
} else if character == "<" {
|
||||
try scanMarkupDeclaration(&reader)
|
||||
} else {
|
||||
// Only markup declarations, PE references, and whitespace may
|
||||
// appear between declarations.
|
||||
throw ParseError.malformedDeclaration(reader.mark)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Processes a conditional section `<![INCLUDE[ … ]]>` or `<![IGNORE[ … ]]>`.
|
||||
/// The keyword may be a parameter-entity reference, so it is parameter-expanded
|
||||
/// first. An INCLUDE section's declarations are scanned; an IGNORE section's
|
||||
/// body is discarded. Nested sections are handled by re-scanning the body.
|
||||
private mutating func scanConditionalSection(_ reader: inout Reader, depth: Int, at mark: Mark) throws {
|
||||
reader.consume("<![")
|
||||
var keywordRaw = ""
|
||||
while let character = reader.peek(), character != "[" {
|
||||
keywordRaw.append(character)
|
||||
reader.advance()
|
||||
}
|
||||
reader.consume("[")
|
||||
let body = readConditionalBody(&reader)
|
||||
if expandParameterReferences(keywordRaw).trimmingXMLWhitespace() == "INCLUDE", depth < maxDepth {
|
||||
var sub = Reader(body)
|
||||
try scanDeclarations(&sub, depth: depth + 1, terminatedByBracket: false, at: mark)
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads the body of a conditional section up to its matching `]]>`, keeping
|
||||
/// any nested `<![ … ]]>` sections intact so an INCLUDE can re-scan them.
|
||||
private mutating func readConditionalBody(_ reader: inout Reader) -> String {
|
||||
var body = ""
|
||||
var nesting = 1
|
||||
while reader.peek() != nil {
|
||||
if reader.matches("<![") {
|
||||
nesting += 1
|
||||
reader.consume("<![")
|
||||
body += "<!["
|
||||
} else if reader.matches("]]>") {
|
||||
reader.consume("]]>")
|
||||
nesting -= 1
|
||||
if nesting == 0 { break }
|
||||
body += "]]>"
|
||||
} else if let character = reader.peek() {
|
||||
body.append(character)
|
||||
reader.advance()
|
||||
}
|
||||
}
|
||||
return body
|
||||
}
|
||||
|
||||
private mutating func scanMarkupDeclaration(_ reader: inout Reader) throws {
|
||||
if reader.matches("<!ENTITY") {
|
||||
try scanEntityDeclaration(&reader)
|
||||
@ -180,10 +145,11 @@ struct DTDScanner {
|
||||
throw ParseError.reservedProcessingInstructionTarget(mark)
|
||||
}
|
||||
skip(&reader, until: "?>")
|
||||
} else if reader.matches("<!") {
|
||||
skip(&reader, until: ">")
|
||||
} else {
|
||||
reader.advance()
|
||||
// Any other construct here (an unknown or lowercase declaration
|
||||
// keyword, space after '<!', a DOCTYPE inside a subset, a bare '<')
|
||||
// is not well-formed.
|
||||
throw ParseError.malformedDeclaration(reader.mark)
|
||||
}
|
||||
}
|
||||
|
||||
@ -191,9 +157,14 @@ struct DTDScanner {
|
||||
/// entity's replacement text and scanning its declarations. Bounded by depth
|
||||
/// and the expansion budget.
|
||||
private mutating func scanParameterReference(_ reader: inout Reader, depth: Int, at mark: Mark) throws {
|
||||
let refMark = reader.mark
|
||||
reader.consume("%")
|
||||
let name = scanName(&reader)
|
||||
reader.consume(";")
|
||||
// PEReference is '%' Name ';' exactly: no space after '%' (an empty
|
||||
// name) and the semicolon must follow the name directly.
|
||||
guard !name.isEmpty, reader.consume(";") else {
|
||||
throw ParseError.invalidReference("%\(name)", refMark)
|
||||
}
|
||||
guard depth < maxDepth, let replacement = doctype.parameterEntities[name] else {
|
||||
return
|
||||
}
|
||||
@ -224,20 +195,28 @@ struct DTDScanner {
|
||||
}
|
||||
}
|
||||
|
||||
mutating func notationName(_ reader: inout Reader) -> String {
|
||||
reader.skipSpace()
|
||||
return scanName(&reader)
|
||||
}
|
||||
|
||||
/// `<!NOTATION S Name S (ExternalID | PublicID) S? '>'` (production 82):
|
||||
/// the identifier is required and nothing may follow it but whitespace.
|
||||
private mutating func scanNotationDeclaration(_ reader: inout Reader) throws {
|
||||
let mark = reader.mark
|
||||
reader.consume("<!NOTATION")
|
||||
guard reader.peek()?.isWhitespace == true else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
reader.skipSpace()
|
||||
let name = scanName(&reader)
|
||||
let name = try scanStrictName(&reader, at: mark)
|
||||
guard reader.peek()?.isWhitespace == true else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
reader.skipSpace()
|
||||
let id = try parseStrictExternalID(&reader, requireSystem: false, at: mark)
|
||||
skip(&reader, until: ">")
|
||||
if !name.isEmpty, let id, doctype.notations[name] == nil {
|
||||
guard let id = try parseStrictExternalID(&reader, requireSystem: false, at: mark) else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
reader.skipSpace()
|
||||
guard reader.consume(">") else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
if doctype.notations[name] == nil {
|
||||
doctype.notations[name] = id
|
||||
}
|
||||
}
|
||||
|
||||
57
Sources/Parsing/DoctypeScannerConditional.swift
Normal file
57
Sources/Parsing/DoctypeScannerConditional.swift
Normal file
@ -0,0 +1,57 @@
|
||||
/// Conditional sections of a DTD subset (`<![INCLUDE[ ... ]]>` /
|
||||
/// `<![IGNORE[ ... ]]>`, productions 61-65), split from the scanner body to
|
||||
/// keep it under the length caps.
|
||||
extension DTDScanner {
|
||||
/// Processes a conditional section `<![INCLUDE[ … ]]>` or `<![IGNORE[ … ]]>`.
|
||||
/// The keyword may be a parameter-entity reference, so it is parameter-expanded
|
||||
/// first; after expansion it must be exactly `INCLUDE` or `IGNORE` (case-
|
||||
/// sensitive, production 61). An INCLUDE section's declarations are scanned;
|
||||
/// an IGNORE section's body is discarded. Nested sections are handled by
|
||||
/// re-scanning the body, and an unterminated section is not well-formed.
|
||||
mutating func scanConditionalSection(_ reader: inout Reader, depth: Int, at mark: Mark) throws {
|
||||
reader.consume("<![")
|
||||
var keywordRaw = ""
|
||||
while let character = reader.peek(), character != "[" {
|
||||
keywordRaw.append(character)
|
||||
reader.advance()
|
||||
}
|
||||
guard reader.consume("[") else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
let keyword = expandParameterReferences(keywordRaw).trimmingXMLWhitespace()
|
||||
guard keyword == "INCLUDE" || keyword == "IGNORE" else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
guard let body = readConditionalBody(&reader) else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
if keyword == "INCLUDE", depth < maxDepth {
|
||||
var sub = Reader(body)
|
||||
try scanDeclarations(&sub, depth: depth + 1, terminatedByBracket: false, at: mark)
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads the body of a conditional section up to its matching `]]>`, keeping
|
||||
/// any nested `<![ … ]]>` sections intact so an INCLUDE can re-scan them.
|
||||
/// Returns nil when the input ends before the section is balanced.
|
||||
mutating func readConditionalBody(_ reader: inout Reader) -> String? {
|
||||
var body = ""
|
||||
var nesting = 1
|
||||
while reader.peek() != nil {
|
||||
if reader.matches("<![") {
|
||||
nesting += 1
|
||||
reader.consume("<![")
|
||||
body += "<!["
|
||||
} else if reader.matches("]]>") {
|
||||
reader.consume("]]>")
|
||||
nesting -= 1
|
||||
if nesting == 0 { return body }
|
||||
body += "]]>"
|
||||
} else if let character = reader.peek() {
|
||||
body.append(character)
|
||||
reader.advance()
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
@ -17,10 +17,7 @@ extension DTDScanner {
|
||||
}
|
||||
reader.skipSpace()
|
||||
}
|
||||
let name = scanName(&reader)
|
||||
guard !name.isEmpty else {
|
||||
throw ParseError.invalidEntityDeclaration(mark)
|
||||
}
|
||||
let name = try scanStrictName(&reader, at: mark)
|
||||
guard reader.peek()?.isWhitespace == true else {
|
||||
throw ParseError.invalidEntityDeclaration(mark)
|
||||
}
|
||||
@ -56,11 +53,13 @@ extension DTDScanner {
|
||||
if reader.peek()?.isWhitespace == true {
|
||||
reader.skipSpace()
|
||||
if reader.consume("NDATA") {
|
||||
// NDATA is for unparsed general entities only.
|
||||
guard !isParameter else {
|
||||
// NDATA is for unparsed general entities only, and `S Name`
|
||||
// must follow (production 76).
|
||||
guard !isParameter, reader.peek()?.isWhitespace == true else {
|
||||
throw ParseError.invalidEntityDeclaration(mark)
|
||||
}
|
||||
notation = notationName(&reader)
|
||||
reader.skipSpace()
|
||||
notation = try scanStrictName(&reader, at: mark)
|
||||
}
|
||||
} else if reader.peek() != ">" {
|
||||
throw ParseError.invalidEntityDeclaration(mark)
|
||||
@ -74,8 +73,9 @@ extension DTDScanner {
|
||||
}
|
||||
|
||||
/// Validates an entity value literal: every character must be an XML Char,
|
||||
/// and every `&` must begin a complete reference (`&name;` or a character
|
||||
/// reference); a bare ampersand is not well-formed.
|
||||
/// every `&` must begin a complete reference (`&name;` or a character
|
||||
/// reference), and every `%` must begin a complete parameter-entity
|
||||
/// reference (production 9 admits no bare `%` or `&` in an EntityValue).
|
||||
func validateEntityLiteral(_ value: String, at mark: Mark) throws {
|
||||
let characters = Array(value)
|
||||
var index = 0
|
||||
@ -84,15 +84,16 @@ extension DTDScanner {
|
||||
guard character.unicodeScalars.allSatisfy(PureXML.Parsing.XMLCharacter.isChar) else {
|
||||
throw ParseError.invalidCharacter(mark)
|
||||
}
|
||||
if character == "&" {
|
||||
if character == "&" || character == "%" {
|
||||
var probe = index + 1
|
||||
if probe < characters.count, characters[probe] == "#" { probe += 1 }
|
||||
if character == "&", probe < characters.count, characters[probe] == "#" { probe += 1 }
|
||||
let bodyStart = probe
|
||||
while probe < characters.count, characters[probe] != ";", !characters[probe].isWhitespace, characters[probe] != "&" {
|
||||
let stops: Set<Character> = [";", "&", "%"]
|
||||
while probe < characters.count, !stops.contains(characters[probe]), !characters[probe].isWhitespace {
|
||||
probe += 1
|
||||
}
|
||||
guard probe > bodyStart, probe < characters.count, characters[probe] == ";" else {
|
||||
throw ParseError.invalidReference("&", mark)
|
||||
throw ParseError.invalidReference(String(character), mark)
|
||||
}
|
||||
index = probe
|
||||
}
|
||||
@ -100,6 +101,42 @@ extension DTDScanner {
|
||||
}
|
||||
}
|
||||
|
||||
/// Scans a Name strictly: the first character must satisfy NameStartChar
|
||||
/// and the rest NameChar, so `-ge` or `.pe` is rejected where the lenient
|
||||
/// continuation-character scan would accept it.
|
||||
mutating func scanStrictName(_ reader: inout Reader, at mark: Mark) throws -> String {
|
||||
guard let first = reader.peek(), first.unicodeScalars.allSatisfy(PureXML.Parsing.XMLCharacter.isNameStart) else {
|
||||
throw ParseError.expectedName(mark)
|
||||
}
|
||||
return scanName(&reader)
|
||||
}
|
||||
|
||||
/// Consumes the external subset's optional text declaration
|
||||
/// `<?xml VersionInfo? EncodingDecl S? ?>` (production 77): unlike the
|
||||
/// document's XML declaration, the version is optional, the encoding is
|
||||
/// required, and `standalone` is not allowed.
|
||||
mutating func scanTextDeclaration(_ reader: inout Reader, at mark: Mark) throws {
|
||||
// Reader copies share their pull source, so look ahead in place: only
|
||||
// '<?xml' followed by whitespace opens a text declaration ('<?xml-…'
|
||||
// is an ordinary PI target and '<?xml?>' falls through to the
|
||||
// reserved-target check).
|
||||
guard reader.matches("<?xml"), reader.peek(5)?.isWhitespace == true else {
|
||||
return
|
||||
}
|
||||
reader.consume("<?xml")
|
||||
var text = ""
|
||||
while !reader.matches("?>") {
|
||||
guard let character = reader.advance() else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
text.append(character)
|
||||
}
|
||||
reader.consume("?>")
|
||||
guard PureXML.Parsing.XMLDeclaration.parseTextDeclaration(text) != nil else {
|
||||
throw ParseError.malformedDeclaration(mark)
|
||||
}
|
||||
}
|
||||
|
||||
/// Parses an external identifier strictly: `SYSTEM S literal`, or
|
||||
/// `PUBLIC S pubid-literal S system-literal` (the system literal is required
|
||||
/// when `requireSystem` is true, as for entities and the DOCTYPE itself).
|
||||
|
||||
@ -227,6 +227,11 @@ public extension PureXML.Parsing {
|
||||
let mark = reader.mark
|
||||
reader.consume("<?")
|
||||
let target = try scanName()
|
||||
// Production 16: the target and the data must be separated by
|
||||
// whitespace, so `<?target+++?>` is not well-formed.
|
||||
if let next = reader.peek(), !next.isWhitespace, !reader.matches("?>") {
|
||||
throw ParseError.unexpectedCharacter(next, reader.mark)
|
||||
}
|
||||
reader.skipSpace()
|
||||
var data = ""
|
||||
var length = 0
|
||||
|
||||
@ -41,6 +41,26 @@ public extension PureXML.Parsing {
|
||||
return declaration
|
||||
}
|
||||
|
||||
/// Parses an external entity's text declaration (production 77):
|
||||
/// `VersionInfo? EncodingDecl S?`. The version is optional, the
|
||||
/// encoding is required, and `standalone` is not allowed.
|
||||
static func parseTextDeclaration(_ data: String) -> XMLDeclaration? {
|
||||
var scanner = PseudoAttributeScanner(data)
|
||||
guard let pairs = scanner.scan(), !pairs.isEmpty else { return nil }
|
||||
var remaining = pairs
|
||||
var declaration = XMLDeclaration()
|
||||
if remaining.first?.name == "version" {
|
||||
guard isValidVersion(remaining[0].value) else { return nil }
|
||||
declaration.version = remaining[0].value
|
||||
remaining.removeFirst()
|
||||
}
|
||||
guard remaining.count == 1, remaining[0].name == "encoding",
|
||||
isValidEncodingName(remaining[0].value)
|
||||
else { return nil }
|
||||
declaration.encoding = remaining[0].value
|
||||
return declaration
|
||||
}
|
||||
|
||||
/// Whether the pseudo-attribute names appear in the grammar's fixed order
|
||||
/// with no repeats: version, then encoding, then standalone.
|
||||
private static func isOrdered(_ names: [String]) -> Bool {
|
||||
|
||||
113
Tests/DTDSubsetStrictnessTests.swift
Normal file
113
Tests/DTDSubsetStrictnessTests.swift
Normal file
@ -0,0 +1,113 @@
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
/// The DTD-subset strictness landed with the OASIS/NIST conformance pass:
|
||||
/// unknown or malformed markup declarations, the exact `%name;` parameter-
|
||||
/// reference grammar, strict declaration names, NDATA rules, notation tails,
|
||||
/// conditional-section keywords and balance, external-subset error
|
||||
/// propagation with text declarations, and PI target separation.
|
||||
@Suite("DTD subset strictness")
|
||||
struct DTDSubsetStrictnessTests {
|
||||
private func limits() -> PureXML.Parsing.Limits {
|
||||
PureXML.Parsing.Limits(allowDoctype: true)
|
||||
}
|
||||
|
||||
private func parses(_ xml: String) -> Bool {
|
||||
(try? PureXML.parse(xml, limits: limits())) != nil
|
||||
}
|
||||
|
||||
private func parses(_ xml: String, externalSubset: String) -> Bool {
|
||||
let resolver = PureXML.Parsing.EntityResolver(
|
||||
resolveEntity: { _, _ in nil },
|
||||
resolveExternalSubset: { _ in externalSubset },
|
||||
)
|
||||
return (try? PureXML.parse(xml, limits: limits(), resolver: resolver)) != nil
|
||||
}
|
||||
|
||||
private let externalHost = "<!DOCTYPE doc SYSTEM \"sub.dtd\">\n<doc/>"
|
||||
|
||||
@Test("Unknown, lowercase, or space-separated declaration keywords are rejected")
|
||||
func test_declarationKeywords() {
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY><!DUNNO junk>]>\n<doc/>"))
|
||||
#expect(!parses("<!DOCTYPE doc [<!element doc EMPTY>]>\n<doc/>"))
|
||||
#expect(!parses("<!DOCTYPE doc [<! ENTITY ge \"v\">]>\n<doc/>"))
|
||||
#expect(parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY>]>\n<doc/>"))
|
||||
}
|
||||
|
||||
@Test("A parameter-entity reference is '%' Name ';' exactly")
|
||||
func test_parameterReferenceGrammar() {
|
||||
let declare = "<!ELEMENT doc EMPTY><!ENTITY % pe \"<!---->\">"
|
||||
#expect(!parses("<!DOCTYPE doc [\(declare)%pe<!---->]>\n<doc/>"))
|
||||
#expect(!parses("<!DOCTYPE doc [\(declare)% pe;]>\n<doc/>"))
|
||||
#expect(!parses("<!DOCTYPE doc [\(declare)%pe ;]>\n<doc/>"))
|
||||
#expect(parses("<!DOCTYPE doc [\(declare)%pe;]>\n<doc/>"))
|
||||
}
|
||||
|
||||
@Test("Entity and parameter-entity names must start with a name-start character")
|
||||
func test_strictEntityNames() {
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY><!ENTITY -ge \"v\">]>\n<doc/>"))
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY><!ENTITY % .pe \"v\">]>\n<doc/>"))
|
||||
#expect(parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY><!ENTITY ge \"v\">]>\n<doc/>"))
|
||||
}
|
||||
|
||||
@Test("NDATA requires whitespace and a strict notation name")
|
||||
func test_ndataNames() {
|
||||
let notation = "<!NOTATION n SYSTEM \"x\">"
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY>\(notation)<!ENTITY ge SYSTEM \"e\" NDATA>]>\n<doc/>"))
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY>\(notation)<!ENTITY ge SYSTEM \"e\" NDATA -n>]>\n<doc/>"))
|
||||
#expect(parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY>\(notation)<!ENTITY ge SYSTEM \"e\" NDATA n>]>\n<doc/>"))
|
||||
}
|
||||
|
||||
@Test("A notation declaration requires its identifier and a clean tail")
|
||||
func test_notationTail() {
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY><!NOTATION n SYSTEM \"\"\">]>\n<doc/>"))
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY><!NOTATION n>]>\n<doc/>"))
|
||||
#expect(parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY><!NOTATION n PUBLIC \"p\">]>\n<doc/>"))
|
||||
}
|
||||
|
||||
@Test("An entity value literal admits no bare '%' or '&'")
|
||||
func test_entityLiteralReferences() {
|
||||
#expect(!parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><!ENTITY % e \"asdf%\">"))
|
||||
#expect(!parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><!ENTITY % e \"asdf&\">"))
|
||||
#expect(parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><!ENTITY % e \"a%pe;b\">"))
|
||||
}
|
||||
|
||||
@Test("Conditional-section keywords are case-sensitive INCLUDE or IGNORE and must balance")
|
||||
func test_conditionalSections() {
|
||||
#expect(!parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><![TEMP[<!ATTLIST doc a CDATA #IMPLIED>]]>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<![include[<!ELEMENT doc EMPTY>]]>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><![ignore[]]>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<![INCLUDE[<!ELEMENT doc EMPTY>] ]>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><![IGNORE[<![]]>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><![IGNORE[ x ]]>]]>"))
|
||||
#expect(parses(externalHost, externalSubset: "<![INCLUDE[<!ELEMENT doc EMPTY>]]>"))
|
||||
#expect(parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY><![IGNORE[ <![ x ]]> ]]>"))
|
||||
}
|
||||
|
||||
@Test("External-subset grammar violations reject the document")
|
||||
func test_externalSubsetPropagates() {
|
||||
#expect(!parses(externalHost, externalSubset: "<!DOCTYPE doc [<!ELEMENT doc EMPTY>]>"))
|
||||
#expect(parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY>"))
|
||||
}
|
||||
|
||||
@Test("A text declaration allows version and requires encoding, never standalone")
|
||||
func test_textDeclaration() {
|
||||
#expect(parses(externalHost, externalSubset: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!ELEMENT doc EMPTY>"))
|
||||
#expect(parses(externalHost, externalSubset: "<?xml encoding=\"UTF-8\"?>\n<!ELEMENT doc EMPTY>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<!ELEMENT doc EMPTY>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<?xml version=\"1.0\"?>\n<!ELEMENT doc EMPTY>"))
|
||||
}
|
||||
|
||||
@Test("A processing-instruction target must be separated from its data")
|
||||
func test_processingInstructionSeparation() {
|
||||
#expect(!parses("<?pitarget+++?>\n<doc/>"))
|
||||
#expect(parses("<?pitarget +++?>\n<doc/>"))
|
||||
#expect(parses("<?pitarget?>\n<doc/>"))
|
||||
}
|
||||
|
||||
@Test("The internal subset admits only declarations, PE references, and whitespace")
|
||||
func test_subsetJunk() {
|
||||
#expect(!parses("<!DOCTYPE doc [<!ELEMENT doc EMPTY> junk ]>\n<doc/>"))
|
||||
#expect(!parses(externalHost, externalSubset: "<!ELEMENT doc EMPTY> ]]>"))
|
||||
}
|
||||
}
|
||||
113
Tests/W3COasisSuiteTests.swift
Normal file
113
Tests/W3COasisSuiteTests.swift
Normal file
@ -0,0 +1,113 @@
|
||||
import Foundation
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
/// Opt-in runner for the W3C XML Conformance Test Suite's OASIS/NIST section.
|
||||
/// The suite's license permits redistribution only as the unmodified archive,
|
||||
/// so it is never vendored into this repository: download `xmlts` from w3.org,
|
||||
/// extract it, and point `XMLCONF_ROOT` at the extracted `xmlconf` directory
|
||||
/// to run. Without the variable the suite is skipped. The section's own
|
||||
/// manifest (`oasis/oasis.xml`) is parsed with PureXML itself and drives the
|
||||
/// case classification: `valid` and `invalid` documents must parse (invalid
|
||||
/// means well-formed but DTD-invalid, which a parser must accept), `not-wf`
|
||||
/// documents must be rejected, and the single `error` case is optional
|
||||
/// behavior by definition and is skipped. Foundation is used here for file
|
||||
/// access only; the library target remains Foundation-free.
|
||||
@Suite("W3C oasis conformance (opt-in via XMLCONF_ROOT)")
|
||||
struct W3COasisSuiteTests {
|
||||
private var root: String? {
|
||||
ProcessInfo.processInfo.environment["XMLCONF_ROOT"]
|
||||
}
|
||||
|
||||
/// Cases the suite marks not-well-formed that this implementation
|
||||
/// knowingly accepts. The baseline is exact: any regression in the cases
|
||||
/// that DO get rejected is still caught.
|
||||
private let knownNotWFAccepted: Set<String> = []
|
||||
|
||||
/// `valid`/`invalid` cases this implementation knowingly rejects.
|
||||
private let knownPassRejected: Set<String> = []
|
||||
|
||||
private struct ManifestCase {
|
||||
let id: String
|
||||
let type: String
|
||||
let uri: String
|
||||
let namespaceAware: Bool
|
||||
}
|
||||
|
||||
private func manifest(in directory: String) throws -> [ManifestCase] {
|
||||
let source = try String(contentsOfFile: directory + "/oasis.xml", encoding: .utf8)
|
||||
let document = try PureXML.parse(source)
|
||||
return try PureXML.xpath("//TEST", over: document).compactMap { selection in
|
||||
guard let element = selection.element else { return nil }
|
||||
func attribute(_ name: String) -> String? {
|
||||
element.attributes.first { $0.name.description == name }?.value
|
||||
}
|
||||
guard let id = attribute("ID"), let type = attribute("TYPE"), let uri = attribute("URI") else {
|
||||
return nil
|
||||
}
|
||||
return ManifestCase(id: id, type: type, uri: uri, namespaceAware: attribute("NAMESPACE") != "no")
|
||||
}
|
||||
}
|
||||
|
||||
private func limits() -> PureXML.Parsing.Limits {
|
||||
PureXML.Parsing.Limits(allowDoctype: true)
|
||||
}
|
||||
|
||||
/// Resolves relative SYSTEM identifiers against the case's directory, so
|
||||
/// cases with external DTD files load their siblings.
|
||||
private func resolver(directory: String) -> PureXML.Parsing.EntityResolver {
|
||||
PureXML.Parsing.EntityResolver(
|
||||
resolveEntity: { _, id in
|
||||
(try? String(contentsOfFile: directory + "/" + id.systemID, encoding: .utf8))
|
||||
},
|
||||
resolveExternalSubset: { id in
|
||||
(try? String(contentsOfFile: directory + "/" + id.systemID, encoding: .utf8))
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
/// Reads a case's raw bytes and decodes through PureXML's own byte
|
||||
/// decoder (a decode failure counts as rejection for not-wf cases).
|
||||
private func parse(file: String, in directory: String) throws -> PureXML.Model.Node {
|
||||
let data = try Data(contentsOf: URL(fileURLWithPath: directory + "/" + file))
|
||||
let source = try PureXML.Parsing.ByteDecoder.decode([UInt8](data))
|
||||
return try PureXML.parse(source, limits: limits(), resolver: resolver(directory: directory))
|
||||
}
|
||||
|
||||
@Test("Every oasis not-wf case is rejected by the strict parser")
|
||||
func test_notWellFormedRejected() throws {
|
||||
guard let root else {
|
||||
return // Opt-in: suite not present.
|
||||
}
|
||||
let directory = root + "/oasis"
|
||||
var unexpectedlyAccepted: [String] = []
|
||||
for testCase in try manifest(in: directory) where testCase.type == "not-wf" {
|
||||
guard !knownNotWFAccepted.contains(testCase.uri) else { continue }
|
||||
if (try? parse(file: testCase.uri, in: directory)) != nil {
|
||||
unexpectedlyAccepted.append(testCase.uri)
|
||||
}
|
||||
}
|
||||
#expect(unexpectedlyAccepted.isEmpty, "accepted \(unexpectedlyAccepted.count) not-wf cases: \(unexpectedlyAccepted)")
|
||||
}
|
||||
|
||||
@Test("Every oasis valid and invalid case parses (both are well-formed)")
|
||||
func test_wellFormedParses() throws {
|
||||
guard let root else {
|
||||
return // Opt-in: suite not present.
|
||||
}
|
||||
let directory = root + "/oasis"
|
||||
var failures: [String: String] = [:]
|
||||
for testCase in try manifest(in: directory) where testCase.type == "valid" || testCase.type == "invalid" {
|
||||
// NAMESPACE='no' cases use pre-namespace names (colons in arbitrary
|
||||
// positions); PureXML is namespace-aware, like libxml2's default
|
||||
// mode, so the manifest's own flag excludes them.
|
||||
guard testCase.namespaceAware, !knownPassRejected.contains(testCase.uri) else { continue }
|
||||
do {
|
||||
_ = try parse(file: testCase.uri, in: directory)
|
||||
} catch {
|
||||
failures[testCase.uri] = String(describing: error)
|
||||
}
|
||||
}
|
||||
#expect(failures.isEmpty, "\(failures.count) well-formed cases failed: \(failures)")
|
||||
}
|
||||
}
|
||||
@ -15,16 +15,16 @@ struct W3CXMLSuiteTests {
|
||||
ProcessInfo.processInfo.environment["XMLTS_ROOT"]
|
||||
}
|
||||
|
||||
/// The six cases this implementation knowingly accepts although the suite
|
||||
/// marks them not-well-formed, all deliberate: 063 and 107 (conditional and
|
||||
/// CDATA sections in the internal subset) and 160-162 (parameter-entity
|
||||
/// references in the internal subset) are extensions this package supports
|
||||
/// as features, and 141 expects the 1998 suite's name-character classes
|
||||
/// where this package implements XML 1.0 Fifth Edition's (U+0E5C is a name
|
||||
/// character there). The baseline is exact, so any regression in the 180
|
||||
/// cases that DO pass is still caught.
|
||||
/// The five cases this implementation knowingly accepts although the suite
|
||||
/// marks them not-well-formed, all deliberate: 063 (conditional sections in
|
||||
/// the internal subset) and 160-162 (parameter-entity references in the
|
||||
/// internal subset) are extensions this package supports as features, and
|
||||
/// 141 expects the 1998 suite's name-character classes where this package
|
||||
/// implements XML 1.0 Fifth Edition's (U+0E5C is a name character there).
|
||||
/// The baseline is exact, so any regression in the 181 cases that DO pass
|
||||
/// is still caught.
|
||||
private let knownNotWFDeviations: Set<String> = [
|
||||
"063.xml", "107.xml", "141.xml", "160.xml", "161.xml", "162.xml",
|
||||
"063.xml", "141.xml", "160.xml", "161.xml", "162.xml",
|
||||
]
|
||||
|
||||
/// valid/sa/114.xml: a CDATA section inside an entity value must protect
|
||||
|
||||
Loading…
Reference in New Issue
Block a user