feat(validation): streaming DTD ID/IDREF integrity
Complete the DTD streaming validator with document-scoped ID/IDREF checks, held in bounded sets (the identifier values, not the tree): a duplicate ID is reported at its repeat occurrence, and forward IDREFs resolve at the end of the stream via finish(). The streaming-vs-tree agreement test now covers a duplicate ID as well. DTD streaming validation (content, attributes, ID/IDREF) is now complete; RELAX NG and XSD streaming remain. Part of #107, #105
This commit is contained in:
parent
b5fa216a7c
commit
f9f9985aac
@ -23,7 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Added
|
||||
|
||||
- Streaming (reader-driven) validation (#107, in progress): `PureXML.validate(streaming:dtd:)` and `PureXML.Validation.StreamingDTDValidator` validate a document against a DTD while it is pulled event by event (the libxml2 `xmlTextReader` validation model), holding only the open-element stack rather than the whole tree. Each element's content model and attributes are checked the moment it closes, by applying the exact per-element DTD rules to a shallow element synthesized from the streamed child names, so streaming and tree validation report the same problems. Errors carry a libxml2 `xmlGetNodePath`-style coding path. RELAX NG / XSD streaming and document-scoped ID/IDREF streaming remain on the issue.
|
||||
- Streaming (reader-driven) DTD validation (#107, in progress): `PureXML.validate(streaming:dtd:)` and `PureXML.Validation.StreamingDTDValidator` validate a document against a DTD while it is pulled event by event (the libxml2 `xmlTextReader` validation model), holding only the open-element stack rather than the whole tree. Each element's content model and attributes are checked the moment it closes, by applying the exact per-element DTD rules to a shallow element synthesized from the streamed child names. Document-scoped ID/IDREF integrity is tracked in bounded sets (the identifier values, not the tree): a duplicate ID is reported at its repeat occurrence, and forward IDREFs resolve at the end of the stream. Streaming and tree validation report the same problems (a test asserts matching reasons), and errors carry a libxml2 `xmlGetNodePath`-style coding path. RELAX NG and XSD streaming remain on the issue.
|
||||
- Output encoding (#106): `Serializer.serialize(_:encoding:)` and `PureXML.serialize(_:encoding:)` now emit bytes in a chosen encoding (the libxml2 `xmlSaveToFilename(..., encoding)` model), not only UTF-8. A new `PureXML.Parsing.ByteEncoder` covers the Unicode transformation formats, the single-byte legacy families (ISO-8859, Windows code pages, KOI8), and the multi-byte CJK encodings (Shift-JIS, EUC-JP, EUC-KR, GBK, GB18030, Big5). The CJK and single-byte inverses are built by running the existing, tested decoders over every valid byte sequence and recording the shortest sequence per scalar, so each encoder is by construction the exact inverse of its decoder (no new data, no separately maintained encode tables); GB18030's astral range uses the four-byte pointer formula. The declaration carries the encoding's canonical name, a UTF-16/32 stream is preceded by a byte-order mark, and any scalar the target encoding cannot represent is written as a decimal numeric character reference, so output always round-trips.
|
||||
- Big5 (#103): the byte decoder gains the WHATWG `big5` algorithm, for both the whole-buffer and streaming decoders, selected by the `big5` declaration name (and the `big5-hkscs` / `cn-big5` aliases). A lead in `0x81`-`0xFE` followed by a trail in `0x40`-`0x7E` or `0xA1`-`0xFE` resolves through the index, covering Traditional Chinese and the HKSCS extensions including the astral CJK ideographs (Extension B and the Compatibility Supplement). Four pointers decode to a base letter plus a combining mark (a two-scalar sequence). The 18,590-entry index is vendored verbatim from the WHATWG `index-big5.txt`. Big5 completes the major CJK families libxml2 reaches through iconv (Japanese, Korean, Simplified and Traditional Chinese).
|
||||
- Full GB18030 (#102): the byte decoder gains the GB18030 four-byte sequences on top of the GBK two-byte subset, for both the whole-buffer and streaming decoders, selected by the `gb18030` declaration name. A lead in `0x81`-`0xFE` followed by a digit opens a four-byte form whose pointer is mapped through the WHATWG `index-gb18030-ranges` linear ranges, reaching the supplementary planes (the `189000` range maps to `U+10000` and beyond). The 207 ranges are vendored verbatim from the WHATWG `index-gb18030-ranges.txt`. GB18030 is China's mandatory standard, supported in libxml2 through iconv.
|
||||
|
||||
@ -99,7 +99,7 @@ flowchart TB
|
||||
|
||||
PureXML is a working, dependency-free XML library today: parse, emit, validate,
|
||||
query, and stream documents on macOS, Linux, Windows, and WASM. The test suite
|
||||
currently runs **781 tests in 119 suites** (`swift test`).
|
||||
currently runs **782 tests in 119 suites** (`swift test`).
|
||||
|
||||
### Shipped (libxml2-aligned surface)
|
||||
|
||||
|
||||
@ -16,6 +16,11 @@ public extension PureXML.Validation {
|
||||
private let rules: [Validation<PureXML.Model.Element, DTDSchema>]
|
||||
private var stack: [StreamingDTDFrame] = []
|
||||
private var collected: [ValidationError] = []
|
||||
// Document-scoped ID/IDREF state, bounded to the identifier values, not
|
||||
// the tree. IDREFs may point forward, so they resolve at finish().
|
||||
private var idCounts: [String: Int] = [:]
|
||||
private var idDuplicatePath: [String: [PathKey]] = [:]
|
||||
private var pendingReferences: [StreamingIDReference] = []
|
||||
|
||||
public init(schema: DTDSchema, strict: Bool = false) {
|
||||
self.schema = schema
|
||||
@ -47,11 +52,32 @@ public extension PureXML.Validation {
|
||||
}
|
||||
}
|
||||
|
||||
/// The errors found so far. Call after the last event for the full set.
|
||||
/// The per-element errors found so far (content models and attributes).
|
||||
/// Document-scoped ID/IDREF errors are added by ``finish()``.
|
||||
public var errors: [ValidationError] {
|
||||
collected
|
||||
}
|
||||
|
||||
/// The complete error set, including the document-scoped ID/IDREF checks
|
||||
/// that can only be resolved once every element has been seen. Call after
|
||||
/// the last event.
|
||||
public mutating func finish() -> [ValidationError] {
|
||||
var result = collected
|
||||
for (value, count) in idCounts where count > 1 {
|
||||
result.append(ValidationError(
|
||||
reason: "duplicate ID '\(value)' (declared \(count) times)",
|
||||
at: idDuplicatePath[value] ?? [],
|
||||
))
|
||||
}
|
||||
for reference in pendingReferences where idCounts[reference.value] == nil {
|
||||
result.append(ValidationError(
|
||||
reason: "IDREF '\(reference.value)' on <\(reference.element)> matches no ID",
|
||||
at: reference.path,
|
||||
))
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// MARK: Internals
|
||||
|
||||
/// The coding path for an opening element, recording it as a child of the
|
||||
@ -75,6 +101,34 @@ public extension PureXML.Validation {
|
||||
+ (frame.hasText ? [.text("x")] : [])
|
||||
let element = PureXML.Model.Element(name: frame.name, attributes: frame.attributes, children: children)
|
||||
collected += rules.flatMap { $0.apply(to: element, at: frame.path, in: schema) }
|
||||
collectIdentifiers(frame)
|
||||
}
|
||||
|
||||
/// Records the closing element's ID and IDREF attribute values, so the
|
||||
/// document-scoped ID/IDREF integrity check can run at ``finish()``. A
|
||||
/// duplicate ID is located at its repeat occurrence; an IDREF at the
|
||||
/// element that carries it.
|
||||
private mutating func collectIdentifiers(_ frame: StreamingDTDFrame) {
|
||||
let name = frame.name.description
|
||||
guard let declarations = schema.attributes[name] else { return }
|
||||
for declaration in declarations {
|
||||
guard let value = frame.attributes.first(where: {
|
||||
$0.name.description == declaration.name || $0.name.localName == declaration.name
|
||||
})?.value else { continue }
|
||||
switch declaration.type {
|
||||
case .id:
|
||||
idCounts[value, default: 0] += 1
|
||||
if idCounts[value] == 2 { idDuplicatePath[value] = frame.path }
|
||||
case .idReference:
|
||||
pendingReferences.append(StreamingIDReference(value: value, element: name, path: frame.path))
|
||||
case .idReferences:
|
||||
for token in value.split(whereSeparator: { $0.isWhitespace }) {
|
||||
pendingReferences.append(StreamingIDReference(value: String(token), element: name, path: frame.path))
|
||||
}
|
||||
case .cdata, .enumeration, .notation, .nmToken, .nmTokens, .entity, .entities:
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -94,10 +148,18 @@ public extension PureXML {
|
||||
while let event = try reader.next() {
|
||||
validator.consume(event)
|
||||
}
|
||||
return validator.errors
|
||||
return validator.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// A pending IDREF awaiting resolution at the end of the stream: the referenced
|
||||
/// value, the element that carries it, and that element's coding path.
|
||||
private struct StreamingIDReference {
|
||||
let value: String
|
||||
let element: String
|
||||
let path: [PureXML.Validation.PathKey]
|
||||
}
|
||||
|
||||
/// One open element while streaming: its name and attributes, the coding path to
|
||||
/// it, and the direct-child names and text flag accumulated until it closes.
|
||||
private struct StreamingDTDFrame {
|
||||
|
||||
@ -33,15 +33,31 @@ struct StreamingDTDValidationTests {
|
||||
#expect(errors.contains { $0.reason.contains("id") })
|
||||
}
|
||||
|
||||
@Test("A duplicate ID is reported and a forward IDREF resolves at finish")
|
||||
func test_idAndIdref() throws {
|
||||
let dtd = try schema(
|
||||
"<!ELEMENT root (a*)><!ELEMENT a EMPTY><!ATTLIST a id ID #IMPLIED ref IDREF #IMPLIED>",
|
||||
)
|
||||
// A duplicate ID.
|
||||
let dupes = try PureXML.validate(streaming: "<root><a id=\"x\"/><a id=\"x\"/></root>", dtd: dtd)
|
||||
#expect(dupes.contains { $0.reason.contains("duplicate ID 'x'") })
|
||||
// A forward IDREF (the id appears after the ref) resolves with no error.
|
||||
#expect(try PureXML.validate(streaming: "<root><a ref=\"y\"/><a id=\"y\"/></root>", dtd: dtd).isEmpty)
|
||||
// A dangling IDREF is reported.
|
||||
let dangling = try PureXML.validate(streaming: "<root><a ref=\"z\"/></root>", dtd: dtd)
|
||||
#expect(dangling.contains { $0.reason.contains("IDREF 'z'") })
|
||||
}
|
||||
|
||||
@Test("Streaming validation agrees with tree validation on the same document")
|
||||
func test_matchesTreeValidation() throws {
|
||||
let internalSubset = "<!ELEMENT list (item+)><!ELEMENT item (#PCDATA)><!ATTLIST item id ID #IMPLIED>"
|
||||
let doc = """
|
||||
<!DOCTYPE list [<!ELEMENT list (item+)><!ELEMENT item (#PCDATA)>]>
|
||||
<list><item>a</item><stray/><item>b</item></list>
|
||||
<!DOCTYPE list [\(internalSubset)]>
|
||||
<list><item id="d">a</item><stray/><item id="d">b</item></list>
|
||||
"""
|
||||
let tree = try PureXML.validateAgainstInternalDTD(doc)
|
||||
let dtd = try schema("<!ELEMENT list (item+)><!ELEMENT item (#PCDATA)>")
|
||||
let body = "<list><item>a</item><stray/><item>b</item></list>"
|
||||
let dtd = try schema(internalSubset)
|
||||
let body = "<list><item id=\"d\">a</item><stray/><item id=\"d\">b</item></list>"
|
||||
let streamed = try PureXML.validate(streaming: body, dtd: dtd)
|
||||
// Same problems surface (the reasons match; the streamed path always indexes).
|
||||
#expect(streamed.map(\.reason).sorted() == tree.map(\.reason).sorted())
|
||||
|
||||
Loading…
Reference in New Issue
Block a user