feat(validation): streaming RELAX NG validation
Add RelaxNG.validate(streaming:), driving the Brzozowski derivative directly from the parse-event stream (the libxml2 xmlTextReader model). The derivative is already incremental: the after-threaded pattern chain is the validation stack, so only the residual pattern and a light per-element frame (for the childrenDeriv ignorable-whitespace rule) are retained, never the tree. A differential test asserts streaming agrees with the trusted tree engine across valid, invalid, whitespace, and mixed-content documents. XSD streaming remains. Part of #107, #105
This commit is contained in:
parent
f9f9985aac
commit
051ef02fdb
@ -23,7 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Added
|
||||
|
||||
- Streaming (reader-driven) DTD validation (#107, in progress): `PureXML.validate(streaming:dtd:)` and `PureXML.Validation.StreamingDTDValidator` validate a document against a DTD while it is pulled event by event (the libxml2 `xmlTextReader` validation model), holding only the open-element stack rather than the whole tree. Each element's content model and attributes are checked the moment it closes, by applying the exact per-element DTD rules to a shallow element synthesized from the streamed child names. Document-scoped ID/IDREF integrity is tracked in bounded sets (the identifier values, not the tree): a duplicate ID is reported at its repeat occurrence, and forward IDREFs resolve at the end of the stream. Streaming and tree validation report the same problems (a test asserts matching reasons), and errors carry a libxml2 `xmlGetNodePath`-style coding path. RELAX NG and XSD streaming remain on the issue.
|
||||
- Streaming (reader-driven) DTD validation (#107, in progress): `PureXML.validate(streaming:dtd:)` and `PureXML.Validation.StreamingDTDValidator` validate a document against a DTD while it is pulled event by event (the libxml2 `xmlTextReader` validation model), holding only the open-element stack rather than the whole tree. Each element's content model and attributes are checked the moment it closes, by applying the exact per-element DTD rules to a shallow element synthesized from the streamed child names. Document-scoped ID/IDREF integrity is tracked in bounded sets (the identifier values, not the tree): a duplicate ID is reported at its repeat occurrence, and forward IDREFs resolve at the end of the stream. Streaming and tree validation report the same problems (a test asserts matching reasons), and errors carry a libxml2 `xmlGetNodePath`-style coding path. RELAX NG streaming is also available via `RelaxNG.validate(streaming:)`: the Brzozowski derivative is itself incremental, so it is driven directly from the event stream, retaining only the residual pattern (the `after`-threaded chain is the validation stack) and a light per-element frame for ignorable-whitespace handling; a differential test asserts it agrees with the tree engine across valid, invalid, whitespace, and mixed-content documents. XSD streaming remains on the issue.
|
||||
- Output encoding (#106): `Serializer.serialize(_:encoding:)` and `PureXML.serialize(_:encoding:)` now emit bytes in a chosen encoding (the libxml2 `xmlSaveToFilename(..., encoding)` model), not only UTF-8. A new `PureXML.Parsing.ByteEncoder` covers the Unicode transformation formats, the single-byte legacy families (ISO-8859, Windows code pages, KOI8), and the multi-byte CJK encodings (Shift-JIS, EUC-JP, EUC-KR, GBK, GB18030, Big5). The CJK and single-byte inverses are built by running the existing, tested decoders over every valid byte sequence and recording the shortest sequence per scalar, so each encoder is by construction the exact inverse of its decoder (no new data, no separately maintained encode tables); GB18030's astral range uses the four-byte pointer formula. The declaration carries the encoding's canonical name, a UTF-16/32 stream is preceded by a byte-order mark, and any scalar the target encoding cannot represent is written as a decimal numeric character reference, so output always round-trips.
|
||||
- Big5 (#103): the byte decoder gains the WHATWG `big5` algorithm, for both the whole-buffer and streaming decoders, selected by the `big5` declaration name (and the `big5-hkscs` / `cn-big5` aliases). A lead in `0x81`-`0xFE` followed by a trail in `0x40`-`0x7E` or `0xA1`-`0xFE` resolves through the index, covering Traditional Chinese and the HKSCS extensions including the astral CJK ideographs (Extension B and the Compatibility Supplement). Four pointers decode to a base letter plus a combining mark (a two-scalar sequence). The 18,590-entry index is vendored verbatim from the WHATWG `index-big5.txt`. Big5 completes the major CJK families libxml2 reaches through iconv (Japanese, Korean, Simplified and Traditional Chinese).
|
||||
- Full GB18030 (#102): the byte decoder gains the GB18030 four-byte sequences on top of the GBK two-byte subset, for both the whole-buffer and streaming decoders, selected by the `gb18030` declaration name. A lead in `0x81`-`0xFE` followed by a digit opens a four-byte form whose pointer is mapped through the WHATWG `index-gb18030-ranges` linear ranges, reaching the supplementary planes (the `189000` range maps to `U+10000` and beyond). The 207 ranges are vendored verbatim from the WHATWG `index-gb18030-ranges.txt`. GB18030 is China's mandatory standard, supported in libxml2 through iconv.
|
||||
|
||||
@ -99,7 +99,7 @@ flowchart TB
|
||||
|
||||
PureXML is a working, dependency-free XML library today: parse, emit, validate,
|
||||
query, and stream documents on macOS, Linux, Windows, and WASM. The test suite
|
||||
currently runs **782 tests in 119 suites** (`swift test`).
|
||||
currently runs **785 tests in 120 suites** (`swift test`).
|
||||
|
||||
### Shipped (libxml2-aligned surface)
|
||||
|
||||
|
||||
@ -263,3 +263,72 @@ extension PureXML.Schema.RelaxNGEngine {
|
||||
return counts
|
||||
}
|
||||
}
|
||||
|
||||
/// One open element while streaming a RELAX NG derivative: whether it has had an
|
||||
/// element child yet, and the text buffered since the last structural event. The
|
||||
/// `after`-threaded pattern is the validation stack; this frame only carries the
|
||||
/// bookkeeping `childrenDeriv` needs to treat ignorable whitespace correctly.
|
||||
private struct RelaxNGStreamFrame {
|
||||
var hadElementChild = false
|
||||
var pendingText = ""
|
||||
}
|
||||
|
||||
public extension PureXML.Schema {
|
||||
/// The residual derivative plus per-element whitespace bookkeeping carried
|
||||
/// across the events of a streamed RELAX NG validation.
|
||||
struct RelaxNGStreamingState {
|
||||
var current: PureXML.Schema.Pattern
|
||||
fileprivate var frames: [RelaxNGStreamFrame]
|
||||
}
|
||||
}
|
||||
|
||||
extension PureXML.Schema.RelaxNGEngine {
|
||||
/// The initial streaming state for a start pattern, with a document-level frame.
|
||||
func streamingStart(_ start: PureXML.Schema.Pattern) -> PureXML.Schema.RelaxNGStreamingState {
|
||||
PureXML.Schema.RelaxNGStreamingState(current: start, frames: [RelaxNGStreamFrame()])
|
||||
}
|
||||
|
||||
/// Derives the state by one parse event, the streaming form of `childDeriv`.
|
||||
/// Text is buffered and applied at structural boundaries so ignorable
|
||||
/// whitespace is treated exactly as the tree walk's `childrenDeriv` does.
|
||||
func streamingConsume(_ event: PureXML.Parsing.Event, into state: inout PureXML.Schema.RelaxNGStreamingState) {
|
||||
switch event {
|
||||
case let .startElement(name, attributes):
|
||||
flushText(&state, closing: false)
|
||||
if !state.frames.isEmpty { state.frames[state.frames.count - 1].hadElementChild = true }
|
||||
let opened = startTagOpenDeriv(state.current, name)
|
||||
let attributed = attributes
|
||||
.filter { !Self.isNamespaceDeclaration($0) }
|
||||
.reduce(opened) { attributeDeriv($0, $1) }
|
||||
state.current = startTagCloseDeriv(attributed)
|
||||
state.frames.append(RelaxNGStreamFrame())
|
||||
case let .characters(text), let .cdata(text):
|
||||
if !state.frames.isEmpty { state.frames[state.frames.count - 1].pendingText += text }
|
||||
case .endElement:
|
||||
flushText(&state, closing: true)
|
||||
state.current = endTagDeriv(state.current)
|
||||
if !state.frames.isEmpty { state.frames.removeLast() }
|
||||
case .comment, .processingInstruction:
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the streamed document satisfied the schema (the residual is nullable).
|
||||
func streamingValid(_ state: PureXML.Schema.RelaxNGStreamingState) -> Bool {
|
||||
nullable(state.current)
|
||||
}
|
||||
|
||||
private func flushText(_ state: inout PureXML.Schema.RelaxNGStreamingState, closing: Bool) {
|
||||
guard let index = state.frames.indices.last else { return }
|
||||
let text = state.frames[index].pendingText
|
||||
guard !text.isEmpty else { return }
|
||||
state.frames[index].pendingText = ""
|
||||
if !Self.isWhitespace(text) {
|
||||
state.current = textDeriv(state.current, text)
|
||||
} else if closing, !state.frames[index].hadElementChild {
|
||||
// A lone whitespace text child is optional, the `childrenDeriv` rule.
|
||||
state.current = choice(state.current, textDeriv(state.current, text))
|
||||
}
|
||||
// Whitespace among element siblings is ignorable and skipped.
|
||||
}
|
||||
}
|
||||
|
||||
@ -28,6 +28,20 @@ public extension PureXML.Schema {
|
||||
return RelaxNGEngine(defines: defines).matches(start: start, root: .element(root))
|
||||
}
|
||||
|
||||
/// Whether `xml` is valid against the schema, validated while it is pulled
|
||||
/// event by event (the libxml2 `xmlTextReader` model) rather than over a
|
||||
/// built tree. The Brzozowski derivative is itself incremental, so only the
|
||||
/// residual pattern and a light per-element whitespace frame are retained.
|
||||
public func validate(streaming xml: String, limits: PureXML.Parsing.Limits = .default) throws -> Bool {
|
||||
let engine = RelaxNGEngine(defines: defines)
|
||||
var state = engine.streamingStart(start)
|
||||
var reader = PureXML.Parsing.EventReader(xml, limits: limits)
|
||||
while let event = try reader.next() {
|
||||
engine.streamingConsume(event, into: &state)
|
||||
}
|
||||
return engine.streamingValid(state)
|
||||
}
|
||||
|
||||
/// Every way `xml` fails the schema, as located errors with recovery hints,
|
||||
/// so an editor can show all of a faulty document's problems at once rather
|
||||
/// than only the first. An empty array means the document is valid.
|
||||
|
||||
63
Tests/RelaxNGStreamingTests.swift
Normal file
63
Tests/RelaxNGStreamingTests.swift
Normal file
@ -0,0 +1,63 @@
|
||||
@testable import PureXML
|
||||
import Testing
|
||||
|
||||
@Suite("RELAX NG streaming validation")
|
||||
struct RelaxNGStreamingTests {
|
||||
private let grammar = """
|
||||
start = element library { book+ }
|
||||
book = element book { attribute id { text }, element title { text }, author+ }
|
||||
author = element author { text }
|
||||
"""
|
||||
|
||||
private let mixedGrammar = "start = element note { mixed { element b { text }* } }"
|
||||
|
||||
/// Asserts streaming validation agrees with the tree engine, and matches the
|
||||
/// expected verdict. Streaming agreeing with the trusted tree walk on every
|
||||
/// document is the correctness proof for the derivative stepper.
|
||||
private func check(_ rnc: String, _ xml: String, expected: Bool) throws {
|
||||
let schema = try PureXML.Schema.RelaxNG(compact: rnc)
|
||||
let tree = try schema.validate(xml)
|
||||
let streamed = try schema.validate(streaming: xml)
|
||||
#expect(tree == streamed, "streaming disagreed with tree on: \(xml)")
|
||||
#expect(streamed == expected, "wrong verdict for: \(xml)")
|
||||
}
|
||||
|
||||
@Test("Valid documents stream as valid and agree with the tree engine")
|
||||
func test_valid() throws {
|
||||
try check(grammar, "<library><book id=\"1\"><title>T</title><author>A</author></book></library>", expected: true)
|
||||
// Multiple books and authors (oneOrMore).
|
||||
try check(
|
||||
grammar,
|
||||
"<library><book id=\"1\"><title>T</title><author>A</author><author>B</author></book>"
|
||||
+ "<book id=\"2\"><title>U</title><author>C</author></book></library>",
|
||||
expected: true,
|
||||
)
|
||||
// Ignorable whitespace between elements is allowed.
|
||||
try check(grammar, "<library>\n <book id=\"1\"><title>T</title>\n <author>A</author>\n </book>\n</library>", expected: true)
|
||||
}
|
||||
|
||||
@Test("Invalid documents stream as invalid and agree with the tree engine")
|
||||
func test_invalid() throws {
|
||||
// Missing required title.
|
||||
try check(grammar, "<library><book id=\"1\"><author>A</author></book></library>", expected: false)
|
||||
// Missing required attribute.
|
||||
try check(grammar, "<library><book><title>T</title><author>A</author></book></library>", expected: false)
|
||||
// Wrong order (group is ordered: title then author).
|
||||
try check(grammar, "<library><book id=\"1\"><author>A</author><title>T</title></book></library>", expected: false)
|
||||
// Empty library (book+ requires at least one).
|
||||
try check(grammar, "<library></library>", expected: false)
|
||||
// A stray element.
|
||||
try check(grammar, "<library><book id=\"1\"><title>T</title><author>A</author><stray/></book></library>", expected: false)
|
||||
// Character data where only elements are allowed.
|
||||
try check(grammar, "<library>text<book id=\"1\"><title>T</title><author>A</author></book></library>", expected: false)
|
||||
}
|
||||
|
||||
@Test("Mixed content streams correctly (text interleaved with elements)")
|
||||
func test_mixed() throws {
|
||||
try check(mixedGrammar, "<note>hello <b>world</b> there</note>", expected: true)
|
||||
try check(mixedGrammar, "<note>just text</note>", expected: true)
|
||||
try check(mixedGrammar, "<note><b>x</b><b>y</b></note>", expected: true)
|
||||
// A disallowed child element fails even in mixed content.
|
||||
try check(mixedGrammar, "<note>text <i>no</i></note>", expected: false)
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user