NodetteHQ/PureXML
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

test(schema): make the libxml2 differential seed configurable via DIFF_SEED (#385)

Browse Source 
The three generated-vs-libxml2 differentials (instance, facet, determinism) each
ran a single hardcoded seed, so they characterised one fixed slice of the open
input space rather than sampling it. Each now reads an optional DIFF_SEED
environment override (defaulting to its existing seed, so the default run is
unchanged), turning the fixed smoke test into an open-space fuzzer: a sweep over
many seeds explores far more of the unbounded schema/instance space than the
finite XSTS corpus, which is the production-readiness "uncharacterized
correctness" stopper.

A 12-seed sweep of the instance differential immediately surfaced a genuine
content-matcher false positive (cardinal-rule violation) at DIFF_SEED=13371337:
the model `( (choice(a*|(c*)+)){1,3}, b? ){1,3}` rejects `<doc><b/><b/><a/><b/>
<c/><a/></doc>` ("element 'c' is not allowed here") though libxml2 accepts it. A
nested nullable-repetition edge in the graph-structured-stack matcher, captured
for a focused fix; the default seed and every other swept seed hold false
positives at 0.
This commit is contained in:
Mihaela Mihaljevic 2026-06-29 10:44:43 +02:00 committed by GitHub
parent c55522669e
commit 77884f99b8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 6 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
Tests/DeterminismDifferentialTests.swift
View File

@ -87,7 +87,8 @@ struct DeterminismDifferentialTests {
@Test("PureXML's UPA verdict agrees with libxml2 over generated content models")
func test_determinismDifferential() {
guard enabled, let xmllint else { return }
var generator = DeterminismSchemaGenerator(seed: 0xDE7E_5A11_C0DE_F00D)
let seed = ProcessInfo.processInfo.environment["DIFF_SEED"].flatMap { UInt64($0) } ?? 0xDE7E_5A11_C0DE_F00D
var generator = DeterminismSchemaGenerator(seed: seed)
var falsePositives: [String] = []
var falseNegatives: [String] = []
var compared = 0

3
Tests/FacetValidationDifferentialTests.swift
View File

@ -114,7 +114,8 @@ struct FacetValidationDifferentialTests {
@Test("PureXML's facet-validity verdict agrees with libxml2 over generated values")
func test_facetDifferential() {
guard enabled, let xmllint else { return }
var generator = FacetSchemaGenerator(seed: 0xFACE_7D1F_F00D_C0DE)
let seed = ProcessInfo.processInfo.environment["DIFF_SEED"].flatMap { UInt64($0) } ?? 0xFACE_7D1F_F00D_C0DE
var generator = FacetSchemaGenerator(seed: seed)
var tally = Tally()
let schemaFile = NSTemporaryDirectory() + "facet-diff-schema.xsd"
for _ in 0 ..< 2000 {

3
Tests/InstanceValidationDifferentialTests.swift
View File

@ -107,7 +107,8 @@ struct InstanceValidationDifferentialTests {
@Test("PureXML's instance-validity verdict agrees with libxml2 over generated documents")
func test_instanceDifferential() {
guard enabled, let xmllint else { return }
var generator = InstanceSchemaGenerator(seed: 0x1A57_C0DE_5EED_BEEF)
let seed = ProcessInfo.processInfo.environment["DIFF_SEED"].flatMap { UInt64($0) } ?? 0x1A57_C0DE_5EED_BEEF
var generator = InstanceSchemaGenerator(seed: seed)
var tally = Tally()
let schemaFile = NSTemporaryDirectory() + "inst-diff-schema.xsd"
for _ in 0 ..< 1500 {