PureXML
Go to file
Mihaela Mihaljevic 5d38f0c82f
perf(schema): graph-structured-stack content matcher, killing the last 2^K content-model hang (#364)
* perf(schema): graph-structured-stack content matcher, sharing each xs:group (alongside)

The per-instance content-model build inlines every xs:group reference, so a
multiply-referenced nested group (g0=(g1,g1), g1=(g2,g2), ...) explodes to 2^K
states and reset actions: a depth-16 doubling chain already takes seconds, depth
20+ hangs. This compiles the same particle to a counted Recursive Transition
Network instead, where each definitionID group is compiled once and reached by a
call, and simulates it with a graph-structured stack.

ContentRTNBuilder mirrors ContentNFABuilder exactly except that a group with a
definitionID becomes a call into its shared, compiled-once fragment (memoized by
id; the fragment's accept is a return). The callee's counters are deliberately not
propagated to the caller: every counter is reset on its own particle's entry edge,
which is on the path from the callee's start, so a shared fragment resets itself on
every re-entry without the caller re-listing it. Re-listing was the residual
blow-up: it nested the reset-action lists to 2^K even when the states stayed O(K).

ContentRTNMatcher runs configurations (state, counters, gssNode): a call saves the
caller's counters on a GSS node interned by (returnState, savedCounters, position)
and a return restores them, so the shared callee never mixes counter state across
its callers; parents are a set and a popped node replays its pop to a parent added
later (the GLL completeness detail). The build drops to O(groups): a depth-24
doubling chain (2^24 inlined states) compiles to under 480 states in under a
millisecond.

It is gated by a differential (ContentRTNDifferentialTests) against the trusted
inlining matcher ContentNFA as the oracle: the two agree on the accept verdict and
the matched particle over the hand model cases, the named-over-wildcard tie-break,
the shared-group doubling chains that exercise call and return, and a 4000-case
fuzz of nested models with occurrences, wildcards, and shared groups, plus a bound
assertion. Built alongside the inlining matcher and not yet wired into validation,
so no behaviour change and the XSTS buckets are unchanged; the switch-over is gated
on this differential.

* perf(schema): add the streaming/located surface to the GSS matcher, lock it differentially

The content-model validator reads two things off the inlining matcher: matchedParticles
for per-child attribution, and the streaming startStates/step/admissible/isComplete for
located diagnostics (which child breaks the model, and the "expected a, b" hint). The GSS
matcher only had matchesWhole/matchedParticles, so it could not yet replace the inlining
matcher. This adds the streaming surface: admissible works because the closure already
expands call states into the callee's label states, so the follow set has the same shape
as ContentNFA's.

The differential gate now locks the whole surface, not just the accept verdict: driving
both matchers child by child, the first rejected child (step is nil) agrees, the
admissible expectation set agrees at the start and after every accepted child, and
isComplete agrees once the sequence is consumed. The fuzz now stamps group references the
way the parser does, a unique definitionID per distinct group (so a singly-referenced
named group also takes the call path, not only doubled groups), a shared id only for
genuinely shared structure, and some anonymous inline groups, over occurrences and
wildcards. Still alongside, not wired; no behaviour change, XSTS buckets unchanged.

* perf(schema): complete the GSS matcher surface with follow(after:), verified

follow(after:) is the last ContentNFA method the GSS matcher lacked: the completion
path (XSDCompletions) reads it for the acceptable-next labels and whether the content
may end. It composes the streaming surface, and the differential now locks its allowed
set and complete flag to ContentNFA too. The RTN matcher is now a complete drop-in for
the ContentNFA surface the validator and completions read, so the switch-over is a
mechanical builder swap gated on XSTS neutrality. Still alongside; no behaviour change.

* docs(design): record GSS Steps 1-3 done, the switch as a corpus-gated mechanical swap

Updates the counted-content-automaton Step 4 to the current state: the GSS matcher and
its full differential gate are built and green in-repo, and the production switch is now
a three-site builder swap blocked only on the XSTS neutrality measurement, which must run
on a machine that has the corpus. Records the exact swap sites and the gate so the corpus
machine can execute it mechanically.

* perf(schema): validate and complete on the GSS matcher, killing the last 2^K content-model hang

The three call sites that built the inlining content automaton once per validated
element, ComplexValidatorContentModel.sequenceStructureErrors (located structure
errors), ComplexValidator.validateParticle (per-child particle attribution), and
XSDCompletions (acceptable-next labels), now build ContentRTNBuilder and run
ContentRTNMatcher. A schema whose nested <xs:group> references inline to 2^K positions
(the g{i}=(g{i+1},g{i+1}) doubling shape) no longer makes each keystroke-time validation
of an element exponential in the nesting depth: the automaton is built in O(groups).

Validating a short instance against a depth-40 doubling schema (2^40 inlined positions),
the shape an editor sees mid-typing, now completes in milliseconds with a located error
instead of hanging; a new end-to-end test pins it.

Verified conformance-neutral on the full W3C XSTS corpus in release: valid-schemas-rejected
and valid-instances-rejected stay 0, invalid-schemas-accepted and invalid-instances-accepted
unchanged at 8 and 14. The located diagnostics, per-child attribution, and completions are
behaviour-identical by the surface differential (ContentRTNDifferentialTests), which locks
the verdict, matched particle, rejection index, admissible set, isComplete, and follow to
the inlining ContentNFA. ContentNFABuilder/ContentNFA are retained as that oracle.

* docs(design): mark GSS content-matcher Step 4 done, XSTS-neutral in production

Records that the three production sites are switched to the graph-structured-stack
matcher and the switch was measured conformance-neutral on the full W3C XSTS corpus in
release (all four buckets held). Notes the local corpus path and that ContentNFABuilder
is kept as the differential oracle rather than deleted.

* test(schema): compare full matched-particle attribution; bound a cyclic group id

From the adversarial review of the GSS matcher:

- The differential compared only the matched-particle kind (element/wildcard), so it
  could not observe the type/value metadata matchedParticles carries. Every fuzz and
  hand element now carries a per-name fixed value constraint and the comparison projects
  it, so the gate verifies the metadata is carried identically through call/return, not
  only that the kind agrees. A name maps to one constraint, so the comparison does not
  depend on the lowest-state-index tie-break (which only distinguishes same-name
  particles, a UPA violation the compiler rejects before instance validation); the
  matchedParticles doc now records that caveat.

- ContentRTNBuilder.groupFragment memoizes after compiling, so a (malformed) cyclic
  definitionID would recur without bound, the interactive-safety hazard this matcher
  removes. definitionID groups are acyclic by construction (the parser breaks cycles at
  element boundaries, the inlining builder relies on the same), but bound it anyway:
  a cycle degrades to a terminating empty fragment instead of recurring.

No production behaviour change; the differential, full suite, and the full XSTS corpus
in release (16/16, all four buckets held) stay green.
2026-06-28 05:07:37 +02:00
.claude/commands feat(parsing): streaming honors the declared encoding; streaming ISO-2022-JP 2026-06-09 17:33:52 +02:00
.githooks ci: split workflows for truthful badges and attribution gates 2026-06-08 04:37:32 +02:00
.github ci: run Windows only on release tags, not every PR (#329) 2026-06-26 11:50:06 +02:00
Benchmarks feat(xpath,parsing): public streaming surface, linear document-order at scale 2026-06-11 06:06:31 +02:00
docs perf(schema): graph-structured-stack content matcher, killing the last 2^K content-model hang (#364) 2026-06-28 05:07:37 +02:00
PublicAPITests test(api): cover the streaming, SAX, and HTML public entry points (#324) 2026-06-26 10:08:57 +02:00
scripts chore(test): vendor the XSTS archive (unmodified) under Tests/Fixtures (#322) 2026-06-26 09:20:37 +02:00
Sources perf(schema): graph-structured-stack content matcher, killing the last 2^K content-model hang (#364) 2026-06-28 05:07:37 +02:00
Tests perf(schema): graph-structured-stack content matcher, killing the last 2^K content-model hang (#364) 2026-06-28 05:07:37 +02:00
.gitattributes chore: scaffold PureXML pure-Swift XML package 2026-06-07 23:20:20 +02:00
.gitignore chore: extend .gitignore with editor, venv, node, and local-settings entries 2026-06-07 23:35:22 +02:00
.swiftformat style(format): adopt --importgrouping testable-bottom from canonical rules-swift 2026-06-18 20:33:07 +02:00
.swiftlint.yml chore(rules): align lint config and pointer with canonical rules-swift 2026-06-18 20:30:46 +02:00
AGENTS.md Mirror mandatory Swift rules 2026-06-18 22:00:41 +02:00
ATTRIBUTION.md chore: scaffold PureXML pure-Swift XML package 2026-06-07 23:20:20 +02:00
CHANGELOG.md perf(schema): graph-structured-stack content matcher, killing the last 2^K content-model hang (#364) 2026-06-28 05:07:37 +02:00
CLAUDE.md docs(rules): research-first rule, consult research repo and literature before guessing 2026-06-15 00:30:01 +02:00
CODE_OF_CONDUCT.md chore: scaffold PureXML pure-Swift XML package 2026-06-07 23:20:20 +02:00
CONTRIBUTING.md chore: scaffold PureXML pure-Swift XML package 2026-06-07 23:20:20 +02:00
GEMINI.md docs: align stale figures with code and the enforced XSTS gate 2026-06-19 23:32:04 +02:00
LICENSE chore: scaffold PureXML pure-Swift XML package 2026-06-07 23:20:20 +02:00
Package.swift chore(test): vendor the XSTS archive (unmodified) under Tests/Fixtures (#322) 2026-06-26 09:20:37 +02:00
README.md ci: run the XSTS conformance ratchet on every change (#323) 2026-06-26 10:02:41 +02:00
SECURITY.md docs: align stale figures with code and the enforced XSTS gate 2026-06-19 23:32:04 +02:00
SUPPORT.md chore: scaffold PureXML pure-Swift XML package 2026-06-07 23:20:20 +02:00

PureXML

Style and namespacing Swift macOS Swift Linux Swift Windows Swift WASM XSTS conformance License: MIT

PureXML is a dependency-free XML package written entirely in Swift.

The goal is a Linux-, Windows-, and WebAssembly-compatible XML reader/writer that does not pull in libxml2, expat, or Foundation's XMLParser. The package is intentionally strict about portability:

  • no external SwiftPM dependencies
  • no bundled C sources
  • no Foundation requirement in the library target
  • root Swift package layout
  • macOS, Linux, Windows, and WASI build gates

It is a sibling project to PureYAML and follows the same structure, rules, and verification gates.

Roadmap

The roadmap uses the TileDown Mermaid palette: green for shipped work, yellow for review, purple for epic grouping, and gray for open work.

flowchart TB
  classDef done fill:#ddf9e4,stroke:#34c759,color:#111827
  classDef review fill:#fff7d6,stroke:#ffcc00,color:#111827
  classDef epic fill:#f2e5ff,stroke:#af52de,color:#111827
  classDef todo fill:#f2f4f7,stroke:#8e8e93,color:#111827
  LDone["In main now"]:::done
  LReview["PR in review"]:::review
  LEpic["Epic grouping"]:::epic
  LTodo["Open issue, no PR"]:::todo
  Core["Core shipped: #1 push, #19 SAX, #28 encoding, #29 entities, #31 tree, #32 writer, #33 reader, #34 save, #35 chars"]:::done
  Query["Query + validation shipped: #21 #22 #23 query, #30 regex, #2 XSD/RELAX NG, #25 Schematron"]:::done
  Transform["Transform + HTML shipped: #3 XSLT, #20 HTML, #24 XInclude, #26 C14N, #27 Catalog"]:::done
  Parity["Parity completion shipped: #61 (XSLT keys/output, XSD union/import/identity, RELAX NG compact, HTML5 doc structure, C14N 2.0 trim)"]:::done
  Validator["Validation framework + recovering reader shipped: #92 (one located severity-aware ValidationError, never-crash read)"]:::done
  Editor["Editor integration shipped: #94 (source spans, unified lint, schema completions, structured quick-fixes)"]:::done
  Conformance["Conformance push shipped: DTD #72 #73, XSD #75 #76 #77 #78, parsing #95 #96 #98 #100, regex #89, catalog #86, XSLT #81 #82"]:::done
  LDone --> LReview
  LReview --> LEpic
  LEpic --> LTodo
  LTodo --> Core
  Core --> Query
  Query --> Transform
  Transform --> Parity
  Parity --> Validator
  Validator --> Editor
  Editor --> Conformance

The goal is full libxml2 feature parity in pure Swift with zero dependencies, by clean-room reimplementation (behavior reproduced from the private PureXML-research analysis, no upstream source copied). The streaming core, the schema languages (DTD, XSD, RELAX NG, Schematron), XPath 1.0, and the transform/canonicalization stack are shipped; the remaining depth gaps are tracked under epic #71. Deliberate non-goals: network fetching (nanohttp/nanoftp) and the threading/memory infrastructure stay out, and external resolution is opt-in through an injected resolver so XXE stays closed.

The original epics, the documented-subset parity work (#61), the validation framework and recovering reader (#92), and the editor-integration layer (#94) are all shipped (summarized above). The beyond-parity libxml2 conformance audit (#71), its parser-breadth sub-epic (#74), and the parity-frontier epic (#105: output encodings, streaming validation, the last encodings, the analysis findings, and the first W3C-suite pass) are all complete. Feature parity is done; the remaining distance is production readiness for IDE use, tracked under epic #167 (schema authority, differential oracle, located diagnostics). Conformance suite maintenance (#122, closed) leaves #130 (XSLT xalan burn-down) as the only active runner work; see docs/roadmap.md.

flowchart TB
  classDef done fill:#ddf9e4,stroke:#34c759,color:#111827
  classDef review fill:#fff7d6,stroke:#ffcc00,color:#111827
  classDef epic fill:#f2e5ff,stroke:#af52de,color:#111827
  classDef todo fill:#f2f4f7,stroke:#8e8e93,color:#111827
  Epic71["#71 beyond-parity libxml2 audit: complete"]:::epic
  Epic74["#74 parser-breadth sub-epic: complete"]:::epic
  Done["All audit children closed in main: DTD #72 #73, XSD #75 #76 #77 #78, parsing #95 #96 #98 #100, regex #89, catalog #86, XSLT #81 #82, RELAX NG #79, Schematron #80, DOM #90, serialization #91, C14N #85, XPath/XPointer #88, XInclude #87, validators #101, HTML5 #83 #84, encodings #97 #99 #102 #103"]:::done
  Epic71 --> Done
  Epic74 --> Done
  Epic105["#105 parity frontier: complete (#106 #107 #108 output encodings, streaming validation, last encodings; findings #109-#119; first W3C pass #120)"]:::epic
  Epic121["#121 xmlconf full archive: complete"]:::epic
  Oasis["#123 OASIS/NIST: zero deviations"]:::done
  Sun["#124 Sun section: complete, zero deviations"]:::done
  IBM["#125 IBM section: clean to two documented classes"]:::done
  Japanese["#126 japanese section: complete, zero deviations"]:::done
  Eduni["#127 eduni: runner + exact baselines, four classified deviations"]:::done
  StrictSubset["#128 strict internal-subset profile: complete, xmltest 185/186"]:::done
  ScalarLex["#135 scalar-level lexing: complete"]:::done
  NSConstraints["#136 namespace constraints: complete"]:::done
  EncodingMismatch["#137 encoding-declaration mismatches: complete"]:::done
  BaseURI["#138 per-entity base URIs: complete"]:::done
  Epic122["#122 conformance suite maintenance: complete (#130 survivor)"]:::epic
  XSTS["#129 XSTS runner shipped; gates #145-#148 at 0/39/0/26"]:::review
  XSLTTS["#130 XSLT 1.0 suite (xalan-test): ~126 baseline failures"]:::review
  Parity100["Behavioral parity closed: #141 top-level params, #142 public streaming, #143 XPath budget, #144 WASI runtime proof, pl/ru sort collation"]:::done
  Epic139["#139 faster than libxml2, measured + streaming (benchmarks in docs/benchmarks.md; quadratics killed; unsafe authorized)"]:::epic
  Audit140["#140 XSLT validation-framework audit"]:::todo
  Epic139 --> Parity100
  Epic139 --> Audit140
  RNGTS["#131 RELAX NG spec suite: complete, one documented class"]:::done
  C14NTS["#132 C14N spec vectors: complete"]:::done
  Epic167["#167 production readiness for IDE use (v0.2.0)"]:::epic
  Oracle171["#171 schema differential harness"]:::todo
  CrossDoc161["#161 cross-document composition"]:::todo
  Gates145["#145-#148 XSTS gates: burn-down to 0"]:::review
  Loc169["#169 located schema diagnostics"]:::todo
  Epic195["#195 xsi:type and substitution-group derivation correctness"]:::epic
  Epic105 --> Epic121
  Epic105 --> Epic122
  Epic121 --> Oasis
  Epic121 --> Sun
  Epic121 --> IBM
  Epic121 --> Japanese
  Epic121 --> Eduni
  Epic121 --> StrictSubset
  Epic121 --> ScalarLex
  Epic121 --> NSConstraints
  Epic121 --> EncodingMismatch
  Epic121 --> BaseURI
  Epic122 --> XSTS
  Epic122 --> XSLTTS
  Epic122 --> RNGTS
  Epic122 --> C14NTS
  Epic122 --> Epic167
  Epic167 --> Oracle171
  Epic167 --> CrossDoc161
  Epic167 --> Gates145
  Epic167 --> Loc169
  Epic167 --> Epic195

Status

Current focus: production readiness for IDE use (#167). XSTS settled baselines (valid schemas rejected / invalid schemas accepted / valid instances rejected / invalid instances accepted), enforced in Tests/XSTSSuiteTests.swift: 0 / 39 / 0 / 26. See docs/roadmap.md for milestones and open tasks.

PureXML is a working, dependency-free XML library today: parse, emit, validate, query, and stream documents on macOS, Linux, Windows, and WASM. The test suite currently runs 1448 tests in 258 suites (swift test).

Shipped (libxml2-aligned surface)

  • Model (PureXML.Model): Node, Element, Attribute, QualifiedName, and a mutable TreeNode for in-place editing (insert, remove, replace, copy, parent navigation).
  • Parsing (PureXML.Parsing): iterative streaming parser over strings, bytes, or incremental character/byte sources. Pull events via EventReader, SAX2-style callbacks via SAXHandler, and a pull cursor via TextReader (libxml2 xmlTextReader). Handles elements, attributes, text, predefined and numeric entities, comments, CDATA, and processing instructions. Namespace bindings (xmlns) are resolved to URIs on parse.
  • Encoding: UTF-8/16/32 (with BOM sniffing) from raw bytes or streaming byte input, plus the ISO-8859 single-byte family, the Windows code pages (1250-1258), KOI8-R/U, and the CJK multi-byte encodings (Shift-JIS, EUC-JP/KR/TW, GBK, GB18030, Big5, ISO-2022-JP).
  • DTD (opt-in): <!DOCTYPE> is off by default (Limits.allowDoctype: false) to keep XXE and entity-expansion closed. When enabled, the internal subset is parsed; internal general entities expand with a bounded cap; external entities and external subsets are refused unless an injected EntityResolver supplies replacement text.
  • Validation (PureXML.Validation): one composable framework (the OpenAPIKit idiom, parameterized over subject and document) backing structural checks, DTD validation (content models, attribute lists/defaulting, tokenized types, ID/IDREF), XSD (datatypes, complex types, derivation control, wildcards, namespace qualification), RELAX NG (XML and compact syntax), and Schematron (rules, phases, dynamic messages). Errors are located by coding path for editor use.
  • Emitting (PureXML.Emitting): Serializer (save options, quote style, xml:space, line endings) and an incremental, namespace-aware Writer (libxml2 xmlTextWriter) with compact and pretty modes.
  • XPath / XPointer (PureXML.XPath): XPath 1.0 (all axes, the core function library, eval-time namespace bindings) and XPointer (element(), xpointer(), xpath1(), xmlns()).

Consumption modes

You choose how the document is consumed; only the tree mode materializes the whole document. The input itself never has to be a whole string: every mode below also accepts an incremental character or byte source (pulling: / pullingBytes:), so the engine advances step by step with bounded memory and the bytes are pulled on demand. The String overloads are a convenience for when you already hold the text.

  • Pull cursor (step by step): events(_:) / events(pulling:) returns an EventReader; call next() to get one Event at a time. TextReader is the node cursor (the libxml2 xmlTextReader model): read() advances to the next node, you inspect it, read() again. Bounded state, no tree.
  • Push / feed: PushParser.feed(_:)/finish() accepts the document in arbitrary chunks (a socket, a generator); events(feeding:) exposes the same over an AsyncSequence. It retains only the current incomplete token and the open-element stack.
  • SAX: parse(_:sax:) delivers SAX2-style callbacks, no tree.
  • Tree: parse/parseTree build a Model.Node / TreeNode tree for random access, mutation, validation, XPath, and XSLT. This mode (like libxml2's DOM) holds the whole document, because those operations need the full structure.

Safe defaults

By default the parser rejects <!DOCTYPE> and refuses every external reference (EntityResolver.refusing). Enable DTD processing only when you need internal subset validation or internal entities, and wire a resolver only for trusted, in-memory replacements. Network fetching (nanohttp/nanoftp) and libxml2's threading/memory infrastructure are deliberate non-goals.

Remaining work

Feature parity with libxml2's documented surface is complete: the core I/O stack, the query and transform layers, all four schema languages, the validation framework, editor integration, and the encoding set (the ISO-8859 family, the Windows code pages, and the CJK multi-byte encodings) are shipped. The beyond-parity audit epics (#71, its parser-breadth sub-epic #74) and the parity-frontier epic (#105) are closed.

The remaining distance is production readiness for IDE use, tracked under epic #167: driving the XSTS conformance buckets toward zero (currently 0 / 39 / 0 / 26), located schema diagnostics (#169), the schema differential oracle (#171), and cross-document composition (#161). The one active runner burn-down is #130 (the XSLT 1.0 xalan suite). See docs/roadmap.md.

Usage

import PureXML

// Build a tree and emit it (works today).
let element = PureXML.Model.Element(
    "book",
    attributes: [.init("id", "bk101")],
    children: [.element(.init("title", children: [.text("XML Developer's Guide")]))],
)

let xml = PureXML.serialize(.element(element))
try PureXML.validate(.element(element))

// Parse it back into a tree.
let node = try PureXML.parse(xml)

// Or stream events without building a tree (drives chunked input too).
var reader = PureXML.events(xml)
while let event = try reader.next() {
    // handle .startElement / .characters / .endElement / ...
}

// DTD validation (opt-in: allowDoctype + optional resolver for trusted externals).
let issues = try PureXML.validateAgainstInternalDTD(
    xmlWithDoctype,
    limits: .init(allowDoctype: true),
)

// XPath over a parsed tree.
let titles = try PureXML.xpath("//title", over: node)
    .compactMap(\.element)

Attribution

PureXML is informed by the behavior of established XML parsers (libxml2, expat, Foundation's XMLParser) and by the W3C XML 1.0 specification, but it does not copy their implementation into Sources/. See ATTRIBUTION.md.

Development Contract

PureXML must stay dependency-free and portable. Before merging changes:

  • Swift tools version: 6.1
  • Package products: PureXML
  • SwiftPM dependencies: none
  • Hosted CI matrix: macOS, Linux, Windows, and WASM
bash scripts/check-all.sh

That command expands to:

bash scripts/check-style.sh
bash scripts/check-namespacing.sh
bash scripts/check-forbidden-patterns.sh
bash scripts/check-validation-coverage.sh
bash scripts/check-validation-fields.sh
bash scripts/check-changelog-touched.sh
swiftformat . --config .swiftformat --lint
swiftlint --config .swiftlint.yml --strict
swift build
swift test

License

MIT.