EventReader.next(), both initializers, and the Event alias are public
(an internal typealias in a split file had been shadowing the public
Parsing.Event); a new PureXMLPublicAPITests target compiles without
testable import so facade gaps fail CI. The large-corpus benchmarks
exposed document-order sorting as quadratic on flat fan-outs; the fix's
first attempt was itself worse (an eager sibling table built per
predicate evaluation), caught by sampling the live run, and replaced
with a lazy table: single-node sets never touch order keys and a
parent's table builds on its second lookup. XPath at 42 MB drops
27.1 s to 1.57 s (190x to 11.7x); at 210 MB PureXML completes in 9.4 s
the query libxml2 refuses at its 10M node-set cap (#143 tracks our
configurable budget). Benchmark harness: size series to 210 MB, CORPUS
mode for real files (NASA, SwissProt measured), SAX stage,
refusal-aware table, NULL-safe C driver. All conformance suites,
Linux, and WASM verified. Closes#142.
String input copies its UTF-8 once into owned storage (an unsafe
pointer encapsulated in a class lifetime, never escaping the parser)
and scalars decode straight off the pointer, eliminating the
per-character closure chain; ASCII literals match and consume at the
byte level; and plain-ASCII character data scans as a single byte run
with one String built per run. Parse drops 32 percent cumulative
(0.49 s to 0.34 s on the 4.2 MB corpus, libxml2 ratio 30x to 20x).
The W3C suite caught one boundary (a run starting with '>' could
swallow the ]]> check after the character path consumed ]]); the run
scanner leaves a leading '>' to the character path, pinned as a unit
regression. All conformance suites, Linux, and WASM verified.
A declared 16/32-bit encoding over BOM-less 8-bit bytes is rejected; a
BOM followed by a declaration outside its family is rejected, reading
the declaration through the BOM's own byte pattern; and an external
entity whose text declaration names a higher XML version than the
document is refused (errata E38). The eduni not-wf baseline is empty.
Reader and decoder helpers split for the length caps.
Closes#137
The namespace layer rejects malformed qualified names, declarations of
the xmlns prefix, misbindings of the xml prefix and either reserved
namespace name (including as the default namespace), and prefix
undeclaration; attributes must be distinct by expanded name; PI targets
and entity/notation names are NCNames; and DTD-typed xmlns values bind
their normalized form. Clears all 18 namespace cases from the eduni
baseline; xmltest valid/sa/012 joins the documented deviations per the
suite's own NAMESPACE='no' flag.
Closes#136
The eduni sections run with exact classified baselines. Fixes: 3.3.3
CDATA attribute-value normalization (literal whitespace to space before
reference decoding; char-ref whitespace survives; tokenized collapse on
SPACE only); production-68 lenient undeclared entities in content and
attribute values (validity finding, reference kept literal, nothing
fetched); errata E15 (references/comments/PIs/whitespace in EMPTY,
char-ref whitespace as data in element content, double-escape shape
included); errata E2 duplicate enumeration tokens; errata E14
declarations completed inside PE replacements (VC: Proper
Declaration/PE Nesting); NCName ID values; namespace declarations
subject to the every-attribute-declared rule. Reader validity machinery
split to EventReaderValidity for caps.
Closes#127
The IBM section (1094 cases) runs clean to two documented classes:
279 pre-Fifth-Edition character-class cases (P85-89, count asserted
exactly) and 2 PE-in-internal-subset feature cases. Real gaps closed:
a document needs a root element (production 1); entity references in
literals must be Names or character references; WFC: No Recursion
checked at declaration over the general/parameter reference graph;
undeclared PE references split into WFC (standalone, or no external
declarations possible) versus deferred validity finding, also for
attribute-default references; refused external PEs are neither; and
groups split across PE replacements report VC: Proper Group/PE
Nesting. Entity machinery split to DoctypeScannerEntities for caps.
Closes#125
The scanner records declaration provenance (internal-subset entities,
element models, attribute lists); the validator reports the three
standalone='yes' dependencies on the external subset: a default that
would be supplied, a non-CDATA normalization that would change a value,
and whitespace in externally-declared element content. The parser hides
externally-declared entities from standalone documents, so referencing
one reports as undeclared (WFC: Entity Declared, 2.9). The Sun section
is fully clean: not-wf 56/56, valid 28/28, invalid 74/74, no baseline.
Closes#134Closes#124
A general entity whose replacement text contains markup (directly or
transitively) is reparsed as content at the reference (4.4.2 Included):
the replacement is WFC-validated, cycle-checked, and budgeted, then
pushed back into the reader stream, so its elements become elements in
the tree and the pull-event stream rather than escaped character data.
Character references in internal entity literals now expand at
declaration time for general entities too (4.4.5), making char-ref
markup splice and the Appendix D double escape render as a literal
ampersand; the reference-time WFC validates remaining references
instead of re-expanding. Text-only entities keep the coalescing
single-pass path. Sun valid is 28/28.
Closes#133
The OASIS/NIST section of the W3C conformance suite runs via a new
XMLCONF_ROOT-gated runner driven by the section's own manifest (parsed
with PureXML): 247/247 not-wf rejected, all 100 well-formed cases parse,
zero deviations. Closing its 24 gaps makes the DTD subset grammar
strict: exact PEReference shape, strict declaration names and keywords,
NDATA and notation tails, no bare '%' in entity values, case-sensitive
balanced conditional sections, junk between declarations rejected,
external-subset violations now propagate (with text-declaration
support, production 77), and a PI target must be separated from its
data. The xmltest baseline tightens to 181/186: CDATA in the internal
subset (107) is now rejected, five documented deviations remain.
Closes#123
Character references now require lowercase 'x' for hex and a referenced code
point inside the XML Char production (rejecting �, , ,
surrogates, and X). The reserved 'xml' processing-instruction target
(any case) is legal only as the declaration at the document's very first
bytes: whitespace or a comment before <?xml ...?>, a declaration inside
content or after the root, and case variants like <?XML?> are all rejected.
Comments and PI data are checked for the XML Char production like content.
not-wf/sa: 109/186 rejected (from 93); valid/sa unchanged at 118/119; all
846 suite tests green, baseline rebased to the exact 77 remaining (the
internal-subset grammar and entity replacement-text classes on #120).
Part of #120, #105
Add five well-formedness checks the W3C xmltest suite surfaced: '--' inside
comments (also rejecting the ---> ending), raw '<' in attribute values, the
literal ]]> sequence in character data, attributes not separated by
whitespace, and characters outside the XML Char production in content, each
a located ParseError. The attribute scanners move to EventReaderAttributes
to stay under the type-body cap.
Add the opt-in W3C xmltest runner (XMLTS_ROOT-gated; the suite is never
vendored, James Clark's license permits redistribution only as the unmodified
archive; Foundation is used in this one test file for file access only).
Baseline: valid/sa 118/119, not-wf/sa 93/186 rejected (up from 80), the
remaining 93 carried as an exact known-deviations baseline so progress and
regressions are both visible. The two remaining leniency classes (internal-
subset DTD grammar, entity replacement-text well-formedness) are #120.
Part of #105 (Tier 2)
Broaden the TextReader pull cursor toward libxml2's xmlTextReader:
- Attribute nodes: moveToFirstAttribute/moveToNextAttribute/
moveToAttribute(_:)/moveToElement reposition the cursor onto each
attribute as an .attribute node and back.
- Namespace accessors: namespaceURI, localName, prefix from the current
element's or attribute's resolved name.
- xml:lang: inherited down the tree, overridable per element, reset when
it goes out of scope.
- Position: lineNumber/columnNumber report where each node begins, so a
streaming consumer can locate findings in the source. EventReader now
records each node's start mark.
DOCTYPE and entity-reference nodes are not surfaced (the DTD is exposed
through documentType, and entities are expanded inline); whitespace is
not split into significant/insignificant (needs the content model);
baseURI needs a document URI. The accurate positioning path for editor
findings remains readTree with source ranges.
Closes#98
Parse the XML declaration's pseudo-attributes instead of skipping the
<?xml?> processing instruction:
- version, encoding, and standalone are surfaced on a new XMLDeclaration,
returned from parseWithDocumentType and via PureXML.xmlDeclaration(_:).
- The pseudo-attributes are validated: fixed order (version, encoding,
standalone), known names only, and standalone must be yes or no. A
malformed declaration is rejected.
- The declared version is now readable, a prerequisite for XML 1.1.
parseWithDocumentType returns a ParsedDocument struct (node,
documentType, declaration) rather than a 2-tuple; the member names are
unchanged so existing call sites are unaffected.
Closes#95
Slice 1 of #94 (editor integration). The keystone: a located validation finding
can now be mapped back to a source range.
- TreeNode gains a sourceRange (start/end Mark); SourceRange is a half-open span.
- PureXML.readTree builds a mutable, parent-aware, best-effort tree whose every
node carries its span (an element from the < of its start tag to past its end
tag; leaves over the text they consumed). Never throws, recovers in place,
returns located diagnostics.
- TreeNode.node(at:) navigates a validation coding path to its node using the
same canonical convention the validators build paths with (a sibling index
only when a name repeats), and sourceRange(at:) returns the span. So
Document.validate's ValidationError.codingPath resolves to characters in the
document.
Tests prove the spans, the navigation, the end-to-end bridge (an XSD finding to
a source range), and that ranged reading of invalid input never crashes.
Closes#93. Recovery moves into a recovering mode on EventReader, so malformed
input is repaired in place rather than dropped to the next markup boundary:
- A duplicate attribute is recorded and the later occurrence dropped, keeping the
element. <a id='1' id='2'>y</a> now reads as <a id="1">y</a> with one
diagnostic (was an empty tree with three).
- A mismatched or unexpected end tag pops the open stack to the matching ancestor,
implicitly closing the elements nested inside it. <a><b><c>x</b></a> reads as
<a><b><c>x</c></b></a> with a single diagnostic.
- Pre-root junk reports "unexpected character" instead of the misleading
"content after the root element".
In recovering mode next() no longer throws: it records a Diagnostic and emits
best-effort events, popping to a match on a mismatch and stopping on truncation
so the builder closes what is open. The strict path is unchanged. readRecovering
becomes a plain non-throwing drain that takes the reader's diagnostics. Recovery
stays deterministic; new tests pin the kept-element and pop-to-match results.
Slice 2 of #92. Add PureXML.read, a deterministic recovering reader: it never
throws and never traps, returning the maximal best-effort tree plus one located
Diagnostic per problem.
- ReadResult { node, diagnostics } is the Equatable result type, so a
deterministic read can be compared whole.
- Diagnostic carries a message and a structured source Mark; a thrown ParseError
maps to one directly (ParseError.mark exposes the position each case carries).
- The reader resynchronizes after a failure (EventReader.recover(from:)):
progress-aware, so a failure that already consumed the bad construct retries
from there, while a stuck failure skips to the next markup start; it always
advances when input remains, guaranteeing termination.
- The tree builder assembles leniently: elements left open by truncation are
closed, an end tag with no open element is dropped. Well-formed input yields
exactly parse(_:)'s tree with no diagnostics.
Tests pin exact trees and diagnostics for truncation, mismatched tags, and stray
junk, prove determinism (same bytes give an identical ReadResult), and fuzz 400
seeded inputs (arbitrary plus truncated valid docs) asserting no crash.
Recovery here is resync-based; finer in-reader recovery (keep the element on a
duplicate attribute, pop to a matching open tag) is the next slice.
Add internal parameter entities (parsed, stored, and expanded within the
DTD, including bare %name; references that inject markup declarations,
bounded by depth and the expansion budget). Record external general
entities and the external subset identifier, and load them only through an
injected EntityResolver (a struct of closures). The default refusing
resolver fetches nothing, so XXE stays closed: an external entity fails as
undefined and the external subset is left unread. Internal declarations win
over the external subset.
Closes#29
Surface <!ATTLIST> declarations from the parser and validate attributes against
them. Reimplemented from the DTD attribute-list spec; no upstream code copied.
- DocumentType gains attributeLists (raw ATTLIST bodies per element); the
DoctypeScanner captures them and the EventReader plumbs them through.
- AttributeListParser parses a body into AttributeDeclaration values
(AttributeType: cdata or enumeration; AttributeDefault: required/implied/
fixed/value).
- DTDSchema now checks #REQUIRED presence, #FIXED value match (when present),
and enumerated-value membership, in addition to content models.
- Tokenized types (ID/IDREF/NMTOKEN) parse but are not yet value-enforced.
- 5 attribute tests; 75 total.
Surface the parsed <!DOCTYPE> internal subset's <!ELEMENT> declarations from the
parser and validate a tree against them. Reimplemented from the DTD content-model
spec; no upstream code copied.
- DoctypeScanner captures <!ELEMENT name model> declarations; the parser surfaces
them as Parsing.DocumentType (via parseWithDocumentType).
- Validation.ContentModel + ContentModelParser parse a model string (EMPTY, ANY,
(#PCDATA), mixed, and element content with sequence/choice and ?/*/+).
- ContentModelMatcher matches the child element-name sequence against a model as
a regular language (NFA-position matching).
- Validation.DTDSchema.validate reports issues for EMPTY/PCDATA/mixed violations,
element content with stray text, and child sequences that do not match.
- PureXML.validateAgainstInternalDTD(_:) parses and validates in one call.
- ATTLIST validation is out of scope. 9 tests; 70 total.
Add opt-in DTD processing behind Limits(allowDoctype:), off by default. When on,
DoctypeScanner reads the <!DOCTYPE> internal subset and honors internal
<!ENTITY name "value"> general entities, which EntityDecoder expands recursively
in text and attribute values.
Safety is mandatory and reimplemented from the Expat amplification-accounting
model (no upstream code copied):
- only entity-expanded characters count against a shared budget
(Limits.maxEntityExpansion); overflow raises amplificationLimitExceeded, which
defeats billion-laughs.
- a visiting set detects recursive entities (recursiveEntity).
- undeclared references raise undefinedEntity.
- external entities are never stored or loaded, so XXE stays closed.
The default still rejects <!DOCTYPE>. 8 DTD tests (incl. billion-laughs and the
external-entity refusal); 49 total.
Add namespace resolution following the XML Namespaces rules, reimplemented in
Swift from libxml2's SAX2 model; no upstream code copied.
- QualifiedName gains an optional resolved namespaceURI (nil for names built
directly or in no namespace), with a resolved(namespaceURI:) helper.
- NamespaceContext maintains a scope stack of xmlns bindings as elements open
and close, resolving element and attribute names against it.
- A default namespace applies to unprefixed element names but never to
attributes; the xml prefix is built in; xmlns declarations are preserved as
attributes so documents round-trip; an unbound prefix raises
ParseError.undefinedNamespacePrefix. End-tag matching stays lexical.
- Namespace-free documents are unchanged (URIs remain nil).
- 7 namespace tests; 39 total.
Add PureXML.Parsing.Limits: configurable caps on nesting depth (default 256,
matching libxml2), name length, and content length, enforced during scanning in
the event reader. This protects against pathological input (deeply nested or
oversized documents) and keeps the recursive Model.Node value safe to build and
tear down (extreme depth otherwise overflows on ARC release). Threaded through
Parser.parse and the top-level parse/events API. Reimplemented from libxml2's
bounded-by-default posture; no upstream code copied.
Replace the notImplemented stub with a real parser built on a pull-based,
streaming event core that never requires the whole document in memory.
- Parsing.EventReader: pulls characters through a source closure with a bounded
lookahead buffer and emits one Parsing.Event at a time, holding only an
open-element stack. Drives chunked/streamed input.
- Parsing.Parser: builds a Model.Node tree iteratively over the event core
(explicit stack, no recursion).
- Reader: streaming character cursor (closure-backed, bounded buffer) replacing
the previous whole-string Array.
- EntityDecoder: the five predefined entities and decimal/hex char references.
- Handles elements, attributes, text, comments, CDATA, processing instructions,
and an XML declaration in the prolog. Rejects <!DOCTYPE> by default (no
DTD/XXE/entity-expansion exposure).
- Top-level API: parse(_:), parse(pulling:), events(_:), events(pulling:).
- 21 tests including event-stream and chunked-source coverage.
