Commit Graph

10 Commits

Author SHA1 Message Date
82e70679ef feat(validation): 3.3.3 attribute normalization, 4.3.1 text declarations, 4.4.5 PE char-refs
Tokenized (non-CDATA) attribute values normalize before validity
checks, so enumerations, NOTATION lists, and the ID/IDREF table see
collapsed values. External parsed entities lose their leading text
declaration (validated: version optional, encoding required, never
standalone) instead of carrying it into replacement text. Character
references in PE literals expand at declaration time, making the
Appendix D '%zz;' example parse. A PI target inside the DTD subset
must be separated from its data. Sun baselines tighten to not-wf 55/56
and valid 26/28; the two remaining valid failures share one cause
(replacement text with markup is included as character data, filed
separately).
2026-06-10 06:43:25 +02:00
5671fb3ce1 feat(parsing): strict DTD subset grammar, oasis section of xmlconf fully clean
The OASIS/NIST section of the W3C conformance suite runs via a new
XMLCONF_ROOT-gated runner driven by the section's own manifest (parsed
with PureXML): 247/247 not-wf rejected, all 100 well-formed cases parse,
zero deviations. Closing its 24 gaps makes the DTD subset grammar
strict: exact PEReference shape, strict declaration names and keywords,
NDATA and notation tails, no bare '%' in entity values, case-sensitive
balanced conditional sections, junk between declarations rejected,
external-subset violations now propagate (with text-declaration
support, production 77), and a PI target must be separated from its
data. The xmltest baseline tightens to 181/186: CDATA in the internal
subset (107) is now rejected, five documented deviations remain.

Closes #123
2026-06-10 06:29:13 +02:00
32d42410b9 feat(parsing): entity replacement-text well-formedness, fifth suite pass
Implement the replacement-text constraint at the point of reference: when a
general entity is included, its replacement (character references expanded
as at declaration time) must reparse as balanced content in isolation, via a
new EntityReplacementGrammar mini-parser. This rejects a bare or incomplete
reference ('&', '&#9' unterminated), tags spanning the entity boundary
('</foo><foo>'), a raw '<' or incomplete reference inside an embedded
attribute value, '<' followed by a non-name character, and a reserved-target
PI in the replacement, exactly the suite's semantics, including that an
UNREFERENCED entity may carry any value and that only the first declaration
of a name binds. Attribute-list defaults must reference already-declared
internal (not external/unparsed/forward-declared) entities with a
non-recursive chain. A CDATA section inside replacement text now shields its
content from reference expansion, fixing the last valid-side case.

W3C xmltest final standing: valid/sa 119/119, not-wf/sa 180/186, with the
six remaining deviations all deliberate and documented in the baseline:
internal-subset conditional/CDATA sections (063, 107) and parameter-entity
references (160-162) are features this package supports, and 141 expects the
1998 suite's name-character classes where this package implements XML 1.0
Fifth Edition's. Zero false positives across all 846 tests at every step.

Closes #120
2026-06-10 05:47:42 +02:00
dd0ff5d086 feat(parsing): fourth W3C-suite strictness pass, declaration grammar
Make the DTD declaration grammar strict: <!ENTITY> requires its whitespace
(after the keyword, around %, after the name), a real name (not #DEFAULT),
nothing but '>' after the value, NDATA preceded by whitespace and forbidden
on parameter entities; entity-value literals are validated (XML Chars only,
every & a complete reference); SYSTEM/PUBLIC external identifiers parse
strictly (PUBLIC needs both literals for entities and the DOCTYPE, public-only
stays legal for notations) with PubidChar validation; <!ATTLIST> bodies are
validated against XML 1.0 productions 52-60 via a new DTDAttListGrammar
(spacing, attribute types, enumerations, quoted defaults); the reserved xml
PI target is rejected inside the subset. Also: strict UTF-8 decoding (no
silent replacement of malformed sequences), XML Char checks in attribute
values and CDATA, the declaration's version/encoding value grammar, and
required whitespace between pseudo-attributes. The strict machinery lives in
DoctypeScannerStrict/DTDAttListGrammar to stay under the length caps.

W3C xmltest: not-wf/sa 162/186 rejected (from 129; 80 at first contact),
valid/sa 118/119, zero false positives across all 846 tests at every step.
The remaining 24-case baseline is exactly: the entity replacement-text class
(19) and the deliberate internal-subset extensions this package supports as
features (conditional sections, PEs in the internal subset: 5).

Part of #120, #105
2026-06-10 05:34:11 +02:00
805993e656 feat(parsing): strict <!ELEMENT> content-model grammar
Third W3C-suite strictness pass: DTDContentModelGrammar validates each
element declaration's content specification against the XML 1.0 grammar
(productions 46-51) at DTD scan time, EMPTY/ANY, Mixed (with #PCDATA first,
names only, the trailing * required with alternatives, no nesting), and
children (one connector per group, quantifiers bound directly to their
particle, balanced parentheses, no empty groups, no SGML leftovers like
'- -', 'O O', '&' connectors, or inclusion/exclusion suffixes). The element
declaration also requires whitespace after <!ELEMENT and after the name. The
lenient ContentModelParser still drives validation behavior; this is a
well-formedness gate. not-wf/sa: 129/186 rejected (from 109); valid/sa
unchanged; zero false positives across all 846 suite tests; baseline rebased
to the exact 57 remaining (declaration-grammar details and the entity
replacement-text class).

Part of #120, #105
2026-06-10 05:20:33 +02:00
7f1aada240 feat(dtd): conditional sections, notations, NDATA, external parameter entities
Deepen DTD processing in the doctype scanner:

- Conditional sections are processed, not skipped: INCLUDE sections have
  their declarations scanned, IGNORE sections are discarded, the keyword
  may be a parameter-entity reference, and nested sections are handled.
- <!NOTATION> declarations are parsed and stored on DocumentType.
- Unparsed entities (<!ENTITY ... NDATA n>) are parsed and stored with
  their notation, distinct from general entities.
- External parameter entities are loaded through the injected resolver so
  their declarations expand; the default refusing resolver loads nothing,
  keeping XXE closed.
- NOTATION attributes are a distinct type, validated against the declared
  notations (value must be listed and the notation declared).

Tracking unparsed entities also lets ENTITY/ENTITIES attribute values be
validated against the declared NDATA entities, closing the gap left when
#73 could only check name syntax.

Closes #72
2026-06-08 14:39:42 +02:00
c75c97aa9b feat(dtd): external and parameter entities, secure by default
Add internal parameter entities (parsed, stored, and expanded within the
DTD, including bare %name; references that inject markup declarations,
bounded by depth and the expansion budget). Record external general
entities and the external subset identifier, and load them only through an
injected EntityResolver (a struct of closures). The default refusing
resolver fetches nothing, so XXE stays closed: an external entity fails as
undefined and the external subset is left unread. Internal declarations win
over the external subset.

Closes #29
2026-06-08 03:08:20 +02:00
5cbe938826 feat(validation): add DTD attribute (ATTLIST) validation
Surface <!ATTLIST> declarations from the parser and validate attributes against
them. Reimplemented from the DTD attribute-list spec; no upstream code copied.

- DocumentType gains attributeLists (raw ATTLIST bodies per element); the
  DoctypeScanner captures them and the EventReader plumbs them through.
- AttributeListParser parses a body into AttributeDeclaration values
  (AttributeType: cdata or enumeration; AttributeDefault: required/implied/
  fixed/value).
- DTDSchema now checks #REQUIRED presence, #FIXED value match (when present),
  and enumerated-value membership, in addition to content models.
- Tokenized types (ID/IDREF/NMTOKEN) parse but are not yet value-enforced.
- 5 attribute tests; 75 total.
2026-06-08 02:07:01 +02:00
f02561b01d feat(validation): add DTD content-model validation
Surface the parsed <!DOCTYPE> internal subset's <!ELEMENT> declarations from the
parser and validate a tree against them. Reimplemented from the DTD content-model
spec; no upstream code copied.

- DoctypeScanner captures <!ELEMENT name model> declarations; the parser surfaces
  them as Parsing.DocumentType (via parseWithDocumentType).
- Validation.ContentModel + ContentModelParser parse a model string (EMPTY, ANY,
  (#PCDATA), mixed, and element content with sequence/choice and ?/*/+).
- ContentModelMatcher matches the child element-name sequence against a model as
  a regular language (NFA-position matching).
- Validation.DTDSchema.validate reports issues for EMPTY/PCDATA/mixed violations,
  element content with stray text, and child sequences that do not match.
- PureXML.validateAgainstInternalDTD(_:) parses and validates in one call.
- ATTLIST validation is out of scope. 9 tests; 70 total.
2026-06-08 01:59:00 +02:00
01a35cc6e6 feat(parser): optional DTD with internal entities and billion-laughs defense
Add opt-in DTD processing behind Limits(allowDoctype:), off by default. When on,
DoctypeScanner reads the <!DOCTYPE> internal subset and honors internal
<!ENTITY name "value"> general entities, which EntityDecoder expands recursively
in text and attribute values.

Safety is mandatory and reimplemented from the Expat amplification-accounting
model (no upstream code copied):
- only entity-expanded characters count against a shared budget
  (Limits.maxEntityExpansion); overflow raises amplificationLimitExceeded, which
  defeats billion-laughs.
- a visiting set detects recursive entities (recursiveEntity).
- undeclared references raise undefinedEntity.
- external entities are never stored or loaded, so XXE stays closed.

The default still rejects <!DOCTYPE>. 8 DTD tests (incl. billion-laughs and the
external-entity refusal); 49 total.
2026-06-08 01:17:09 +02:00