From 884ff7d9a1c49eb6f2e94a595579a49634fbf753 Mon Sep 17 00:00:00 2001 From: Mihaela Mihaljevic Date: Wed, 24 Jun 2026 18:17:12 +0200 Subject: [PATCH] docs: record the libyaml port as complete and the sole parsing engine Marks every step of the port plan done and adds a status section: the port is integrated as PureYAML's only parser, verified byte-for-byte against libyaml on 307/308 cases, and scores 275/308 on the canonical event gate, ahead of libyaml's own 252 and the prior hand-rolled parser's 267. --- docs/LIBYAML_PORT.md | 40 +++++++++++++++++++++++++++++++--------- 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/docs/LIBYAML_PORT.md b/docs/LIBYAML_PORT.md index d637f52..92336fb 100644 --- a/docs/LIBYAML_PORT.md +++ b/docs/LIBYAML_PORT.md @@ -1,5 +1,28 @@ # Faithful libyaml port +## Status: complete and integrated + +The port is finished and is now PureYAML's **only** parsing engine. Every step +below has landed, the hand-rolled approximation has been deleted, and `parseEvents`, +`parseStreaming`, and `parseStreamStreaming` all run on the port. + +- **Faithfulness**: verified byte-for-byte against the libyaml reference event + dumper across all 308 vendored valid cases. The port matches libyaml on 307; the + single difference (`L24T/01`) is the port being more correct than libyaml. +- **Conformance** (canonical `test.event` gate, of 308): libyaml itself scores + 252, the old hand-rolled parser scored 267, and the port scores **275** after + closing real YAML 1.2 gaps libyaml lacks (reserved directives ignored with a + warning, newer `%YAML` minor versions, multi-line flow keys, bare documents + after `...`, tab-only blank lines, JSON UTF-16 surrogate-pair `\u` escapes, full + `ns-anchor-char` anchor/alias names). +- **Integration boundary** (`Sources/Parsing/Parser.Events.swift`): converts + libyaml's 0-based marks to PureYAML's 1-based, translates scanner/parser errors + to the diagnostic-aware `ParseError`, and enforces a 64-level nesting bound so + deep input throws `nestingTooDeep` rather than overflowing the recursive composer. + +Remaining open gaps and their clusters are tracked by `knownEventGaps` in +`Tests/CanonicalEventConformanceTests.swift`. + ## Why PureYAML's original scanner is a hand-rolled approximation of libyaml's @@ -33,17 +56,16 @@ is a translation of the algorithm into pure Swift, not a binding to libyaml. more indented than the current level, which fixes the `- a:` / same-column `b:` bug by construction. Implemented in `Sources/Parsing/LibYAML.IndentEngine.swift`, unit-tested in `Tests/LibYAMLIndentEngineTests.swift`. -2. **Simple-key mechanism**: `save_simple_key`, `remove_simple_key`, +2. **Simple-key mechanism** (DONE): `save_simple_key`, `remove_simple_key`, `stale_simple_keys`, the `simple_keys` stack and `simple_key_allowed` flag. -3. **Token-fetch core**: `fetch_more_tokens`, `fetch_next_token`, +3. **Token-fetch core** (DONE): `fetch_more_tokens`, `fetch_next_token`, `scan_to_next_token`, and the `fetch_*` dispatch (`fetch_key`, `fetch_value` with its empty-scalar and roll-indent logic, `fetch_block_entry`, `fetch_flow_*`, document indicators, stream start/end). -4. **Scalar scanners**: `scan_plain_scalar`, `scan_flow_scalar`, +4. **Scalar scanners** (DONE): `scan_plain_scalar`, `scan_flow_scalar`, `scan_block_scalar` (+ breaks), `scan_anchor`, `scan_tag`, `scan_directive`. -5. **Reader**: confirm the existing `Reader` covers `CACHE`/`SKIP`/`READ` and - UTF-8 marks, or port the needed pieces. -6. **Parser**: port `parser.c` to compose events from the token stream. -7. **Integration**: route `parseEvents` / `parseStream` through the port; delete - the approximation once the canonical event gate and libyaml differential are - green with zero regressions. +5. **Reader** (DONE): scalar-based `Reader` with `peek`/`skip`/`skipLine` and marks. +6. **Parser** (DONE): `parser.c` ported to compose events from the token stream. +7. **Integration** (DONE): `parseEvents` / `parseStream` run on the port and the + approximation has been deleted; the canonical event gate and libyaml + differential are green with zero regressions.