NodetteHQ/PureYAML
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
9ce57e86e5
PureYAML/docs/LIBYAML_PORT.md
Mihaela Mihaljevic c3033c356d feat(parsing): begin faithful libyaml scanner port (indent engine) 
PureYAML's scanner is a hand-rolled approximation of libyaml's algorithm; its indentation and simple-key bugs are the gap between the two. Rather than keep patching the approximation, re-derive the scanner as a faithful, function-by-function Swift port of libyaml's scanner.c (parser.c to follow), so correctness is by construction. The no-C-runtime constraint is preserved: this translates the algorithm into pure Swift, it does not bind to libyaml.

First piece: the indentation engine under PureYAML.Parsing.LibYAML, a line-for-line port of yaml_parser_roll_indent and yaml_parser_unroll_indent. A block collection opens only when content is strictly more indented than the current level, which fixes the '- a:' / same-column 'b:' mis-nesting by construction. Unit-tested against that exact case (sibling vs nested vs flow-inert). Lands as a parallel implementation verified against libyaml built from the CYaml source; swapped in only once it matches the reference with no regression. docs/LIBYAML_PORT.md records the plan. No public behavior changed yet.
2026-06-24 02:46:29 +02:00

2.7 KiB
Raw Blame History

Faithful libyaml port

Why

PureYAML's original scanner is a hand-rolled approximation of libyaml's algorithm. Its bugs (block-mapping indentation, simple keys, empty values) are the gap between that approximation and the proven reference. Rather than keep patching the approximation, the scanner is being re-derived as a faithful, function-by-function Swift port of libyaml's scanner.c, with the parser to follow from parser.c. Correctness is then by construction: each Swift function mirrors its C counterpart and is verified against libyaml directly.

The constraint that PureYAML ships no C or generated runtime is preserved: this is a translation of the algorithm into pure Swift, not a binding to libyaml.

Reference and verification

  • Source of truth: PureYAMLResearch/Sources/CYaml/src/scanner.c and parser.c (libyaml 0.2.5).
  • Oracle: the libyaml differential harness (scratchpad/run_events.c compiled against the same CYaml source emits the canonical yeast event stream). Each ported stage is checked token-for-token / event-for-event against it, in addition to the vendored test.event suite ground truth.
  • The port lands as a parallel implementation under PureYAML.Parsing.LibYAML and is only swapped in for the current scanner once it matches the reference on the full suite and the real-world corpus, so the working parser is never regressed mid-port.

Order (each step verified before the next)

  1. Indent engine (DONE): yaml_parser_roll_indent / yaml_parser_unroll_indent and the indents stack. A block collection opens only when content is strictly more indented than the current level, which fixes the - a: / same-column b: bug by construction. Implemented in Sources/Parsing/LibYAML.IndentEngine.swift, unit-tested in Tests/LibYAMLIndentEngineTests.swift.
  2. Simple-key mechanism: save_simple_key, remove_simple_key, stale_simple_keys, the simple_keys stack and simple_key_allowed flag.
  3. Token-fetch core: fetch_more_tokens, fetch_next_token, scan_to_next_token, and the fetch_* dispatch (fetch_key, fetch_value with its empty-scalar and roll-indent logic, fetch_block_entry, fetch_flow_*, document indicators, stream start/end).
  4. Scalar scanners: scan_plain_scalar, scan_flow_scalar, scan_block_scalar (+ breaks), scan_anchor, scan_tag, scan_directive.
  5. Reader: confirm the existing Reader covers CACHE/SKIP/READ and UTF-8 marks, or port the needed pieces.
  6. Parser: port parser.c to compose events from the token stream.
  7. Integration: route parseEvents / parseStream through the port; delete the approximation once the canonical event gate and libyaml differential are green with zero regressions.