feat(parsing): begin faithful libyaml scanner port (indent engine)

PureYAML's scanner is a hand-rolled approximation of libyaml's algorithm; its indentation and simple-key bugs are the gap between the two. Rather than keep patching the approximation, re-derive the scanner as a faithful, function-by-function Swift port of libyaml's scanner.c (parser.c to follow), so correctness is by construction. The no-C-runtime constraint is preserved: this translates the algorithm into pure Swift, it does not bind to libyaml.

First piece: the indentation engine under PureYAML.Parsing.LibYAML, a line-for-line port of yaml_parser_roll_indent and yaml_parser_unroll_indent. A block collection opens only when content is strictly more indented than the current level, which fixes the '- a:' / same-column 'b:' mis-nesting by construction. Unit-tested against that exact case (sibling vs nested vs flow-inert). Lands as a parallel implementation verified against libyaml built from the CYaml source; swapped in only once it matches the reference with no regression. docs/LIBYAML_PORT.md records the plan. No public behavior changed yet.

CHANGELOG.md

@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Begin a faithful Swift port of libyaml's scanner under
+  `PureYAML.Parsing.LibYAML`, to replace the hand-rolled approximation whose
+  indentation and simple-key bugs the test suite surfaced. The first piece is the
+  indentation engine, a line-for-line translation of `yaml_parser_roll_indent` and
+  `yaml_parser_unroll_indent`: a block collection opens only when content is
+  strictly more indented than the current level, which fixes the `- a:` followed
+  by a same-column `b:` mis-nesting by construction. The port is verified against
+  libyaml (built from the `CYaml` source) and lands as a parallel implementation,
+  swapped in only when it matches the reference with no regression. See
+  `docs/LIBYAML_PORT.md`. No public behavior changed yet.
+
 ### Fixed
 
 - Emit no document for a stream whose only content is comments or whitespace. A

Sources/Parsing/LibYAML.IndentEngine.swift

@@ -0,0 +1,74 @@
+extension PureYAML.Parsing.LibYAML {
+    /// Port of the indentation stack from `yaml_parser_t` plus
+    /// `yaml_parser_roll_indent` and `yaml_parser_unroll_indent`. libyaml tracks a
+    /// single `indent` column (the current block collection's indentation, -1 at
+    /// the document root) and a stack of the enclosing levels. A block collection
+    /// is opened only when content is strictly more indented than the current
+    /// level, which is exactly why `- a:` followed by a same-column `b:` keeps `b`
+    /// a sibling and leaves `a` an empty value.
+    struct IndentEngine {
+        /// `parser->indent`. -1 at the document root, matching libyaml.
+        private(set) var indent = -1
+        /// `parser->indents`, the stack of enclosing indentation levels.
+        private(set) var indents: [Int] = []
+        /// `parser->flow_level`; indentation is inert inside flow collections.
+        var flowLevel = 0
+        /// The token queue (`parser->tokens`) the engine appends to or, for
+        /// `roll_indent`, inserts a start token into at a recorded position.
+        private(set) var tokens: [Token] = []
+        /// `parser->tokens_parsed`, the count already dequeued, used to turn an
+        /// absolute token number into an index into the live queue.
+        var tokensParsed = 0
+
+        /// Port of `yaml_parser_roll_indent`. In flow context it does nothing.
+        /// When the column is greater than the current indent, it pushes the
+        /// current indent, adopts the new column, and emits the block start token.
+        /// `number == -1` appends; otherwise the token is inserted at the recorded
+        /// queue position so a `BLOCK-MAPPING-START` precedes a simple key that was
+        /// already queued.
+        mutating func rollIndent(
+            column: Int,
+            number: Int,
+            type: TokenType,
+            mark: PureYAML.Parsing.Mark,
+        ) {
+            if flowLevel != 0 {
+                return
+            }
+            if indent < column {
+                indents.append(indent)
+                indent = column
+                let token = Token(type: type, startMark: mark, endMark: mark)
+                if number == -1 {
+                    tokens.append(token)
+                } else {
+                    tokens.insert(token, at: number - tokensParsed)
+                }
+            }
+        }
+
+        /// Port of `yaml_parser_unroll_indent`. In flow context it does nothing.
+        /// While the current indent is deeper than the column, it emits a
+        /// `BLOCK-END` and pops the enclosing level, closing every block collection
+        /// that the dedent ends.
+        mutating func unrollIndent(column: Int, mark: PureYAML.Parsing.Mark) {
+            if flowLevel != 0 {
+                return
+            }
+            while indent > column {
+                tokens.append(Token(type: .blockEnd, startMark: mark, endMark: mark))
+                indent = indents.removeLast()
+            }
+        }
+
+        /// Dequeue the front token, advancing `tokensParsed` like libyaml's
+        /// `yaml_parser_scan` does when it hands a token to the parser.
+        mutating func dequeue() -> Token? {
+            guard !tokens.isEmpty else {
+                return nil
+            }
+            tokensParsed += 1
+            return tokens.removeFirst()
+        }
+    }
+}

Sources/Parsing/LibYAML.Token.swift

@@ -0,0 +1,10 @@
+extension PureYAML.Parsing.LibYAML {
+    /// A scanner token, mirroring `yaml_token_t`: a kind plus its start and end
+    /// marks. Token-specific payloads (scalar value, anchor name, tag) are added
+    /// as their fetchers are ported.
+    struct Token: Equatable {
+        var type: TokenType
+        var startMark: PureYAML.Parsing.Mark
+        var endMark: PureYAML.Parsing.Mark
+    }
+}

Sources/Parsing/LibYAML.TokenType.swift

@@ -0,0 +1,10 @@
+extension PureYAML.Parsing.LibYAML {
+    /// A scanner token kind, mirroring `yaml_token_type_t`. Only the structural
+    /// kinds the indentation engine produces are modelled here; scalar, anchor,
+    /// tag, and document kinds are added as their fetchers are ported.
+    enum TokenType: Equatable {
+        case blockSequenceStart
+        case blockMappingStart
+        case blockEnd
+    }
+}

Sources/Parsing/LibYAML.swift

@@ -0,0 +1,10 @@
+extension PureYAML.Parsing {
+    /// Faithful Swift port of libyaml's scanner (`CYaml/src/scanner.c`). It is
+    /// built as a parallel implementation, verified token-for-token against
+    /// libyaml through the differential harness, to replace the hand-rolled
+    /// approximation whose indentation and simple-key handling the official test
+    /// suite surfaced as incorrect. The port proceeds function by function; each
+    /// piece mirrors its C counterpart so correctness is by construction rather
+    /// than re-derived. Members live under this namespace in their own files.
+    enum LibYAML {}
+}

Tests/LibYAMLIndentEngineTests.swift

@@ -0,0 +1,58 @@
+@testable import PureYAML
+import Testing
+
+/// Verifies the ported indentation engine reproduces libyaml's `roll_indent` /
+/// `unroll_indent` behavior, in particular the rule that fixes the block-mapping
+/// bug: a block collection opens only when content is strictly more indented than
+/// the current level.
+@Suite("LibYAML Indent Engine")
+struct LibYAMLIndentEngineTests {
+    private let mark = PureYAML.Parsing.Mark.start
+
+    @Test("a same-column key after an empty value is a sibling, not nested")
+    func test_sameColumnKeyIsSibling() {
+        // `- a:` then `  b: c`: the `-` opens a sequence at column 0; the key `a`
+        // opens a mapping at column 2; `a` has no value; `b` arrives at column 2.
+        var engine = PureYAML.Parsing.LibYAML.IndentEngine()
+        engine.rollIndent(column: 0, number: -1, type: .blockSequenceStart, mark: mark)
+        engine.rollIndent(column: 2, number: -1, type: .blockMappingStart, mark: mark)
+        engine.unrollIndent(column: 2, mark: mark)
+        engine.rollIndent(column: 2, number: -1, type: .blockMappingStart, mark: mark)
+        engine.unrollIndent(column: -1, mark: mark)
+
+        // One sequence, one mapping (b is a sibling of a, not nested under it).
+        #expect(engine.tokens.map(\.type) == [
+            .blockSequenceStart,
+            .blockMappingStart,
+            .blockEnd,
+            .blockEnd,
+        ])
+    }
+
+    @Test("a more-indented value opens a nested mapping")
+    func test_moreIndentedValueNests() {
+        // `a:` then `  b: c`: the value is more indented, so it nests.
+        var engine = PureYAML.Parsing.LibYAML.IndentEngine()
+        engine.rollIndent(column: 0, number: -1, type: .blockMappingStart, mark: mark)
+        engine.unrollIndent(column: 2, mark: mark)
+        engine.rollIndent(column: 2, number: -1, type: .blockMappingStart, mark: mark)
+        engine.unrollIndent(column: -1, mark: mark)
+
+        #expect(engine.tokens.map(\.type) == [
+            .blockMappingStart,
+            .blockMappingStart,
+            .blockEnd,
+            .blockEnd,
+        ])
+    }
+
+    @Test("flow level makes indentation inert")
+    func test_flowLevelIsInert() {
+        var engine = PureYAML.Parsing.LibYAML.IndentEngine()
+        engine.flowLevel = 1
+        engine.rollIndent(column: 4, number: -1, type: .blockMappingStart, mark: mark)
+        engine.unrollIndent(column: 0, mark: mark)
+        #expect(engine.tokens.isEmpty)
+        #expect(engine.indent == -1)
+    }
+}

docs/LIBYAML_PORT.md

@@ -0,0 +1,49 @@
+# Faithful libyaml port
+
+## Why
+
+PureYAML's original scanner is a hand-rolled approximation of libyaml's
+algorithm. Its bugs (block-mapping indentation, simple keys, empty values) are
+the gap between that approximation and the proven reference. Rather than keep
+patching the approximation, the scanner is being re-derived as a faithful,
+function-by-function Swift port of libyaml's `scanner.c`, with the parser to
+follow from `parser.c`. Correctness is then by construction: each Swift function
+mirrors its C counterpart and is verified against libyaml directly.
+
+The constraint that PureYAML ships no C or generated runtime is preserved: this
+is a translation of the algorithm into pure Swift, not a binding to libyaml.
+
+## Reference and verification
+
+- Source of truth: `PureYAMLResearch/Sources/CYaml/src/scanner.c` and `parser.c`
+  (libyaml 0.2.5).
+- Oracle: the libyaml differential harness (`scratchpad/run_events.c` compiled
+  against the same `CYaml` source emits the canonical yeast event stream). Each
+  ported stage is checked token-for-token / event-for-event against it, in
+  addition to the vendored `test.event` suite ground truth.
+- The port lands as a parallel implementation under `PureYAML.Parsing.LibYAML`
+  and is only swapped in for the current scanner once it matches the reference on
+  the full suite and the real-world corpus, so the working parser is never
+  regressed mid-port.
+
+## Order (each step verified before the next)
+
+1. **Indent engine** (DONE): `yaml_parser_roll_indent` / `yaml_parser_unroll_indent`
+   and the `indents` stack. A block collection opens only when content is strictly
+   more indented than the current level, which fixes the `- a:` / same-column `b:`
+   bug by construction. Implemented in `Sources/Parsing/LibYAML.IndentEngine.swift`,
+   unit-tested in `Tests/LibYAMLIndentEngineTests.swift`.
+2. **Simple-key mechanism**: `save_simple_key`, `remove_simple_key`,
+   `stale_simple_keys`, the `simple_keys` stack and `simple_key_allowed` flag.
+3. **Token-fetch core**: `fetch_more_tokens`, `fetch_next_token`,
+   `scan_to_next_token`, and the `fetch_*` dispatch (`fetch_key`, `fetch_value`
+   with its empty-scalar and roll-indent logic, `fetch_block_entry`,
+   `fetch_flow_*`, document indicators, stream start/end).
+4. **Scalar scanners**: `scan_plain_scalar`, `scan_flow_scalar`,
+   `scan_block_scalar` (+ breaks), `scan_anchor`, `scan_tag`, `scan_directive`.
+5. **Reader**: confirm the existing `Reader` covers `CACHE`/`SKIP`/`READ` and
+   UTF-8 marks, or port the needed pieces.
+6. **Parser**: port `parser.c` to compose events from the token stream.
+7. **Integration**: route `parseEvents` / `parseStream` through the port; delete
+   the approximation once the canonical event gate and libyaml differential are
+   green with zero regressions.