PureYAML/Sources/Parsing
Mihaela Mihaljevic 41ca600d93 feat(parsing): persist mid-token suspension and expose it as a public streaming API
Plain scalars now suspend and resume mid-token instead of rolling back to a
checkpoint and rescanning from the token's start on every chunk boundary
inside them: scanPlainScalar's loop is an explicit PlainScalarPhase state
machine that throws PlainScalarSuspended the moment a lookahead runs dry, and
scanStep() resumes it directly from Scanner.suspendedPlainScalar. Getting
this right took four rounds of bugs, each caught by the existing whole-suite
streaming differential test: checking ranDry only once at the loop's end
instead of after every peek-dependent decision, resuming a dry-interrupted
content or break run through a pre-check meant only for a fresh entry point,
trusting dispatchToken's own dry-fooled document-marker check, and a nested
blankLineAhead() lookahead independently going dry before an outer check
could catch it.

A synthetic long-scalar test (needed since the real-world benchmark fixture's
scalars are too short to exercise this) then caught a fifth, genuine bug: the
first working resumePlainScalar took the whole suspended value by parameter,
which kept a reference to its accumulated string alive for the entire resume
call and defeated copy-on-write, copying the whole scalar on every resume.
That is O(N^2) in the scalar's length, and measured 29x slower than not
suspending at all at small chunk sizes. Passing state inout directly, with
the alias broken at the call site before the call, fixed it to 3-4x, verified
genuinely linear (not just a smaller quadratic constant) by a permanent test
scaling scalar length 16x at a fixed chunk size.

Also exposes this scanner as public API for the first time: scanStep() and
everything around it were internal-only, so no consumer outside this module,
including the separate PureYAMLStreaming package that actually wants a
resumable token scanner, could call it. PureYAML.Streaming.Scanner wraps
scanStep() directly with no independent logic, translating its tokens into
public Streaming.Token/Kind/ScalarStyle types that don't leak the libyaml
port's internal naming. Proven separately from the internal scanner with its
own differential test, since a wrong case mapping in the translation layer
wouldn't be caught by a proof that never looks at the public types.

Stops short of rewiring PureYAMLStreaming's own document-boundary scanner
onto this; that is separate, cross-repo work with its own build and CI cycle.
2026-07-10 09:10:56 +02:00
..
BlockScalarChomping.swift fix(parsing): classify the keep block-scalar chomping indicator 2026-06-26 07:50:07 +02:00
BlockScalarStyle.swift
CollectionStyle.swift
Event.swift feat(parsing): opt-in located parse on the hand-rolled engine 2026-06-29 07:14:42 +02:00
EventComposer.Merge.swift refactor(parsing): share merge-key resolution between composers 2026-06-26 05:50:28 +02:00
EventComposer.swift Merge conformance/yaml-test-suite-gaps: the libyaml port becomes the engine on main 2026-07-05 00:23:48 +02:00
EventStreamComposedNode.swift feat(parsing): opt-in located parse on the hand-rolled engine 2026-06-29 07:14:42 +02:00
EventStreamComposer.Located.swift feat(parsing): opt-in located parse on the hand-rolled engine 2026-06-29 07:14:42 +02:00
EventStreamComposer.swift Merge conformance/yaml-test-suite-gaps: the libyaml port becomes the engine on main 2026-07-05 00:23:48 +02:00
EventStreamComposerMode.swift refactor(code-style): one top-level type per source file 2026-06-26 06:14:21 +02:00
EventStreamContainer.swift feat(parsing): opt-in located parse on the hand-rolled engine 2026-06-29 07:14:42 +02:00
EventStreamContainerKind.swift refactor(code-style): one top-level type per source file 2026-06-26 06:14:21 +02:00
LibYAML.CharacterClass.swift fix(parsing): treat flow '?' before a non-indicator as scalar content 2026-06-24 18:50:46 +02:00
LibYAML.IndentEngine.swift feat(parsing): add a resumable scanner increment and broaden the mutation suite 2026-07-09 23:38:39 +02:00
LibYAML.NodeProperties.swift feat(parsing): port libyaml parser state machine to Swift 2026-06-24 17:05:59 +02:00
LibYAML.Parser.Block.swift feat(parsing): parse block mappings with empty keys 2026-06-24 18:27:45 +02:00
LibYAML.Parser.Directives.swift feat(parsing): run the whole library on the ported engine 2026-06-24 18:02:22 +02:00
LibYAML.Parser.Document.swift fix(parsing): reject misplaced directives, refresh suite manifest 2026-06-24 19:51:39 +02:00
LibYAML.Parser.Flow.swift fix(parsing): parse a leading ':' flow-sequence entry as an empty-key pair 2026-06-24 19:07:04 +02:00
LibYAML.Parser.Helpers.swift feat(parsing): port libyaml parser state machine to Swift 2026-06-24 17:05:59 +02:00
LibYAML.Parser.Node.swift feat(parsing): exact scalar end positions in the located parse 2026-06-26 02:25:01 +02:00
LibYAML.Parser.swift feat(parsing): exact scalar end positions in the located parse 2026-06-26 02:25:01 +02:00
LibYAML.ParserError.swift feat(parsing): port libyaml parser state machine to Swift 2026-06-24 17:05:59 +02:00
LibYAML.ParserState.swift feat(parsing): port libyaml parser state machine to Swift 2026-06-24 17:05:59 +02:00
LibYAML.Reader.swift feat(parsing): persist mid-token suspension and expose it as a public streaming API 2026-07-10 09:10:56 +02:00
LibYAML.Scanner.Anchor.swift feat(parsing): accept full YAML 1.2 anchor and alias names 2026-06-24 18:11:58 +02:00
LibYAML.Scanner.BlockScalar.swift feat(parsing): add a resumable scanner increment and broaden the mutation suite 2026-07-09 23:38:39 +02:00
LibYAML.Scanner.Directive.swift fix(parsing): reject misplaced directives, refresh suite manifest 2026-06-24 19:51:39 +02:00
LibYAML.Scanner.Fetch.swift feat(parsing): add a resumable scanner increment and broaden the mutation suite 2026-07-09 23:38:39 +02:00
LibYAML.Scanner.FlowScalar.swift feat(parsing): add a resumable scanner increment and broaden the mutation suite 2026-07-09 23:38:39 +02:00
LibYAML.Scanner.Guards.swift feat(parsing): add a resumable scanner increment and broaden the mutation suite 2026-07-09 23:38:39 +02:00
LibYAML.Scanner.PlainScalar.swift feat(parsing): persist mid-token suspension and expose it as a public streaming API 2026-07-10 09:10:56 +02:00
LibYAML.Scanner.SkipToToken.swift feat(parsing): add a resumable scanner increment and broaden the mutation suite 2026-07-09 23:38:39 +02:00
LibYAML.Scanner.Streaming.swift feat(parsing): persist mid-token suspension and expose it as a public streaming API 2026-07-10 09:10:56 +02:00
LibYAML.Scanner.swift feat(parsing): persist mid-token suspension and expose it as a public streaming API 2026-07-10 09:10:56 +02:00
LibYAML.Scanner.Tag.swift feat(parsing): port libyaml tag scanner to Swift 2026-06-24 16:48:22 +02:00
LibYAML.ScannerError.swift feat(parsing): port libyaml simple-key mechanism to Swift 2026-06-24 02:55:07 +02:00
LibYAML.SimpleKey.swift fix(parsing): accept tabs as separation, reject them as indentation 2026-06-24 19:14:12 +02:00
LibYAML.swift feat(parsing): begin faithful libyaml scanner port (indent engine) 2026-06-24 02:46:29 +02:00
LibYAML.Token.swift feat(parsing): begin faithful libyaml scanner port (indent engine) 2026-06-24 02:46:29 +02:00
LibYAML.TokenType.swift feat(parsing): run the whole library on the ported engine 2026-06-24 18:02:22 +02:00
Mark.swift
MergeResolution.swift refactor(parsing): share merge-key resolution between composers 2026-06-26 05:50:28 +02:00
NodeKind.swift
ParseError.LibYAML.swift feat(parsing): run the whole library on the ported engine 2026-06-24 18:02:22 +02:00
ParseError.swift fix(parsing): reject a document marker inside a multi-line scalar 2026-06-27 23:20:52 +02:00
Parser.Events.swift feat(parsing): opt-in located parse on the hand-rolled engine 2026-06-29 07:14:42 +02:00
Parser.ScalarComposition.swift Merge conformance/yaml-test-suite-gaps: the libyaml port becomes the engine on main 2026-07-05 00:23:48 +02:00
Parser.swift feat(parsing): opt-in located parse on the hand-rolled engine 2026-06-29 07:14:42 +02:00
Reader.swift feat(parser): harden diagnostics and streaming 2026-06-06 07:57:48 +02:00
Scalar.swift fix(parsing): enforce the full YAML 1.2 double-quoted escape table 2026-06-27 16:19:25 +02:00
ScalarStyle.swift
TagNormalizer.swift feat(tagged): add tag-preserving validation 2026-06-05 15:35:52 +02:00
Text.swift