Commit Graph

123 Commits

Author SHA1 Message Date
b2d2d2f595 fix(parsing): accept tabs as separation, reject them as indentation
libyaml refuses to skip a tab in the inter-token whitespace whenever a
simple key is allowed, so a tab used as valid separation (after a '-',
'?', or ':' indicator, or before a flow collection) is reported as a
character that cannot start a token. But a tab is legal in-line
separation; it is only illegal as indentation.

The scanner now skips tabs in scan_to_next_token and records in
tabbedSeparation whether one preceded the pending token. A block entry,
an explicit key, or an implicit mapping key promoted by ':' is rejected
when reached through a tab (there the tab would be indentation), while a
scalar value or a flow collection is accepted. Tab-indented keys and
'-\t-' nested entries still reject.

Surpasses libyaml on 6BCT, 6CA3, A2M4, DK95/00, Q5MG, Y79Y/010.
Port score 299 -> 305.
2026-06-24 19:14:12 +02:00
8092897612 fix(parsing): parse a leading ':' flow-sequence entry as an empty-key pair
A flow sequence entry that begins with ':' (e.g. '[ : v ]') is a
single-pair mapping with an empty key, which YAML 1.2 permits but libyaml
rejects by handing the leading VALUE token to its node parser.

Open the implicit single-pair mapping on a leading ':' in the
flow-sequence-entry parser, and leave the VALUE token for the mapping
value state to consume so the value is parsed rather than dropped.
Surpasses libyaml on CFD4. Port score 298 -> 299.
2026-06-24 19:07:04 +02:00
ff199516ac fix(parsing): apply the YAML 1.2 flow ':' rule with JSON-key adjacency
libyaml makes every ':' in a flow collection a value indicator, so it
mis-scans ':x', 'http://foo.com', '::vector', and 'omitted value:,', and
drops the value after an adjacent quoted or collection key. Per the spec a
':' only ends a node when followed by whitespace, end of input, or a flow
indicator, except an adjacent ':' after a JSON key (a quoted flow scalar
or a flow collection used as the key).

Gate the ':' value-indicator dispatch on that rule, tracking on the
pending simple key whether it is a JSON node (set for quoted flow scalars
and flow-collection starts). End the plain scalar at a ':' before a flow
indicator instead of rejecting it as unexpected.

Surpasses libyaml on 4ABK, 58MP, 5T43, DBG4, FRK4, HM87/00, with 9MMW
(mixing adjacent and separated keys) still correct. Port score 292 -> 298.
2026-06-24 19:02:43 +02:00
c3a1e6aaa4 fix(parsing): treat flow '?' before a non-indicator as scalar content
libyaml makes every '?' in flow context a mapping-key indicator, so it
parses '{ ?foo: bar }' as a key 'foo' (dropping the '?') and '[?x]' as an
empty single-pair mapping. Per ns-plain-first a '?' followed by a
flow-safe character (anything but whitespace, end of input, or a flow
indicator) begins a plain scalar.

Gate the '?' key-indicator dispatch on isBlankZ or, in flow context, a
flow indicator, so '?foo' and '?x' stay plain-scalar content. Hoist the
flow-indicator predicate into CharacterClass and reuse it in the plain
scalar scanner. Surpasses libyaml on 652Z and HM87/01. Port score
290 -> 292.
2026-06-24 18:50:46 +02:00
26e09103b4 fix(parsing): accept tabs as block-scalar content past the indentation
The block-scalar break scanner raised 'found a tab character where an
indentation space is expected' for every tab seen while auto-detecting
the content indentation. But a tab at or beyond the content column is
content, not indentation: only a tab below the content column is illegal.

Gate the tab check on a content-column floor (the resolved indent, or the
parent indent + 1 while auto-detecting) instead of firing on every tab in
auto-detect mode. A tab-only line shallower than the content is still
rejected, so tab-as-indentation errors (Y79Y/000 and friends) are
preserved. Surpasses libyaml on 96NN/00, 96NN/01, R4YG, Y79Y/001.
Port score 286 -> 290.
2026-06-24 18:44:25 +02:00
254905fe67 fix(parsing): auto-detect indentation 0 for a root block scalar
The block-scalar scanner used 0 as the 'indentation not yet detected'
sentinel and floored the auto-detected indent at 1, mirroring libyaml's
scan_block_scalar_breaks. A root-level block scalar (parent indent -1)
whose content starts in column 0 therefore detected no content, and the
first content line was reparsed as a fresh document, raising 'did not
find expected document start'.

Switch the sentinel to -1 so 0 is a representable detected indent, and
floor the auto-detected indent at parent indent + 1 (0 at the document
root) instead of 1. Nested block scalars keep a parent indent >= 0, so
their floor stays >= 1 and they are unaffected.

Surpasses libyaml on DK3J, FP8R, and M7A3. Port score 283 -> 286.
2026-06-24 18:36:17 +02:00
3fc3072544 feat(parsing): parse flow mappings with empty keys
A ':' value indicator with no preceding key is also valid inside a flow
mapping, denoting an entry whose key is empty (null). libyaml hands the
bare VALUE token to its node parser and fails with 'did not find expected
node content'. The flow-mapping-key parser now emits the empty scalar key
and lets the value state consume the indicator.

Fixes '{ : }' and '{ key: value, : empty key }', completing the
multi-document NKF9 case. Port score 282 -> 283.
2026-06-24 18:31:40 +02:00
70c3da85a3 feat(parsing): parse block mappings with empty keys
A ':' value indicator with no preceding key on its line denotes a
mapping entry whose key is empty (null), which YAML 1.2 permits but
libyaml rejects with 'did not find expected key'. The block-mapping-key
parser now emits the empty scalar key and leaves the VALUE token for the
value state to consume, rather than throwing.

Fixes the canonical event streams for plain empty-key mappings, explicit
empty keys with anchors and aliases, empty keys nested in sequences, and
mixed empty-key/value entries. Port score 275 -> 282; surpasses libyaml
on 2JQS, 6M2F, M2N8/00, NHX8, S3PD, SM9W/01, UKK6/00.
2026-06-24 18:27:45 +02:00
884ff7d9a1 docs: record the libyaml port as complete and the sole parsing engine
Marks every step of the port plan done and adds a status section: the port
is integrated as PureYAML's only parser, verified byte-for-byte against
libyaml on 307/308 cases, and scores 275/308 on the canonical event gate,
ahead of libyaml's own 252 and the prior hand-rolled parser's 267.
2026-06-24 18:17:12 +02:00
18101b8ddc feat(parsing): accept full YAML 1.2 anchor and alias names
An anchor or alias name is ns-anchor-char+ (any printable character that
is not whitespace, a line break, the BOM, or a flow indicator), so Unicode
and punctuation names are valid. libyaml restricts the name to
alphanumerics and rejects them.

Canonical event conformance: 271 -> 275 of 308 (2SXE, 8XYN, W5VH, Y2GN).
379 tests pass.
2026-06-24 18:11:58 +02:00
df3aa0b5d4 refactor(parsing): delete the hand-rolled parser, leaving the ported engine
The ported libyaml scanner and parser now serve every parsing path, so the
hand-rolled scanner, token cursor, and token event parser are dead. Removes
them and the old Token/NodeResult/NodeProperties types along with the unit
tests that exercised the old scanner directly. The remaining suite drives
the ported engine through the public API.

PureYAML now has a single parsing engine. 379 tests pass.
2026-06-24 18:07:20 +02:00
0588996506 feat(parsing): run the whole library on the ported engine
Routes the value and stream APIs through the ported scanner+parser (the
event API already was), so PureYAML now parses entirely on the libyaml
port. Adds the features needed to match and exceed the previous engine:
JSON UTF-16 surrogate-pair \u escapes, bare documents after an explicit
`...`, reserved-directive handling that still requires a following `---`,
and a 64-level nesting bound enforced on the event stream so deeply nested
input fails with nestingTooDeep instead of overflowing a recursive
consumer. Parser errors are translated to the diagnostic-aware ParseError,
recovering precise codes where identifiable and a readable malformed
fallback otherwise; error-reporting fixtures are rebased onto the port.

Canonical event conformance: 268 -> 271 of 308 (7Z25, HWV9, QT73).
393 tests pass.
2026-06-24 18:02:22 +02:00
d9bf2bda26 feat(parsing): support multi-line flow mapping keys
YAML 1.2 lets a flow-context key be separated from its ':' value indicator
by line breaks, so a flow-level simple key must survive a new line (it is
still bounded to 1024 characters). libyaml invalidates every simple key on
a new line, which wrongly rejects keys like `{"foo"\n: "bar"}` and
multi-line quoted keys.

Canonical event conformance: 259 -> 268 of 308 (4MUZ/00-02, 5MUD, 9SA2,
K3WX, NJ66, UT92, VJP3/01). This passes the hand-rolled parser's 267 and
exceeds libyaml's own 252. 333 tests pass.
2026-06-24 17:46:16 +02:00
c906993f00 feat(parsing): treat tab-containing blank lines as blank
YAML 1.2 separation lines may contain tabs, so a line of only spaces and
tabs is blank. libyaml leaves a leading tab and later rejects it; the
scanner now consumes the trailing blanks of a whitespace-only line so the
break is reached and the line is skipped.

Canonical event conformance: 258 -> 259 of 308 (DK95/03). 333 tests pass.
2026-06-24 17:41:21 +02:00
6ce1f906f9 feat(parsing): surpass libyaml on reserved directives and YAML versions
YAML 1.2 §6.8.1 requires a reserved directive (anything other than %YAML
or %TAG) to be ignored with a warning, and a document whose minor version
is newer than 1.2 to be parsed with a warning. libyaml rejects both. The
scanner now drops a reserved directive's whole line and emits no token,
and the parser accepts any major-version-1 document.

Canonical event conformance: 253 -> 258 of 308 (2LFX, 6LVF, MUS6/05,
MUS6/06, BEC7). 333 tests pass.
2026-06-24 17:37:32 +02:00
561921df42 feat(parsing): route the event API through the ported libyaml engine
parseEvents now runs on the ported scanner+parser instead of the
hand-rolled approximation. Adds 1-based mark conversion at the boundary
(the port stays faithfully 0-based internally), translates the port's
scanner and parser errors into the diagnostic-aware ParseError with
precise codes (unterminatedQuotedString, unterminatedTag, tabIndentation)
and a readable malformed fallback, and retargets the canonical event gate
to track the port's gap set.

The port matches the canonical test.event on 253 of 308 valid cases;
the remaining gaps match libyaml's own behaviour and are the targets for
surpassing libyaml. Golden event and diagnostic expectations are rebased
onto the port where it is faithful or more correct (resolved verbatim
tags, libyaml's stream-end marks, reaching the real unterminated quote).
333 tests pass.
2026-06-24 17:32:29 +02:00
6eddb64dcd fix(parsing): reject a tab where a token must start, add port differential
The scanner now errors with "found character that cannot start any token"
when a tab leads a token in block context, matching libyaml's final guard
in fetch_next_token (a plain scalar may not begin with a blank). Adds a
differential harness (env-gated) that dumps the ported pipeline's yeast
events for the whole suite.

Verified against the libyaml reference dumper across all 308 vendored
valid cases: the port is byte-for-byte identical to libyaml on 307, and
the single difference (L24T/01) is the port matching the canonical
test.event where libyaml drops a trailing line break. 333 tests.
2026-06-24 17:12:36 +02:00
521062d974 feat(parsing): port libyaml parser state machine to Swift
Ports parser.c: the one-event-at-a-time state machine that turns the
scanner's token stream into the YAML event stream. Covers stream and
document framing, directive processing with %TAG handle resolution and
default handles, the node property/tag/anchor production, block and
indentless sequences, block mappings, and flow sequences and mappings
including the single-pair `?` mapping forms.

Output verified against the libyaml reference event dumper: the empty
mapping value, %TAG resolution, and shorthand tags all match byte for
byte. 331 tests, no regression.
2026-06-24 17:05:59 +02:00
ab05580518 feat(parsing): port libyaml directive scanner to Swift
Adds the %YAML and %TAG directive scanner, the final scanner piece.
Ports fetch_directive, scan_directive, scan_directive_name,
scan_version_directive_value/number, and scan_tag_directive_value,
reusing the already-ported tag handle/URI scanners. Adds
versionDirective and tagDirective token kinds.

The libyaml scanner is now fully ported: stream, document markers,
directives, block + flow structure, plain/quoted/block scalars,
anchors, aliases, and tags. 319 tests, no regression.
2026-06-24 16:54:04 +02:00
ed1904534c feat(parsing): port libyaml tag scanner to Swift
Stage 3 of the faithful libyaml scanner port, continued. Translates fetch_tag, scan_tag, scan_tag_handle, scan_tag_uri, and scan_uri_escapes: verbatim (!<uri>), shorthand (!handle!suffix), and local (!suffix) tags, producing the handle and suffix for the parser to resolve. URI %-escapes are decoded by collecting the UTF-8 octets and decoding them as a unit. Unit-tested for a secondary-handle shorthand tag, a local tag, and a verbatim tag. Only directives (%YAML/%TAG) remain unported in the scanner. 317 tests, no regression.
2026-06-24 16:48:22 +02:00
7918eefe14 feat(parsing): port libyaml block-scalar scanner to Swift
Stage 3 of the faithful libyaml scanner port, continued. Translates fetch_block_scalar, scan_block_scalar, and scan_block_scalar_breaks: literal (|) and folded (>) block scalars with the chomping (+/-) and indentation indicators in either order, indentation auto-detection from the first non-empty line (clamped to parser indent + 1), folded joining of equally-indented non-empty lines into a space, and clip/strip/keep chomping of the trailing breaks. Unit-tested for literal clip, folded join, and strip chomping. Only tags and directives remain unported in the scanner. 314 tests, no regression.
2026-06-24 16:40:18 +02:00
526abe9998 feat(parsing): port libyaml anchor and alias scanner to Swift
Stage 3 of the faithful libyaml scanner port, continued. Translates fetch_anchor and scan_anchor: an anchor (&name) or alias (*name) whose name is a run of IS_ALPHA characters, validated to be non-empty and followed by a blank, end of input, or a terminating indicator. The flow/quoted/anchor fetchers are consolidated into one indicator dispatch to stay within the complexity limit. Unit-tested for an anchored scalar value and a later alias. Remaining unported fetchers: tags, block scalars, directives. 311 tests, no regression.
2026-06-24 15:26:49 +02:00
4e7ca8fcb2 feat(parsing): port libyaml quoted-scalar scanner to Swift
Stage 3 of the faithful libyaml scanner port, continued. Translates fetch_flow_scalar and scan_flow_scalar: single- and double-quoted scalars with line-break folding (a single break folds to a space, blank lines to newlines), the single-quote '' escape, double-quoted backslash escapes (the full \0 \a \b \t \n \v \f \r \e space \" / \ \N \_ \L \P table plus \xXX/\uXXXX/\UXXXXXXXX hex), and the escaped end-of-line continuation that suppresses the break. Escapes are decoded to Unicode scalars directly rather than UTF-8 bytes. Unit-tested for the escaped single quote, the escape table including a hex unicode escape, and multi-line folding. The remaining unported fetchers are block scalars, anchors, tags, and directives. 310 tests, no regression.
2026-06-24 15:22:41 +02:00
27514466d0 feat(parsing): port libyaml flow-collection fetchers to Swift
Stage 3 of the faithful libyaml scanner port, continued. Translates fetch_flow_collection_start, fetch_flow_collection_end, and fetch_flow_entry (the [ ] { } , indicators), with the flow-level push/pop already ported. Combined with the plain-scalar scanner's flow-aware termination, the port now tokenizes flow sequences and mappings of plain scalars, and the simple-key splice promotes a flow mapping key. Unit-tested for [a, b] and {k: v}. Remaining unported fetchers: quoted/block scalars, anchors, tags, directives. 307 tests, no regression.
2026-06-24 15:16:45 +02:00
43b20ab7bc feat(parsing): port libyaml token-fetch loop (block + plain core)
Stage 3 of the faithful libyaml scanner port: the token producer. Translates scan_to_next_token, fetch_more_tokens/fetch_next_token dispatch, and the fetchers for stream start/end, document indicators, block entry, key, value (with the simple-key splice and roll-indent), plus scan_plain_scalar with its line-break folding. Reuses the ported indent engine, simple-key mechanism, character classes, and reader. Constructs not yet ported (flow collections, quoted scalars, anchors, tags, directives, block scalars) throw an explicit 'not yet ported' error rather than mis-scan; the port is not yet wired to the public API.

Milestone: the ported scanner tokenizes '- a:' followed by a same-column 'b:' as siblings (one BLOCK-MAPPING-START), matching libyaml, fixing the original block-mapping bug by construction; a more-indented value still nests. Unit-tested for the sibling case, the nested case, a simple pair, and multi-line plain-scalar folding. 305 tests, no regression.
2026-06-24 10:37:35 +02:00
7f94e17e2f feat(parsing): port libyaml input reader to Swift
Stage 3 foundation of the faithful libyaml scanner port. Translates libyaml's input buffer and position tracking (the buffer/mark fields and the PEEK/SKIP/SKIP_LINE macros) into PureYAML.Parsing.LibYAML.Reader. It iterates Unicode scalars rather than grapheme clusters so a CR and an LF are distinct, matching libyaml's byte-oriented scanning (a CR+LF pair is still one line break), and tracks the mark in libyaml's 0-based line and column convention. Unit-tested for peek/skip column tracking and LF and CRLF line breaks. This is the input layer the token fetchers consume.
2026-06-24 03:04:08 +02:00
348af8b77b feat(parsing): port libyaml character-class predicates to Swift
Stage 3 foundation of the faithful libyaml scanner port. Translates the IS_* character-class macros from yaml_private.h (IS_Z, IS_SPACE, IS_TAB, IS_BLANK, IS_BREAK, IS_BREAKZ, IS_SPACEZ, IS_BLANKZ, IS_DIGIT, IS_ALPHA, IS_HEX, IS_ASCII, IS_PRINTABLE) into PureYAML.Parsing.LibYAML.CharacterClass. libyaml inspects UTF-8 bytes; the Swift port operates on Unicode.Scalar, where the multi-byte line breaks NEL (U+0085), LS (U+2028), and PS (U+2029) are single scalars, and end of input is nil, libyaml's \0 sentinel (so isZ(nil) is true and the others are false). These predicates are what the token fetchers dispatch on. Unit-tested for end-of-input, blanks, the full break set, the alpha/digit/hex sets, and the ascii/printable boundaries.
2026-06-24 03:00:28 +02:00
9ce57e86e5 feat(parsing): port libyaml simple-key mechanism to Swift
Stage 2 of the faithful libyaml scanner port. Translates yaml_parser_save_simple_key, yaml_parser_remove_simple_key, yaml_parser_stale_simple_keys, and the flow-level push/pop (yaml_parser_increase_flow_level / decrease) onto a new PureYAML.Parsing.LibYAML.Scanner aggregate that embeds the indent engine. The simple-key mechanism is what lets a quoted scalar, a nested collection, or a property-carrying node become a mapping key when a ':' follows, and is required exactly at the block indent. Modelled faithfully: a potential key goes stale once it spans a line break or exceeds 1024 characters, and a required key that is removed or goes stale is the 'could not find expected colon' error. Unit-tested for the required/not-required, removal-error, staleness, and flow-level cases. No public behavior changed yet.
2026-06-24 02:55:07 +02:00
c3033c356d feat(parsing): begin faithful libyaml scanner port (indent engine)
PureYAML's scanner is a hand-rolled approximation of libyaml's algorithm; its indentation and simple-key bugs are the gap between the two. Rather than keep patching the approximation, re-derive the scanner as a faithful, function-by-function Swift port of libyaml's scanner.c (parser.c to follow), so correctness is by construction. The no-C-runtime constraint is preserved: this translates the algorithm into pure Swift, it does not bind to libyaml.

First piece: the indentation engine under PureYAML.Parsing.LibYAML, a line-for-line port of yaml_parser_roll_indent and yaml_parser_unroll_indent. A block collection opens only when content is strictly more indented than the current level, which fixes the '- a:' / same-column 'b:' mis-nesting by construction. Unit-tested against that exact case (sibling vs nested vs flow-inert). Lands as a parallel implementation verified against libyaml built from the CYaml source; swapped in only once it matches the reference with no regression. docs/LIBYAML_PORT.md records the plan. No public behavior changed yet.
2026-06-24 02:46:29 +02:00
190b80316b fix(parsing): emit no document for comment-only or empty streams
A stream whose only content is comments or whitespace has zero documents in YAML 1.2 (confirmed by both libyaml and the suite's test.event), but PureYAML was emitting a spurious empty null document into the event stream. Stop emitting it; the single-document value API maps an empty document stream to null, so PureYAML.parse("") is unchanged.

Found two ways: the canonical event gate (8G76, 98YD, AVM7) and, independently, differential testing against libyaml built from the CYaml source over the real-world corpus, where a comment-only values.yaml diverged. Raises exact canonical-event conformance to 267 of 308; the real-world differential agreement rises to 113 of 114. Adds a gated PureYAML yeast-event dumper used to drive the libyaml differential.
2026-06-24 02:27:38 +02:00
71e53b7a4c fix(parsing): strip trailing fold from flow plain scalars
A plain scalar inside a flow collection whose next line begins with a flow indicator (',' ']' '}') or the ':' value indicator was retaining the folded line break as a trailing space, so [\nfoo: bar\n] produced the value 'bar ' instead of 'bar'. YAML 1.2 section 7.3.3 strips trailing whitespace from a plain scalar; end the scalar at such a line without folding the break.

Derived from the plain-scalar production, not pattern-matched per case, and verified against the canonical event gate (exact test.event comparison): fixes 4FJ6, 4MUZ/02, CT4Q, LP6E, NKF9, QF4Y, VJP3/01, Y79Y/002, ZF4X, and ZK9H with zero regressions. Exact canonical-event conformance rises from 254 to 264 of 308 valid cases. Adds a gated triage diagnostic to the event gate.
2026-06-24 02:01:08 +02:00
e8987e4cfd feat(parsing): add authoritative canonical-event conformance gate
The conformance number was measured against a lossy oracle: the JSON
projection (in.json) discards anchors, tags, and scalar styles, and exists
for only about half the cases, so "passing" often meant only "parse did not
throw". Measured against the official suite's own test.event event streams,
the true exact conformance is 254 of 308 valid cases (40 emit a wrong event
stream, 14 false-reject), well below the JSON-proxy figure.

Add a canonical-event gate (Tests/CanonicalEventConformanceTests.swift) that
serializes the parser's events into the yaml-test-suite yeast format and
compares them exactly to each case's test.event, checking anchors, tags,
scalar styles, flow vs block, and explicit document markers, for every valid
case. The gate records the gap set and fails on any regression or unrecorded
fix. All 402 test.event files are vendored alongside their inputs.

To make the comparison exact, the parser Event model now records whether a
document boundary was explicit (documentStart/documentEnd carry an `explicit`
flag), distinguishing `---`/`...` from an implicit boundary. This resolves 89
cases that were previously correct but unverifiable without normalization.

No parser behavior changed; this is measurement and model fidelity. The 40
wrong-event and 14 false-reject cases are the real remaining gaps, to be fixed
against this gate.
2026-06-24 01:24:24 +02:00
ce0585f77e test(conformance): track all 402 suite cases; fix 3 regressions
The gap-only manifest stopped guarding a case once it was fixed, so later
work silently regressed previously-fixed cases (caught only by an external
full-suite run, not the gated sweep or full corpus). Replace it with a full
manifest of all 402 official cases (Tests/Fixtures/yaml-test-suite-manifest.yaml),
each carrying its conformant expectation and a passing/gap status, with every
case's files vendored. The gated sweep now asserts every passing case still
passes (the regression guard) and every gap still reproduces, so any regression
fails the build.

The honest gate-enforced count is 329/402 passing (the earlier 348 came from a
more lenient one-time oracle).

Repairs three regressions the new gate revealed: the bucket-6 tab rule
over-rejected tabs in block-scalar content and as separation before a scalar
value (root cause: the batch scanner and the incremental TokenCursor had
divergent line-start loops; unified onto a single scanStep, fixing 96NN/00,
96NN/01, R4YG, Y79Y/001, DK95/00, DK95/03, DK95/07, 6CA3, Q5MG, J3BT); an anchor
on a prior line before an alias key wrongly rejected 26DV; and a tag-only flow
key mis-parsed WZ62. Genuine tab-indentation errors (4EJS, Y79Y/000) still reject.
2026-06-24 00:16:28 +02:00
16da7248c7 fix(parsing): complete flow-collection key and empty-node handling
Implement libyaml's simple-key mechanism for flow context: the scanner records a potential key position before a plain or quoted scalar, an anchor/tag, or a [/{ opener, and a later ':' splices a mappingKey token retroactively in front of that node. This lets a quoted scalar ("k":v), a nested collection ({k}:v), or a property-carrying node (&c c: d) become a flow mapping key. In flow context ':' is always a value indicator and '?' always an explicit-key indicator regardless of the next character (the block-only following-blank lookahead is disabled), so adjacent forms and trailing '?' parse, and multi-line plain keys fold before the ':' is read. The flow parser is rebuilt around the flow_sequence_entry / flow_mapping_key / flow_mapping_value productions, composing empty scalars for absent keys and values. The streaming token cursor now buffers an open flow collection's tokens until it closes so the retroactive key splice stays valid.

Flips 10 official YAML test suite cases (348/402): 4FJ6, 8KB6, 9BXH, 9MMW, CN3R, DFF7, FRK4, NJ66, UT92, CT4Q. Flow collections used as block mapping keys (LX3P, 6BFJ) need the splice in block context and stay tracked. Updates manifest, fixtures, counts, and roadmap.
2026-06-23 23:18:59 +02:00
1875d66d46 fix(parsing): parse multi-line and block scalars at the document root
Model libyaml's parser->indent (the enclosing block-collection column, -1 at the bare root) separately from per-line indentation, via a blockIndent stack that rolls on block-sequence entries and mapping keys, unrolls at each block-line start, and resets at document boundaries. Plain-scalar continuation folding now compares against blockIndent, so a root scalar folds its column-0 continuations (single break to space, blank line to newline, per-line whitespace stripped) while a scalar inside a collection still ends at a sibling line. The ':' value indicator deliberately does not roll the block indent.

A '---' document-start marker may now be followed by the document's root node on the same line ('--- foo', '--- [1, 2]', '--- |', '--- >'), and a block scalar whose body sits at the document root's column 0 is captured and ends at the next '---'/'...' marker.

Flips 33 official YAML test suite cases (338/402): 18 false-rejects and 12 value mismatches now match, and 3 invalid inputs (9KBC, ZVH3, 9MQT/01) are now correctly rejected. M7A3 and NB6Z now parse but with mismatched values (separate multi-doc and tab-folding gaps) and stay tracked. Adds root multi-line plain scalar regression tests; updates manifest, fixtures, counts, and roadmap.
2026-06-23 22:56:33 +02:00
318913baeb fix(emitting): escape CRLF per unicode scalar in double-quoted output
A CR+LF pair forms a single Swift extended grapheme cluster, so quote()'s Character-level switch matched neither '\\r' nor '\\n' and emitted the line break unescaped. The folding parser then read the dumped value back differently, breaking parse/dump/parse stability for any string containing CRLF (surfaced as a round-trip failure on the Zoom OpenAPI corpus seed under PUREYAML_RUN_FULL_CORPUS). Iterating over unicode scalars escapes CR and LF independently. Adds a dumper regression test.
2026-06-23 22:23:40 +02:00
1de1fcf2a8 fix(parsing): reject invalid input that was wrongly accepted
Tighten error detection per the YAML 1.2 spec across four rules: (1) a tab may never supply block-structure indentation (line-start indentation, block-scalar indentation, and separation before a nested block node are rejected; a tab separating an indicator from a scalar value stays valid); (2) a ---/... document marker inside an open quoted scalar or flow collection is rejected instead of silently ending the document; (3) a %YAML/%TAG line that opens a second explicit document with no intervening ... is rejected; (4) an anchor cannot be attached to an alias node.

This repairs 4 of the 5 error-detection regressions introduced by buckets 3-5 (MUS6/01, N782, SU74, Y79Y/009); 9MQT/01 is deferred (its invalidity is in same-line content after the --- marker, which needs document-content scanning that risks the baseline). Rejects 13 accepted-invalid suite cases and fixes 2 more by value (6BCT, Y79Y/010); 305/402 passing.

BEHAVIOR CHANGE: tab-indented block structure (previously parsed permissively) is now rejected per spec; no real-world corpus seed relied on this. Updates the manifest, fixtures, counts, two affected tests (with spec justification), and the roadmap (bucket 6 marked PARTIAL).
2026-06-23 22:13:35 +02:00
b426c04bfa test(conformance): track 5 error-detection regressions from buckets 3-5
A full-suite audit found 5 yaml-test-suite error cases that were correctly rejected at the original baseline but are now wrongly accepted, introduced by the empty-node (bucket 3), anchor/alias-key (bucket 4), and directive/document-boundary (bucket 5) work. They escaped both gates because they were never in the gap manifest. Vendoring them and recording current: accepted so the sweep guards them; they are fixed in the stricter-rejection pass.

Regressions: 9MQT/01 ('...' inside a double-quoted scalar split into documents), MUS6/01 (repeated %YAML across documents), N782 (---/... inside a flow sequence), SU74 (anchored alias as a mapping key), Y79Y/009 (tab after an empty explicit key).
2026-06-23 21:37:00 +02:00
cd05976fac feat(parsing): handle directives and document-end boundaries
Three root causes: (1) a '%' begins a directive only in directive position (before document content, outside flow); everywhere else it is plain content, fixing misfires inside plain scalars, flow mappings, and block scalars. (2) %YAML accepts any 1.x (a higher minor is treated as 1.2 without error), rejects major != 1 and duplicate %YAML, and requires a following '---'; reserved and %TAG directives are accepted and ignored. A directive not followed by a document start is rejected with the new directiveWithoutDocument error. (3) An explicit '...' end marker closes the current document so following content begins a fresh document in parseStream; single-document parse still rejects multi-document input at compose time.

Tests/UnsupportedYAMLGapsFixtures.swift, StreamParsingTests, ScannerTests, and ParsingErrorTests were updated in lockstep to assert the now-correct behavior (percent-after-content is plain text; '...' then content is two documents; %YAML 1.3 is valid while 2.0 and duplicate %YAML are rejected); no test was weakened and multi-document rejection for single-document parse is still covered.

Flips 15 official YAML test suite cases (295/402); SF5V, 9MMA, B63P, H7TQ now correctly rejected. 6LVF and W4TN are reclassified to json-mismatch (blocked by other scalar gaps). Updates the manifest, fixtures, counts, and roadmap.
2026-06-23 21:31:58 +02:00
dfb498a89c fix(parsing): accept anchors, tags, and aliases as mapping keys
A mapping key is a full node: it may carry &anchor and !tag properties and may itself be an *alias. Fixes: (1) anchor/alias name scanning now stops at the first non-name character per yaml_parser_scan_anchor, so *a: reads as alias 'a' plus a mapping indicator instead of name 'a:'; (2) the block-mapping parser enters on a leading node property or alias, attaches same-line key properties to the key node, and emits alias keys directly; (3) an anchor or tag at value position with no same-line node composes an empty (null) value carrying the property, so 'a: &anchor' registers the anchor; (4) the composer registers an anchor's resolved model value (an empty anchored scalar registers null); (5) implicit keys with trailing spaces before the colon are recognized.

Flips 10 official YAML test suite cases to passing (280/402), including G9HC now correctly rejected. 2SXE no longer false-rejects but still value-mismatches on compact-mapping interpretation (a separate gap), so it is reclassified in the manifest rather than removed. Updates the manifest, fixtures, counts, and roadmap.
2026-06-23 21:07:47 +02:00
cdf5f082f8 fix(parsing): support empty (null) nodes in every position
Compose an empty scalar wherever a node is grammatically required but the next token is a structural indicator or line/stream end, mirroring libyaml's process_empty_scalar: after a tag or anchor property with no node, as an absent mapping value (block and flow), as an absent mapping key (both the plain ': v' form and the explicit '?' form), in flow empty key/value forms ({ : }, { , : v }), and for an indented block node following properties. Also fixes a pre-existing block-sequence bug that nested a sibling '-' as a child when only properties remained on the entry line.

Empty scalars resolve through the existing resolver against the empty string, so !!str yields the empty string, bare empty or !!null yields null, and an empty key yields the empty string.

Flips 20 official YAML test suite cases to passing (270/402). CT4Q no longer false-rejects but still value-mismatches on flow plain-scalar folding (a separate gap), so it is reclassified in the manifest rather than removed. Updates the gap manifest, fixtures, counts, and roadmap.
2026-06-23 20:40:40 +02:00
88b67ff0b2 fix(parsing): apply flow folding to quoted scalars
Single (') and double (") quoted multi-line scalars now apply YAML 1.2 flow folding (sections 6.5, 7.3.1, 7.3.2): trailing whitespace before a break is stripped, leading whitespace on continuation lines is stripped, a single break between non-empty lines folds to one space, and N consecutive breaks yield N-1 literal line feeds (first and last lines keep their own leading/trailing spaces as content). A double-quoted backslash at end of line suppresses the break entirely.

Complete the double-quoted escape set: add \xXX and \UXXXXXXXX hex escapes plus the previously missing \0 \a \v \e, literal backslash-space, and named breaks \N \_ \L \P (existing \uXXXX surrogate-pair handling kept). The folding state machine mirrors scanner.c yaml_parser_scan_flow_scalar; implemented in Sources/Parsing/Scalar.swift.

Flips 30 official YAML test suite cases to passing (250/402). Updates the gap manifest, fixtures, count assertions, and roadmap.
2026-06-23 20:17:56 +02:00
3db3e87d39 fix(parsing): correct block scalar folding and chomping
Rewrite literal (|) and folded (>) block scalar content to YAML 1.2 section 8.1: preserve interior blank lines, apply 8.1.3 line folding (a single break between equally indented non-empty lines folds to a space, while blank lines and more-indented lines keep their breaks), preserve tabs in content (only block-indentation columns are stripped), and support the + keep chomping indicator alongside clip and strip.

The block scalar is now scanned as a single unit (header line plus all content lines) with indentation auto-detection in Scanner.BlockScalar.swift, mirroring the reference scanner, instead of being reassembled from a lossy per-line token array.

Flips 24 official YAML test suite cases to passing (220/402). Updates the gap manifest, vendored fixtures, count assertions, and the conformance roadmap.
2026-06-23 20:01:03 +02:00
13a3859e21 test(conformance): track YAML test suite gaps
Vendor the 206 official YAML test suite cases (yaml/yaml-test-suite, data branch, commit 6ad3d2c) that PureYAML does not yet handle correctly, under Tests/Fixtures/yaml-test-suite/, with a manifest recording each gap's category and current behavior.

Add an opt-in conformance sweep (scripts/check-yaml-test-suite.sh, gated behind PUREYAML_RUN_YAML_TEST_SUITE=1) that reproduces every recorded gap, so a future fix must update the manifest. Document the measured result (196/402 passing) and the ordered plan for closing the gaps in docs/YAML_TEST_SUITE_CONFORMANCE.md.

No parser behavior changed.
2026-06-23 19:34:18 +02:00
16a47df118 docs(parsing): document why the nesting bound is necessary and sufficient 2026-06-23 11:08:30 +02:00
64a171bff4 fix(parsing): bound nesting depth so pathological input errors instead of crashing 2026-06-23 09:29:45 +02:00
5d76d184cb fix(parsing): treat leading '#' in block scalars as literal content 2026-06-23 04:00:07 +02:00
c13b7c97fd test: demonstrate yaml/json parsing and generation round-trips
Add a suite covering the four supported conversions around a shared
Codable value: parsing YAML, parsing JSON (via the same API, since YAML
1.2 is a JSON superset), generating YAML, and generating strict JSON
through Foundation's JSONEncoder. Each path asserts exact output and
round-trips back to the original value.
2026-06-08 00:13:56 +02:00
dc8d977f78 fix(license): keep LICENSE to standard MIT text so it is detected as MIT
Move the attribution notice out of LICENSE into ATTRIBUTION.md. The appended
notice dropped the file below GitHub's license-match threshold, so it showed as
Other/NOASSERTION instead of MIT, which blocks dependency adoption.
2026-06-07 20:45:59 +02:00
28cfd3839d chore(hooks): block Cursor attribution locally and in CI
Reject Cursor co-author trailers, cursor.com agent emails, and machine
attribution tells in commit messages and staged content, plus a Cursor
author or committer identity. Add a workflow enforcing the same rule
server-side on push and pull request.
2026-06-07 09:43:29 +02:00