PureYAML/CHANGELOG.md
Mihaela Mihaljevic a11858380b
Some checks failed
Conformance / Conformance (push) Successful in 23s
Gates / Gates (push) Failing after 25s
Linux / Linux (push) Failing after 2s
macOS / macOS (push) Successful in 1m34s
No Cursor attribution / Scan commits (push) Successful in 3s
WASM / WASM (push) Failing after 1m2s
Windows / Windows (push) Has been cancelled
docs: point the links of this repository to git.aleahim.com
2026-11-04 20:50:07 +01:00

1684 lines
119 KiB
Markdown

# Changelog
All notable changes to PureYAML are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- `Tagged.Pair` now stores its key as a full node (`keyNode`), closing
issues #12 and #13: alias mapping keys (`*b : value`, yaml-test-suite
E76Z/26DV) parse on the `Tagged` path instead of throwing
`expectedScalarKey`, key anchors register and re-emit, and a key's
presentation style (a deliberately quoted `"on":`) survives dump and
reparse. `key: String` and `keyTag` remain as read-through computed
accessors (an alias key resolves to its anchored scalar's text), and
the `init(key:keyTag:value:)` convenience still builds plain string
keys, so most call sites compile unchanged. A style-less hand-built
key keeps the traditional plain-when-safe emission convention rather
than inheriting the document scalar policy. The corpus round-trip law
now compares key nodes structurally, and alias-key documents move
from the rejected column to round-tripping.
- Standard-tag typed accessors on `Tagged.Scalar` (issue #11):
`timestampValue` parses a `!!timestamp` scalar under the YAML 1.1
timestamp grammar into the new pure-Swift `Tagged.Timestamp` (calendar
components, timezone offset, and `secondsSinceEpoch` via the
days-from-civil closed form; epoch expectations in the tests are
measured against Python's independent implementation), and
`binaryValue` base64-decodes a `!!binary` scalar into `[UInt8]`,
accepting unpadded and block-scalar-wrapped content and rejecting
corrupt input whole. Yams hands back Foundation `Date`/`Data`; this
package is mechanically Foundation-free, so the bridges are one-liners
the caller owns: `Date(timeIntervalSince1970:)` and `Data(_:)`.
`!!set`/`!!omap` are deliberately skipped: `Model.Mapping` already
preserves order and duplicates without a special tag.
- `Emitting.Options` gains `canonical`, `explicitDocumentStart`,
`explicitDocumentEnd`, and `lineWidth`, completing issue #10 (the
earlier `indentWidth`/`sortKeys` slice deliberately deferred these).
Canonical mode emits the YAML-specification canonical form matching
libyaml's layout: explicit `---`, an explicit tag on every node
(carried tags on the `Tagged` path, the value's implicit tag
otherwise), double-quoted scalars, and flow collections with one
`? key : value,` entry per line; anchors and alias references survive
on the `Tagged` path. `lineWidth` wraps flow collections one entry per
line when the single-line form overflows, and folds plain block
scalars at single-space boundaries onto more-indented continuation
lines; a fold that would manufacture syntax (a segment ending in `:`,
a continuation starting with `#` or an indicator) or disturb
consecutive spaces is vetoed and the line stays long, so the value
always survives reparse. Gated by golden-output tests plus
reparse-equivalence tests for every option.
- Anchors, aliases, and per-node presentation styles now survive the
`Tagged` tree (issue #9, the feature-parity audit's biggest blocker to
switching from Yams). `Tagged.Scalar`/`.Sequence`/`.Mapping` gain
`anchor` and `style` fields (`Tagged.ScalarStyle`: plain, single/double
quoted, literal, folded; `Tagged.CollectionStyle`: block, flow), and
`Tagged.Node` gains an `alias(anchor:node:)` case carrying both the
reference and its resolution, so value-oriented consumers read through it
without a resolution step (`modelValueErasingTags` matches `parse(_:)`'s
expansion) while emitters re-emit the reference. Cycles are impossible by
construction: an anchor registers only after its node fully composes, so
a self-referential alias stays an `undefinedAlias` parse error.
- `PureYAML.dump(_ node: Tagged.Node, options:)` and `Tagged.Dumper`: emit
a tagged tree back to YAML with anchors (`&name`), alias references
(`*name`, never expanded copies), tags (`!!short` for built-ins,
`!<verbatim>` otherwise), and per-node styles that win over the
document-wide policy. Literal blocks handle keep-chomping (`|+`),
strip-chomped single lines (`|-`), leading blank lines, tab-led and
whitespace-only interior lines, and whitespace-led first lines via an
explicit indentation indicator (`|2`); folded scalars re-emit as `>` when
the folding rules invert exactly, degrading to a literal block (value
still exact) otherwise. Plain scalars re-emit their source spelling, so
`NO` resolved as a bool does not normalize to `false` and an empty null
stays empty. The round-trip law is corpus-tested over every real-world
fixture: `parseTagged(dump(parseTagged(d)))` equals `parseTagged(d)`
with exactly one documented style degradation (single-quoted content
containing line breaks re-emits double-quoted; the value is unchanged).
- Add `Emitting.Options.indentWidth` and `.sortKeys` (issue #10's first
slice), closing two of the four gaps a feature-parity audit against Yams
found in the emitter (`PureYAMLYamsComparison`'s `FEATURE-PARITY.md`).
`indentWidth` replaces every hardcoded `indent + 2` in `Dumper.swift`/
`Dumper.Mapping.swift` with `indent + options.indentWidth`; verified with
a round-trip test (dump at a non-default width, reparse, confirm the
value survives) since indentation is purely a rendering choice and must
never affect what a document means. `sortKeys` sorts a mapping's pairs by
their string key before rendering, in both block and flow style, but only
when every key in that mapping is a plain string: a mapping with any
`Model.Key.mapping`/`.sequence` key is left in insertion order regardless
of the option, since there is no well-defined lexicographic ordering
across mixed key kinds to sort by. Off by default, since insertion order
is what makes a duplicate-key-preserving `Mapping` round-trip meaningfully
in the first place.
Deliberately scoped to stop here: `width`-based line wrapping and
canonical mode (the other two gaps the audit found) are each a
correctness surface of their own (real line-folding logic for the former,
a defined canonical form for the latter) and remain open, not silently
dropped from this change's scope.
- Reject raw control characters anywhere in the input, matching libyaml's
own reader-level behavior. Found by directly comparing `PureYAML.parse()`
against Yams on a 221MB real-world OpenAPI/compose-file corpus (150
files): a genuinely public file with corrupted, mojibake-looking encoding
contains a raw C0 control character, which Yams's underlying libyaml
rejects unconditionally at the reader level ("control characters are not
allowed"), but `PureYAML.parse()` silently accepted and built a value tree
from. `PureYAML` already had a printable-character predicate matching
libyaml's `c-printable` set (`CharacterClass.isPrintable`), but it was
only ever applied narrowly to anchor names
(`LibYAML.Scanner.Anchor.swift`), never to the input generally; the
`Validation.PreflightScanner`'s separate `invalidControlCharacterColumn`
check exists but is opt-in via `validate()`, not part of what `parse()`
itself runs.
`Reader` now records the index of the first non-printable scalar in
whatever it has buffered (`init`/`append(_:)`, covering both batch and
streaming input), and `fetchNextToken()` throws once actual scanning has
reached that position (gated on `reader.index >= badIndex`, not just "one
exists"), so a real syntax error earlier in the document is not hidden
behind a control character occurring later, matching the order libyaml's
own byte-by-byte reader would surface them in. New `ParseError` case
`.invalidControlCharacter(line:column:)`, translated from the scanner
error the same way `.tabIndentation` already is.
Verified with 5 new tests (a bare control character rejected at the exact
position; a tab in scalar content, and CR/LF, both still legal, since
`isPrintable` already correctly includes them; a control character 5000
lines into a document still caught; an earlier real syntax error not
masked by a later control character) plus a full re-run of the official
yaml-test-suite conformance (still 333/333 parse-level, 208/208
value-level of comparable cases, unchanged) and the whole-suite streaming
differential (still clean), confirming this closes a real gap without
regressing anything already passing. Negligible throughput impact
measured on the existing 9.5MB benchmark fixture (within normal run-to-run
noise).
Disclosed limitation: the check is applied in `fetchNextToken()`, which
`scanStep()`'s plain-scalar resume path (`resumePlainScalar`, the
streaming API) does not route through, so a token containing a control
character can be returned to a streaming caller one token before this
fires. `scanAll()`/`PureYAML.parse()`, the primary target here (matching
what the corpus comparison actually calls), always calls
`fetchNextToken()` again before returning, so it is caught before a batch
parse completes.
- Add a public `PureYAML.Streaming` namespace (issue #4), exposing the
internal `LibYAML.Scanner.scanStep()` proven below for the first time as a
callable public API rather than a module-internal implementation detail.
Needed because `scanStep()` and everything in
`LibYAML.Scanner.Streaming.swift` were internal-only: no consumer outside
the `PureYAML` module, including the separate `PureYAMLStreaming` incubator
package that issue #4 names as this feature's actual intended user, could
call it. `PureYAML.Streaming.Scanner` (`init()`, `feed(_:)`, `finish()`,
`next() throws -> Outcome`) is a thin wrapper with no logic of its own,
translating `scanStep()`'s `ScanOutcome`/internal `LibYAML.Token` into
public `Outcome`/`Streaming.Token` types under public naming that does not
leak the libyaml port's internal type names (`Streaming.Token.Kind` mirrors
`LibYAML.TokenType`'s 22 cases 1:1; `Streaming.Token.ScalarStyle` mirrors
the scanner's internal `Parsing.ScalarStyle`, a distinct 5-case type from
the existing public `Emitting.ScalarStyle`, which is an output rendering
policy, not a scanned style; `Streaming.Token.range` reuses the existing
public `Model.SourceRange`/`SourcePosition` rather than inventing a new
position type). `Token.Kind.init(_:)`'s mapping switch is split across
three chained initializers purely to stay under the project's cyclomatic
complexity lint limit (a mechanical 1:1 case mapping over 22 cases has no
way to be less than 22-branches complex in one function); the split carries
no semantic meaning.
Proven with a new differential test (`PublicStreamingScannerTests.swift`)
distinct from the existing `StreamingScannerDifferentialTests.swift`: that
existing proof covers whether `scanStep()` itself is correct, but does not
exercise the public wrapper or its translation layer at all, so a wrong
case mapping or a dropped field there would not have been caught by it.
The new proof drives the public `Streaming.Scanner` at several chunk sizes
and asserts it matches the batch scanner's own tokens, translated through
the identical `Streaming.Token.init(_:)` the public scanner itself uses,
clean across the whole yaml-test-suite corpus.
Deliberately scoped to stop here: this makes the public API exist and
proves it correct, but does not yet rewire the separate `PureYAMLStreaming`
package's own `ResumableDocumentScanner` (currently a naive line-based
`---` heuristic scanner with no relationship to real YAML grammar) onto it.
That rewiring is real, separate, cross-repo work with its own build/test/CI
cycle, deferred to a follow-up round rather than folded in here.
- Implement persisted mid-token suspension for plain scalars (issue #4), the
deferred piece the previous entry predicted would eliminate the checkpoint
floor entirely. `scanPlainScalar()`'s loop (`LibYAML.Scanner.PlainScalar.swift`)
is now an explicit `PlainScalarPhase` state machine (`.start`,
`.consumingContent`, `.afterContent`, `.consumingBreaks`, `.afterBreaks`)
that throws `PlainScalarSuspended` (carrying its accumulated `PlainState`,
indent, start mark, and phase) the moment a lookahead runs dry, and
`scanStep()` resumes it directly from `Scanner.suspendedPlainScalar` with no
checkpoint at all, rather than rolling back and rescanning the whole token
from its start on every chunk boundary inside it.
Getting this correct took four rounds of bugs, each found by the existing
whole-yaml-test-suite streaming differential test
(`PUREYAML_RUN_YAML_TEST_SUITE=1 swift test --filter
"StreamingScannerDifferentialTests|YAMLTestSuiteGapsTests" --no-parallel`),
none of them hypothetical: (1) checking `reader.ranDry` only once at the
loop's end is not enough, since a nested lookahead
(`documentMarkerAhead()`, the `:`-followed-by-blank check, `blankLineAhead()`)
can independently go dry and produce a wrong-but-plausible answer that gets
used by the next step before an end-of-loop check would ever see the taint;
fixed by checking immediately after every peek-dependent decision, before
its result is used for anything. (2) A resume must never redo a pre-check
that only applies at a fresh entry point (the document-marker/comment check
in `.start`, the blank-or-break guard in `.afterContent`): resuming a
dry-interrupted content or break run via `.start`/`.afterContent` again can
wrongly end a scalar right at a literal `#` or reinterpret an in-progress
break run, since those positions are not guaranteed to be fresh boundaries.
Fixed with dedicated `.consumingContent`/`.consumingBreaks` phases that
re-enter the same in-progress call with no pre-check. (3) `dispatchToken()`
itself has an `isDocumentMarker` pre-check, ahead of ever calling
`fetchPlainScalar()`, that can suffer the identical dry-lookahead problem;
since nothing can undo a wrong dispatch after the fact, `scanStep()` only
trusts a suspend enough to resume it when real content was already
accumulated (`phase == .start` with an empty `state.string` retries from
scratch instead). (4) `blankLineAhead()` inside `consumePlainBreaks` is
itself a multi-character lookahead distinct from the single
already-confirmed `peek()` before it, so it can independently go dry; fixed
by checking `ranDry` right after computing its result, before a
tab-violation decision trusts it.
A synthetic worst-case test was added for this specifically because the
9.5MB real-world fixture used above has no plain scalars long enough to
exercise it (`test_measureLongPlainScalar`, one 1MB single-line plain
scalar). It caught a fifth bug, a real one, not a suspicion: the first
working version of `resumePlainScalar` took the whole `PlainScalarSuspended`
as a by-value parameter and read `var state = suspended.state` inside it,
but `suspended.startMark` being read again at the very end kept the whole
parameter (including its `state.string` reference) alive as an ordinary
borrowed value for the call's entire duration, aliasing `state.string` to
it throughout `runPlainScalarLoop` and forcing copy-on-write to fully copy
the entire accumulated scalar on every single resume; for a scalar rebuilt
across many small chunks that is O(N^2) in the scalar's length, not the
O(N) this feature exists to guarantee, and measured **29.05x** at
chunk-size 16 on the synthetic case, worse than not suspending at all.
Passing `state` `inout` directly, with the alias broken at the call site
(`scanStep()`'s new `resumeSuspendedPlainScalarIfNeeded()`, overwriting
`suspended.state` with a fresh, unrelated value immediately after copying
it out and before ever calling into `resumePlainScalar`) fixed it to
**3.67x** at the same chunk size, with no correctness regression
(whole-suite differential clean in both debug and release, full default
suite of 597 tests passing).
A single before/after data point does not by itself distinguish "fixed" from
"smaller constant, still quadratic", so a second, permanent test
(`test_longPlainScalarStaysLinear`) checks the actual complexity claim: per-
byte cost at a fixed chunk size (16) across a 16x range of scalar lengths
(250KB to 4MB), asserting the largest measured per-byte cost stays under 3x
the smallest (generous headroom over the ~1.08x actually measured; a real
O(N^2) regression would blow well past 3x as length grows). Genuinely O(N)
work should hold per-byte cost flat regardless of total length, which this
confirms rather than assumes from the single 1MB case above.
Measured honestly rather than assumed to have delivered the predicted win:
on the 9.5MB real-world fixture, the checkpoint floor and rescan-driven
tail are statistically unchanged from before this feature existed
(floor 1.75x versus the prior entry's 1.75x-1.76x; 10.74x at chunk-size 16
versus the prior entry's 11.54x, both within measurement noise across
runs). The prior entry's prediction that mid-token suspension "would
eliminate the floor too" is corrected here: it eliminates rescanning
specifically for plain scalars that themselves span a chunk boundary, but
`github-rest-api.yaml`'s plain scalars are short enough that this was
never a meaningful contributor to its measured floor in the first place,
and the floor's actual cause (a `Scanner` struct copy taken on every
`fetchNextToken()` call whenever `!isFinal`, regardless of token type,
established in the entry below) is untouched by a plain-scalar-specific
fix. The synthetic long-scalar test above is the fixture that actually
demonstrates this feature's value (a 29x-worse-than-no-suspension bug,
fixed down to 3-4x); the real-world fixture simply does not have the
shape of input this feature targets.
- Add `Tests/StreamingScannerPerformanceTests.swift` (issue #4), measuring
`scanStep()`'s overhead over `scanAll()` on the largest real-world fixture
(github-rest-api.yaml, 9.5MB) across a spread of chunk sizes, gated behind
`PUREYAML_STREAMING_PERF_MEASURE=1`. First pass ran in debug mode (release
test builds hit an unrelated `PureYAMLJSON` test-target issue) and showed a
~1.5x floor even in a control with `isFinal: true` from construction, where
`ranDry` can never be set and no rollback ever happens; fixed by skipping the
checkpoint entirely in that case, which only closed part of the gap in debug
numbers (1.49x to 1.38x).
`sample`(1) profiling of that debug build pointed at `mutating throws`
methods on multi-field structs (`consumePlainContent`, called from every
plain scalar), which is a known trigger for defensive retain/copy traffic
under `-Onone` that the optimizer removes in release. Confirmed by working
around the release-test-build issue with `-Xswiftc -enable-testing`: in
release, the `isFinal: true` control measured **1.04x**, essentially batch
parity, meaning the checkpoint-skip fix was actually a full fix for that
case, not the small, unexplained one it looked like in debug mode.
What remains splits into two distinct costs, decomposed by one more
measurement rather than assumed: feeding the whole 9.5MB file as a single
chunk (`isFinal` stays false until the true end, so at most one token in the
entire file could ever actually be rescanned) still measured 1.75x, the same
as 1,000,000-scalar chunks (1.74x, ~10 chunks) and 65536-scalar chunks
(1.74x). Real rescanning is not the floor's cause, since there is
essentially no rescanning to have caused it here. The floor is the cost of
copying the whole `Scanner` on every `fetchNextToken()` call whenever
`isFinal` is false, whether or not that copy is ever used to roll back;
since a chunked run's `isFinal` only becomes true once, on the last chunk,
that copy is taken for virtually every token in the file. Only *shrinking*
chunk sizes below roughly 4096 scalars adds a second, genuinely
rescan-driven cost on top of that floor (11.54x at 16 scalars).
Two follow-up checks, not assumed: ruled out buffer-population strategy as a
contributor (`Reader.init`'s `Array(string.unicodeScalars)` versus
`Reader.append(_:)`'s `append(contentsOf:)` measured 1.00x against each
other on the same 9.5MB string, no scanning involved), and re-profiled with
`sample`(1) on the release build this time (the first profile, in the entry
above, was debug-only and already shown to be misleading for magnitude; it
turns out to also be misleading for location). The release profile's
hottest symbols are String-building-adjacent
(`_StringGuts.append`, `UnicodeScalarView.distance(from:to:)`,
`consumePlainContent`) rather than a separately named "copy of Scanner"
symbol, most likely because the optimizer inlines the checkpoint's struct
copy directly into `scanStep()` (190 samples on its own, disproportionate to
its few lines of source) rather than outlining it as in debug, and the extra
live reference the checkpoint holds to `indentEngine.tokens` forces
conservative uniqueness handling that shows up as slower string mutation
nearby rather than as its own line item.
The structural cause, confirmed by an A/B the project already had (not a
new one): the checkpoint code path is still the only difference between the
1.04x `isFinal: true` control and the 1.75x-1.76x single-chunk/large-chunk
results, both driven through the identical per-token loop. This corrects the
previous entry's conclusion in an important way: the deferred persisted mid-
token suspension would not merely eliminate the rescan-driven tail while
leaving the floor untouched, it would eliminate the floor too, since it
removes the need for a checkpoint entirely rather than making one cheaper.
A cheaper checkpoint, absent mid-token suspension, is not yet designed:
every field the checkpoint covers can be mutated before a dry point is
reached within a single fetch, so no smaller subset of fields is obviously
safe to skip, and `isFinal: true` remains the only provably safe condition
for skipping it outright.
- Broaden the real-seed mutation property suite (issue #6) with three mutation
classes named in that issue but not yet covered: `invalidControlCharacter`
(a raw control byte injected into a line), `mixedTabSpaceIndentation` (a tab
after leading spaces, distinct from the existing pure-tab case), and
`truncatedFlowCollection` (an unterminated `[a, b, c` appended after the real
seed sample, exercising the deep parser's `malformed` error rather than the
preflight scanner). The test now also asserts diagnostic quality, not just
pass/fail: every matching diagnostic's `line` must fall within the mutated
document, and its `column`, when present, must be 1-based. Some `ParseError`
cases (`.tabIndentation`) are line-only by design, so column is checked when
present rather than required.
- Add a resumable, one-token-at-a-time scanner entry point, `Scanner.scanStep()`
(internal), the first increment of true chunked input streaming (issue #4).
`Reader` gains `isFinal`/`append(_:)`/`markFinal()`: past the buffered end while
`!isFinal`, `peek(_:)` flags `ranDry` instead of asserting true end of input.
`scanStep()` snapshots the scanner before each `fetchNextToken()` and restores it
in full whenever a fetch runs dry, so a fetch spanning a chunk boundary is
retried from scratch after more input arrives rather than misread as an early
end of input. Every existing call site keeps `isFinal: true` (the default), so
`scanAll()` and all batch parsing are unchanged; `ranDry` can never be set when
`isFinal` is true, by construction, not just by test.
A token already queued is not always safe to release: libyaml's simple-key
mechanism can still splice a KEY token in front of it once a later `:` is found,
so `scanStep()` ports libyaml's `yaml_parser_fetch_more_tokens` release
condition, holding the queue front until no possible simple key still targets
that position.
Proof: `Tests/StreamingScannerDifferentialTests.swift` drives `scanStep()` at
several chunk sizes (1, 2, 3, 7, 64, whole-input) over representative cases
always, and over the entire yaml-test-suite manifest (402 cases) behind
`PUREYAML_RUN_YAML_TEST_SUITE=1`, asserting the resulting token sequence (or
thrown error) is byte-identical to `scanAll()`.
That proof surfaced a real bug, now fixed: `lastTokenForbidsInlineSequence()`
(`LibYAML.Scanner.Guards.swift`) read `indentEngine.tokens.last?.type` to check
whether a `key: - a` value indicator bars an inline block sequence. Batch mode
never dequeues, so the queue never empties and `.last` always works; `scanStep()`
releases and discards tokens as soon as they are safe, so the queue can be empty
exactly when this check runs. `IndentEngine` now tracks `lastToken` explicitly,
kept in sync with `tokens.last` after every enqueue/insert/roll/unroll, which is
provably identical to the old `tokens.last` read in batch mode (nothing is ever
dequeued there) and survives dequeuing in streaming mode.
Not yet wired into `PureYAMLStreaming`: `scanStep()` and its supporting types are
internal, and true mid-scalar suspension (a plain/block/flow scalar spanning a
chunk boundary currently re-scans from the last released token rather than
resuming mid-token) is deferred, tracked at
<https://git.aleahim.com/NodetteHQ/PureYAML/issues/4>.
### Changed
- The validation framework's public types are renamed to match the
OpenAPIKit validation idiom's canonical names: `Check` is now
`Validation`, `Issue` is `ValidationError`, `Issue.Collection` is the
top-level `ValidationErrorCollection` (with `.issues` renamed to
`.values`), `SubjectContext` is `ValidationContext`, `BuiltinCheck` is
`BuiltinValidation`, and `AnyCheck` is `AnyValidation`. `ValidationError.description`
drops its severity prefix and renders `"<reason> at root of document"` /
`"<reason> at path: <path>"`. `Validator` is now a reference-type
`final class` with a two-tier `nonReferenceDefaultValidations`/
`referenceDefaultValidations` split, generic over both the walked
`Subject` type and the cross-cutting `Document` via a new `Traversable`
protocol; `Tagged.Validator` is now a type alias over the same shared
class instead of its own duplicated traversal implementation. The JSON
representability rule set (`PureYAMLJSON`) is built with the same tiered
constructor. Adds the `unwrap` combinator.
- `parseTagged(_:)`/`parseTaggedStream(_:)` now preserve alias references
as `alias` nodes instead of expanding them into copies of the anchored
node, and populate the new `anchor`/`style` fields from the source.
Consumers switching exhaustively over `Tagged.Node` gain a case;
`Tagged.NodeKind` gains `.alias`. Resolved values are unchanged:
`modelValueErasingTags` and `Tagged.Constructor` read through aliases.
`Tagged.Validator` walks each anchored subtree once at its definition
site and does not re-walk it per alias reference.
### Fixed
- A plain scalar that starts with `0` and continues with a sign is text, not an
octal integer. `Int(_, radix:)` accepts a leading sign, so `0-3` (a CPU set, a
port range) read as the integer -3, and `0x-1`, `0b-1` and `0o-1` as negative
numbers. After a radix prefix only digits may follow. Octal (`0755`),
hexadecimal, binary and signed integers read as before.
- A string the emitter wrote without quotes could be read back as something
else, so the string `yes` (and `no`, `on`, `off` in every case), `0x1F`,
`0b1`, `0o17`, `1_000`, `.inf` and `.nan` did not survive a dump and parse.
`PureYAML.Parsing.PlainScalar` is now the one definition of how a plain
scalar reads, and the parser and the emitter both take it from there; the
emitter quotes every string that does not read back as itself. Also quoted
now: `<<` (the merge key) and `...` (the document end marker), and a string
with a CR+LF break, which was written raw because Swift counts CR+LF as one
character. `PlainScalarRoundTripTests` holds 128 awkward strings at the
root, as a mapping key and value, and as a sequence item, in block and flow
style. `PureYAML.JSON.Convert` no longer refuses `{"s":"yes"}`.
- Stream dump no longer doubles the `---` document marker under
`explicitDocumentStart` or `canonical`. The stream path now owns marker
emission and renders each document body marker-free, so a two-document
stream reparses as two documents instead of four (an empty document
ahead of each real one).
- `Tagged` flow-mapping keys now use flow-aware plain-scalar safety, so a
key containing a flow delimiter (`a,b`, `[`, `{`) is quoted instead of
emitted bare, which previously reparsed as several entries.
- Plain scalars ending in `:` now fall back to quoting on both the model
and `Tagged` emit paths. A trailing colon reparses as a mapping key
with an empty value, so the emitter must never leave one plain.
- A mapping key that is an alias to a defined but non-scalar anchor now
raises `expectedScalarKey` instead of the misleading `undefinedAlias`;
a genuinely undefined alias key still raises `undefinedAlias`.
- `Tagged` duplicate-key detection now keys on the resolved scalar value
plus effective tag rather than the source spelling, so `0x1` and `1`
(both `int(1)`) are reported as duplicates while `1` (`int`) and `"1"`
(`str`) are not, and custom-tagged keys stay distinct.
- Dumping a value nested deeper than the emitter's nesting limit (the
same bound the parser enforces) now returns a documented, reparseable
marker document instead of overflowing the stack. Such a value cannot
round-trip through PureYAML's own parser, so it is refused rather than
emitted as YAML the parser would reject.
- `Tagged.Timestamp` accepts a leap-second value of `60` per the YAML 1.1
timestamp grammar, treating it as 60 elapsed seconds in the epoch math.
- `Tagged.Dumper` honors `sortKeys` in its block and canonical paths for
mappings whose keys are all plain, untagged, unanchored string scalars,
matching the model dumper's `orderedPairs` rule.
- `parse`/`parseStreaming` now reject an anchor-expansion ("billion
laughs") bomb instead of accepting it in milliseconds and handing a
caller a value whose logical size is exponential in the alias nesting
depth. `enforceNestingLimit` bounds physical collection depth in the
raw event stream, which does nothing against a handful of shallow,
small anchors each aliased many times with each alias nested inside
the next: a 25-level, fan-out-9 document (~1.5KB) expands past 9^25
logical nodes while raw event depth stays at 2. `EventStreamComposer`
now charges a total expanded-node budget incrementally as the document
streams through -- every scalar, alias resolution, and completed
container charges its own marginal cost, with each anchor's expanded
size cached at definition and looked up in O(1) when later aliased --
so a bomb is rejected within a few alias levels without ever
materializing or walking the exponential tree it would otherwise
produce. A document with genuinely wide but non-exponential alias
reuse (the same anchor referenced thousands of times at one level, no
nesting) still parses normally.
- `TaggedNodeCorpusRoundTripTests` now normalizes `URL.lastPathComponent`
through an explicit `as String?` coercion before use: this toolchain's
swift-corelibs-foundation returns `String?` where Darwin's Foundation
returns `String`, and the mismatch failed the Linux build outright
(an unrelated pre-existing gap surfaced while verifying the
anchor-expansion fix above on the Linux CI relay).
## [0.2.0] - 2026-07-05
### Changed
- Merge the libyaml-port line into main. The ported engine is the sole parser;
the hand-rolled `Scanner`/`TokenEventParser` files and their post-fork
patches are removed. Adaptations made in the merge: the JSON-representability
rules move from the retired legacy `Rule` type onto the Check algebra
(`BuiltinCheck.mappingKeysAreStrings`, `BuiltinCheck.numbersAreFinite` in
`PureYAMLJSON`); the permissive tag-handle pre-pass
(`Scanner.tagHandleIssues`) is removed because the ported engine rejects an
undefined handle at parse time (official suite case QLJ7), so the validation
report carries it as a parse diagnostic; and tests pinned to old-engine
behavior or error text were updated to the reference-verified behavior
(libyaml rejects an explicit-key continuation ending at a value colon, and
suite case QB6E requires rejecting a quoted continuation at column 0).
The unreleased entries below come from two parallel lines of work, merged
in one step: the libyaml-port line (listed first), whose ported engine is
now the sole parsing engine, and the earlier hand-rolled-engine hardening
line (listed second). Engine-internal file references in the second group
are historical; the behaviors they describe are carried by the port and
guarded by the whole-suite conformance gate.
### Changed
- Refresh `docs/YAML_TEST_SUITE_CONFORMANCE.md` to match the manifest: the
headline is 402 passing, 0 gap; the "remaining gaps" narrative is retired; a
closing section records the libyaml-port cutover and the final fix series,
and the manifest is named as the source of truth over the prose.
### Added
- The located parse now reports an exact end position for scalars, not just a start: the
scalar event carries the scanner token's end mark, so a `LocatedValue.scalar`'s range covers
the whole token (end exclusive, one past the last character) instead of being start-anchored.
Collections already spanned open to close. The event dump compares value and style, not marks,
so conformance is unaffected; the whole-suite gates stay green.
- Add an opt-in located parse, `PureYAML.parseLocated(_:)` and
`PureYAML.parseStreamLocated(_:)`, returning a `Model.LocatedValue` tree parallel
in shape to `Value` where every node carries its source span (1-based line and
column) and located children. Collections span from their opening to closing
event, scalars carry a start-anchored span, and aliases and merge-inherited pairs
point at their reference site. The plain `Value`, its equality, and the
conformance oracle are untouched: locations are built only when the streaming
composer is asked for them, so the default value path is byte-identical. A
whole-suite test proves the located projection equals a plain parse for every
passing case, including through merge keys and aliases.
- Begin a faithful Swift port of libyaml's scanner under
`PureYAML.Parsing.LibYAML`, to replace the hand-rolled approximation whose
indentation and simple-key bugs the test suite surfaced. The first piece is the
indentation engine, a line-for-line translation of `yaml_parser_roll_indent` and
`yaml_parser_unroll_indent`: a block collection opens only when content is
strictly more indented than the current level, which fixes the `- a:` followed
by a same-column `b:` mis-nesting by construction. The port is verified against
libyaml (built from the `CYaml` source) and lands as a parallel implementation,
swapped in only when it matches the reference with no regression. See
`docs/LIBYAML_PORT.md`. No public behavior changed yet.
### Fixed
- Reject an under-indented or tab-indented continuation in a multi-line quoted
scalar. A quoted scalar's continuation must be indented past the parent block, so
`"a`\n`b"` (continuation in column 0) and `"bar`\n`\tbaz"` (tab indentation) are
malformed. The error is deferred until the closing quote is found, so an
unterminated quote still reports the missing quote (its root cause) rather than
the indentation symptom. Surpasses libyaml on QB6E and DK95/01, bringing
whole-suite conformance to 402/402.
- Reject a tab used to indent flow-collection content. A tab is never indentation,
so flow content reached through a tab below the continuation indent (`- [`\n
`\tfoo`) is malformed, even though the tab inflates the column. A tab at or past
the indent stays separation, and a tab-only blank line stays blank. Surpasses
libyaml on Y79Y/003.
- Reject a block scalar whose leading empty line is more indented than its first
content line. libyaml auto-detects the deeper indentation and silently drops the
content; the auto-detection now errors when the content line is less indented
than an earlier empty line. Surpasses libyaml on S98Z.
- Reject a multi-line implicit key in a flow sequence. A flow-sequence single-pair
implicit key is single-line (`c-s-implicit-json-key`), so its `:` must sit on the
key's line: both `[ "key"\n :value ]` and `[ key\n : value ]` are malformed. A
flow MAPPING adjacent value may still span lines (`{ "foo"\n :bar }` stays
valid), so the rejection is scoped to flow levels that are sequences, tracked
alongside the flow-level stack. The key/`:` line comparison catches the plain-key
form too, which folds its line break. Surpasses libyaml on ZXT5 and DK4H.
- Reject a block sequence that opens on the same line as an earlier node. A `-`
entry mid-line after an inline mapping value (`key: - a`) or a node property
(`&anchor - x`) is invalid, while a `-` after another `-` stays a valid compact
sequence (`- - a`) and an explicit key's value on a later line (`? k`\n`: - a`)
stays valid. The inline case is told apart by a pending simple key (an explicit
key's has gone stale across the line break). Surpasses libyaml on 5U3A and SY6V.
- Reject a flow-collection continuation line that is not indented past the parent
block. YAML requires multi-line flow content to be more indented than the
enclosing block context, so `flow: [a,` / `b]` (with `b` in column 0) and the
equivalent multi-line flow mapping are malformed; libyaml does not enforce it. A
closing `]`/`}` or a leading `,` may still sit at any indentation. Surpasses
libyaml on 9C9N and VJP3/00.
- Reject content after a `...` document-end marker on the same line. Only
whitespace and a comment may follow `...`; unlike `--- x`, a `... x` cannot
carry inline content. Surpasses libyaml on 3HFZ.
- Resolve a scalar with the non-specific `!` tag as a string. The `!` tag forces
resolution by kind alone, with no implicit typing, so `! 12` is the string "12"
rather than the integer 12. The typed value now honours it (the event stream
already did). Surpasses libyaml on S4JQ.
- Reject a bare `-` before a flow indicator inside a flow collection. Per
`ns-plain-first` a `-` begins a plain scalar in flow only when a flow-safe
character follows; `[-]` and `[-, -]` are malformed, not a sequence of `-`
scalars. libyaml reads the `-` as a scalar. Surpasses libyaml on G5U8 and YJV2.
- Reject a comment that is not preceded by whitespace. A `#` only begins a comment
after a space, a tab, or at the start of a line; a `#` adjacent to the previous
token (`key: "value"# c`, `[ a ]#c`, `[ a,#c`, a `>#c` block scalar header) is
malformed. The check uses the character immediately before the `#`, since a plain
scalar consumes its own trailing blanks. libyaml accepts the adjacent form.
Surpasses libyaml on 9JBA, CVW2, SU5Z, and X4QW. Whole-suite conformance is now
388/402.
- Reject a directive that follows a document without an explicit `...` end marker.
Directives apply to the document that follows, so the previous document must be
closed with `...` before them; after an implicit end a `%YAML`/`%TAG`/reserved
directive is now an error. Also reject a `%YAML` version followed immediately by
`#` with no separating space (`%YAML 1.1#...`), since a comment requires leading
separation. Restores reject-side conformance lost when the libyaml port became
the sole engine (9HCY, EB22, MUS6/00, MUS6/01, RHX7).
- Refresh `yaml-test-suite-manifest.yaml` (now version 3): the ported engine's
whole-suite conformance is 384/402, up from the recorded 331, after the
canonical-event and reject-side work. 53 cases the manifest still marked as gaps
now pass.
- Keep the final line break of a block scalar that ends at end of input without a
trailing newline. A whitespace-only last line still counts as one line break
under keep (`+`) chomping, so `- |+` followed by a spaces-only unterminated line
yields `"\n"` rather than `""`. With this the ported engine emits the exact
canonical event stream for every valid case in the vendored YAML test suite
(308/308), surpassing libyaml (252) and the previous hand-rolled parser (267).
Surpasses libyaml on JEF9/02.
- End a root-level block scalar at a `---` or `...` document marker in column 0.
Now that a root block scalar can take indentation 0, its content loop reaches
column-0 lines, so a marker line must end the scalar instead of being folded
into it. Without this a `--- |` literal swallowed the following `...`, the next
directives, and later documents. Surpasses libyaml on W4TN.
- Fold over a tab-only blank line inside a multi-line plain scalar. A plain
scalar may span blank lines, and a blank line that contains a tab is still
blank, so the tab is not indentation. libyaml rejects any tab below the scalar
indentation; the scanner now rejects it only when it indents real content, not
when the rest of the line is blank. Surpasses libyaml on DK95/04.
- Accept a tab as in-line separation while still rejecting it as indentation. A
tab after a `-`, `?`, or `:` indicator, or before a flow collection, is valid
separation, but libyaml refuses to skip it whenever a simple key is allowed and
reports "found character that cannot start any token". The scanner now skips
tabs in the inter-token whitespace and records whether one preceded the token;
a block indicator or an implicit mapping key reached through a tab is rejected
(the tab would be indentation), while a scalar value or flow collection is
accepted. So `- foo:\tbar`, `-\t-1`, `: -\tb`, `\t[ ]`, `\t{}`, and a
space-then-tab continuation parse, while tab-indented keys and `-\t-` style
nested entries are still rejected. Surpasses libyaml on 6BCT, 6CA3, A2M4,
DK95/00, Q5MG, and Y79Y/010.
- Parse a flow sequence entry that begins with `:` as a single-pair mapping with
an empty key (`[ : v ]`). libyaml hands the leading VALUE token to its node
parser and fails. The flow-sequence-entry parser now opens the implicit mapping
on a leading `:` and leaves the indicator for the value state, so the value is
parsed rather than dropped. Surpasses libyaml on CFD4.
- Apply the YAML 1.2 flow `:` rule instead of libyaml's "always a value
indicator in flow". In a flow collection a `:` only ends a node when it is
followed by whitespace, the end of input, or a flow indicator, or when it is
adjacent to a JSON key (a quoted scalar or a flow collection used as the key).
Otherwise the `:` is plain-scalar content. So `[:x]` and `{x: :x}` keep the
scalar `:x`, `http://foo.com` and `::vector` stay single scalars, `"key":value`
and `{JSON: like}:adjacent` bind the value across the adjacent `:`, and
`omitted value:,` is a key with an empty value rather than a scan error. The
pending simple key now records whether it is a JSON node, and the plain scalar
scanner ends (rather than rejecting) at a `:` before a flow indicator. Surpasses
libyaml on 4ABK, 58MP, 5T43, DBG4, FRK4, and HM87/00, with 9MMW (mixed
adjacent/separate keys) still correct.
- Treat `?` in a flow collection as plain-scalar content unless it is followed by
whitespace, the end of input, or a flow indicator. libyaml makes every `?` in
flow context a mapping-key indicator, dropping the `?` from `?foo`, but
`ns-plain-first` lets a `?` followed by a flow-safe character begin a plain
scalar. So `{ ?foo: bar }` keys on `?foo` and `[?x]` is the scalar `?x` rather
than an empty single-pair mapping. Surpasses libyaml on 652Z and HM87/01.
- Accept a tab as block-scalar content once the indentation is satisfied. The
block-scalar break scanner rejected any tab seen while auto-detecting the
content indentation, but a tab at or beyond the content column is content, not
indentation. The tab is now an error only below the content column (the parent
indent + 1 floor while auto-detecting), so `foo: |-` / `bar` lines whose
content begins with a tab parse, while a tab used as indentation (a tab-only
line shallower than the content) is still rejected. Surpasses libyaml on
96NN/00, 96NN/01, R4YG, and Y79Y/001.
- Auto-detect indentation 0 for a root-level block scalar. The block-scalar
scanner used 0 as the "indentation not yet detected" sentinel and floored the
detected indent at 1 (as libyaml does in `scan_block_scalar_breaks`), so a
`--- >` or `--- |` whose content begins in column 0 detected no content and the
following line was misread as a new document ("did not find expected document
start"). The sentinel is now -1, leaving 0 a representable detected indent, and
the floor is the parent indent + 1 (0 at the document root). Surpasses libyaml
on DK3J, FP8R, and M7A3.
- Parse block mappings with empty (null) keys. A `:` value indicator with no
preceding key on its line denotes a mapping entry whose key is empty, which
YAML 1.2 permits but libyaml rejects with "did not find expected key". The
block-mapping-key parser now emits the empty scalar key and lets the value
state consume the indicator, fixing the canonical event streams for `: a`
style mappings, explicit empty keys (`? &a a` / `: *a`), empty keys nested in
sequences (`- :`, `- ? : x`), and mixed empty-key/value entries. Surpasses
libyaml on these seven test-suite cases (2JQS, 6M2F, M2N8/00, NHX8, S3PD,
SM9W/01, UKK6/00).
- Parse flow mappings with empty (null) keys. The same `:`-without-a-key entry
is valid inside `{ ... }`, where libyaml hands the bare VALUE token to its node
parser and fails with "did not find expected node content". The flow-mapping-key
parser now emits the empty scalar key for a leading `:`, so `{ : }` and
`{ key: value, : empty key }` parse, completing the multi-document NKF9 case.
- Emit no document for a stream whose only content is comments or whitespace. A
comment-only or empty stream has zero documents in YAML 1.2 (matching libyaml
and the test suite), but the event stream was emitting a spurious empty null
document. Found by differential testing against libyaml over real-world YAML
(a comment-only `values.yaml`) in addition to the canonical event gate. The
single-document value API still maps an empty stream to `null`, so
`PureYAML.parse("")` is unchanged. Fixes 8G76, 98YD, and AVM7, raising exact
canonical-event conformance to 267 of 308.
- Strip the trailing folded line break from a flow-context plain scalar. A plain
scalar inside `[...]`/`{...}` whose next line begins with a flow indicator
(`,` `]` `}`) or the `:` value indicator was retaining the line break folded to
a trailing space (for example `[\nfoo: bar\n]` produced the value `"bar "`),
which YAML 1.2 section 7.3.3 forbids. Derived from the plain-scalar production
and verified against the canonical event gate: fixes 10 official YAML test suite
cases (4FJ6, 4MUZ/02, CT4Q, LP6E, NKF9, QF4Y, VJP3/01, Y79Y/002, ZF4X, ZK9H)
with no regression, raising exact canonical-event conformance to 264 of 308.
### Added
- Add an authoritative canonical-event conformance gate. The parser's event
stream is serialized into the official YAML test suite event (yeast) format and
compared exactly to each case's vendored `test.event`, which checks anchors,
tags, scalar styles, flow vs block, and explicit document markers for every
valid case (including the roughly half with no `in.json` projection). This
replaces the JSON-projection oracle, which was lossy: measured against ground
truth, 254 of 308 valid suite cases emit exactly correct events (40 emit a
wrong event stream, 14 false-reject), where the JSON proxy had reported a higher
figure. The gate records the gap set and fails on any regression or unrecorded
fix. All 402 cases' `test.event` files are vendored alongside their inputs.
### Changed
- Record whether a document boundary was explicit in the parser `Event` model
(`documentStart`/`documentEnd` now carry an `explicit` flag), so a `---`/`...`
marker is distinguished from an implicit boundary. This is required for exact
canonical-event comparison and resolves 89 cases that were previously correct
but unverifiable.
### Fixed
- Make the YAML test suite conformance gate comprehensive and repair the
regressions it revealed. The gap-only manifest (which deleted a case once fixed,
so nothing guarded it afterward and later work could silently regress it) is
replaced by a full manifest of all 402 official cases
(`Tests/Fixtures/yaml-test-suite-manifest.yaml`), each with its `id`, `name`,
`localPath`, conformant `expected` behavior, `oracle`, `byteCount`, `lineCount`,
and a `status` (`passing` | `gap`). Every case's files are vendored under
`Tests/Fixtures/yaml-test-suite/`. The gated sweep
(`PUREYAML_RUN_YAML_TEST_SUITE=1`) now asserts every `passing` case still passes
and every `gap` still reproduces, so a regression in any passing case fails the
sweep. Result at the pinned commit: **329 of 402 passing, 73 gap.**
- Stop over-rejecting tabs. A tab is invalid only as block-structure indentation
(`scanner.c` `scan_to_next_token` / `scan_block_scalar_breaks` /
`scan_plain_scalar`). Block indentation is now measured in spaces only; a tab is
left for the token scanner, which allows it inside block-scalar content once a
leading space fixes the content column (96NN/00, 96NN/01, R4YG, Y79Y/001), as
separation before a scalar value (DK95/00) or a flow node (6CA3, Q5MG, J3BT), and
on a whitespace-only blank line (DK95/03, DK95/07), while still rejecting genuine
tab indentation (4EJS, Y79Y/000, Y79Y/004-008, DK95/06). The batch scanner and the
incremental token cursor were unified onto a single `scanStep`, since their
divergence previously let the streaming parser miss a tab error the batch scanner
caught.
- Restore anchors/aliases as mapping keys for `26DV`: an anchor on a `key:` value
line followed by an indented `*alias :` key now anchors the nested mapping rather
than the alias key (which cannot carry properties); `&b *alias` on one line is
still rejected (SU74).
- Restore flow bare-tag/anchor keys for `WZ62` (`{ foo : !!str, !!str : bar, }` to
`{"foo":"", "":"bar"}`): a flow key introduced by a spliced `mappingKey` uses the
same key parser as the implicit form, so a bare tag before the `:` composes an
empty tagged scalar key instead of a mapping-as-key.
- Complete flow-collection (`[...]`, `{...}`) parsing for keys, empty nodes, and
multi-line scalars, fixing 10 official YAML test suite cases and removing them
from the conformance gap manifest. The scanner now implements libyaml's
simple-key mechanism inside flow context: a potential key position is recorded
before a plain or quoted scalar, an anchor or tag, or a nested `[`/`{` opener,
and a later `:` value indicator splices a `mappingKey` token in front of that
node so a collection or property-carrying node can be a flow-mapping key. In
flow context a `:` is now always a value indicator and a `?` always an explicit
key indicator regardless of the following character (matching libyaml's
`flow_level || IS_BLANKZ_AT` checks), so `"k":v`, `{k}:v`, and a trailing `?`
parse correctly; the implicit-key forward lookahead is restricted to block
context. A multi-line plain key folds in full before its `:` is read. The flow
parser was rebuilt around libyaml's flow productions (`flow_sequence_entry`,
`flow_mapping_key`/`value`), composing empty scalar keys and values where
absent. The streaming token cursor holds an open flow collection's tokens until
it closes so the retroactive key splice stays valid. Fixed false-rejects 4FJ6,
8KB6, 9BXH, 9MMW, CN3R, DFF7, FRK4, NJ66, UT92 and corrected values for CT4Q.
The flow-node-as-block-mapping-key cases LX3P and 6BFJ remain tracked (they need
the block-context simple-key splice, a separate change).
- Parse multi-line plain scalars and block scalars at the document root, fixing 35
official YAML test suite cases and removing them from the conformance gap
manifest. The scanner now tracks the enclosing block-collection indent
(libyaml's `parser->indent`) separately from per-line indentation, defaulting to
-1 at the bare document root, so a plain scalar's continuation lines fold using
the spec rule "more indented than the enclosing block": a root scalar folds its
column-0 continuations (single break to space, blank line to newline, per-line
whitespace stripped) while a scalar inside a collection still ends at a sibling
line. A `---` document-start marker may now be followed by the document's root
node on the same line (`--- foo`, `--- [1, 2]`, `--- |`, `--- >`), and a block
scalar whose body sits at the document root's column 0 captures that body and
ends at the next `---`/`...` marker. Fixed false-rejects 35KP, 4CQQ, 4Q9F, 6JQW,
82AN, 93WF, 96L6, 9MQT/00, 9YRD, B3HG, DK3J, EX5H, EXG3, FP8R, RZT7, T26H, UGM3,
XLQ9; corrected values for 27NA, 2G84/02, 2G84/03, 4WA9, 6FWR, 6LVF, A2M4, K54U,
KSS4, L383, T5N4, W4TN; and started correctly rejecting the invalid 9KBC, ZVH3,
9MQT/01. The still-imperfect M7A3 (multi-document `...` splitting) and NB6Z (tabs
on empty plain-scalar lines) now parse but with mismatched values and stay
tracked. Added root multi-line plain scalar regression tests.
- Escape carriage returns and line feeds in double-quoted output per unicode
scalar rather than per Character. A CR+LF pair forms a single Swift extended
grapheme cluster, so the previous Character-level escape switch matched neither
`\r` nor `\n` and emitted the pair unescaped; the folding parser then read it
back differently, breaking parse/dump/parse stability for values containing
CRLF (observed on the Zoom OpenAPI corpus seed). Added a dumper regression test.
- Tightened error detection so 15 more official YAML test suite cases are handled
correctly, repairing four error-detection regressions and removing them from the
conformance gap manifest. Tabs are now rejected when they would supply
block-structure indentation: a tab at the start of a block line, a tab in a
block scalar's indentation, and a tab in separation position that precedes a
nested block node (a `-` entry, a `?` key, or a compact mapping key) all fail,
while a tab used purely as in-line separation before a scalar value stays valid
(4EJS, Y79Y/000, Y79Y/004, Y79Y/005, Y79Y/006, Y79Y/008, Y79Y/009 now reject;
6BCT and Y79Y/010 now parse with matching values; DC7X and J3BT no longer
falsely reject). A `---`/`...` document marker at column 0 is now rejected when
it appears inside an open flow collection (N782) or inside an open quoted scalar
(RXY3, 5TRB) rather than silently ending the document. A second `%YAML`/`%TAG`
directive section that opens a new document without an intervening `...` suffix
is rejected (MUS6/01). An anchor or tag attached to an alias node, and an
anchor-plus-alias on one node, are rejected (SU74, SR86). Remaining
accepted-invalid cases that need `---`-line content scanning or full simple-key
tracking (for example 2G84/00, 2G84/01, 9MQT/01, ZCZ6) are left tracked in the
gap manifest.
- Added YAML directive handling and completed document-boundary support, fixing
three root causes. First, a `%` now begins a directive only in directive
position (before any document content and before the `---` that starts a
document, including immediately after a `...` end marker); everywhere else, and
in any flow context, `%` is ordinary plain text rather than a misfiring
directive. Second, `%YAML` is parsed leniently per the spec: a `1.x` version is
accepted (a higher minor is treated as 1.2 with a warning rather than an
error), while an unsupported major version, a duplicate `%YAML` in one
document, or extra non-comment fields are rejected; reserved and unknown
directives (and `%TAG` handle definitions) are accepted and ignored, and a
directive must be followed by a `---` document start. Third, a `...` end marker
now closes the current document so that following content begins a fresh
document in the multi-document stream parser (`parseStream`), while the
single-document `parse` still rejects multi-document input. This makes 11
official YAML test suite cases parse with matching values (2LFX, 5TYM, 6WLZ,
6ZKB, 7Z25, 9DXL, 9WXW, BEC7, MUS6/03, MUS6/05, MUS6/06), correctly rejects
four previously accepted-invalid cases (9MMA, B63P, H7TQ, SF5V), and unblocks
6LVF and W4TN from parse errors to value comparisons that now depend on
separate scalar gaps (`---`-line content and literal block chomping).
- Rewrote literal (`|`) and folded (`>`) block scalar content handling to follow
YAML 1.2 §8.1 semantics, fixing four root causes: interior blank lines are no
longer dropped from literal scalars; folded scalars now apply true §8.1.3 line
folding (a single break between equally indented non-empty lines folds to a
space, blank lines stay literal newlines, and more-indented lines keep both
their leading whitespace and their surrounding breaks unfolded); tabs inside
block scalar content are preserved instead of being rewritten as spaces (only
the block indentation columns are stripped); and the `+` keep chomping
indicator is now supported alongside clip and strip. The block scalar is now
scanned as a single unit (header line plus all content lines) with indentation
auto-detection, mirroring the reference scanner, rather than reassembled from a
lossy per-line token array. This makes 24 additional official YAML test suite
cases pass: 4QFQ, 5LLU, 6VJK, 7T8X, 96NN/00, 96NN/01, DWX9, F6MC, F8F9, H2RW,
JEF9/00, JEF9/01, K527, K858, L24T/00, L24T/01, M29M, M9B4, MJS9, P2AD, R4YG,
TS54, W9L4, and Y79Y/001.
- Implemented YAML 1.2 flow (quoted) scalar line folding for both single (`'`)
and double (`"`) quoted scalars, mirroring the reference scanner's
`yaml_parser_scan_flow_scalar`. Multi-line quoted scalars now fold correctly:
trailing whitespace before a break is stripped, leading whitespace on a
continuation line is stripped, a single break between non-empty lines folds to
one space, and N consecutive breaks yield N-1 literal line feeds (the first and
last lines keep their own leading and trailing spaces as content). A double
quoted backslash at end of line now suppresses the break entirely (no space, no
newline). Completed the double quoted escape set with `\xXX` (2 hex digits) and
`\UXXXXXXXX` (8 hex digits), and added the previously missing `\0 \a \v \e`,
literal `\<space>`, and the named breaks `\N \_ \L \P`, alongside the existing
`\uXXXX` (including surrogate pairs). This makes 30 additional official YAML
test suite cases pass: 3RLN/00, 3RLN/01, 3RLN/02, 3RLN/03, 3RLN/04, 3RLN/05,
4ZYM, 565N, 5GBF, 6WPF, 7A4E, 8UDB, 9SA2, 9TFX, DE56/00, DE56/01, DE56/02,
DE56/03, DE56/04, DE56/05, DK95/02, DK95/08, G4RS, NAT4, NP9H, PRH3, Q8AD,
T4YY, TL85, and XV9V.
- Composed an empty (null) plain scalar wherever a node is grammatically required
but absent, instead of raising "expected a YAML node". This single production
mirrors `parser.c`'s empty-scalar fallback and covers every position a node can
go missing: after a tag or anchor property with no following node, an absent
mapping value, an absent mapping key (both the plain `: value` form and the
explicit `?` form), and the empty key and value forms inside flow collections.
An empty node carrying an anchor or tag is explicit; a bare absent node with no
property in a grammatically-required position is still a parse error, matching
the reference parser. The empty scalar resolves through the normal scalar
resolver against an empty value, so `!!str` yields `""`, `!!null` and a bare
empty yield null, and an empty key resolves like any other empty scalar.
Implemented in `Sources/Parsing/TokenEventParser.EmptyNode.swift` with the
flow and block call sites in the sibling `TokenEventParser.*` files. This makes
20 additional official YAML test suite cases pass: 2JQS, 2XXW, 5WE3, 6M2F,
7W2P, BU8L, CFD4, JTV5, KK5P, LE5A, M2N8/00, NHX8, NKF9, PW8X, RZP5, S3PD,
SM9W/01, UKK6/02, WZ62, and XW4D. CT4Q now parses but still mismatches its
expected JSON (a separate flow multi-line plain-scalar folding gap) and is
re-recorded as `json-mismatch`.
- Accepted node properties (anchor, tag) and aliases in mapping-key position, so
a mapping key is parsed as the full node it is: it may carry `&anchor` and/or
`!tag` properties, and it may itself be an `*alias`. Fixed four root causes.
(1) The scanner now terminates an anchor or alias name at the first character
outside libyaml's `IS_ALPHA` set (ASCII letters, digits, `-`, `_`), mirroring
`yaml_parser_scan_anchor`, so `*alias:` reads the alias `alias` followed by a
`:` mapping indicator instead of swallowing the colon into the name. (2) The
block-mapping parser now begins a pair at leading properties or an alias, not
only at the key indicator the scanner emits after them, and properties on the
same line as the first key attach to that key node rather than to the mapping
(`&a a: b` anchors the key `a`, while `&a` alone on its own line anchors the
map). (3) An anchor or tag at value position with no node following on the same
line now composes an empty (null) value that still carries the property, so
`a: &anchor` registers `anchor` as null and a later `*anchor` resolves. (4) An
anchored scalar now registers its resolved value (so an empty anchored scalar
registers null, not `""`). The implicit-key scanner also now recognizes a key
with trailing spaces before its `:` (`"top1" :`). Implemented in
`Sources/Parsing/Scanner.swift`, `Scanner.Helpers.swift`,
`TokenEventParser.swift`, `TokenEventParser.EmptyNode.swift`,
`TokenEventParser.Helpers.swift`, `TokenEventParser.Block.swift`,
`TokenEventParser.Flow.swift`, `TokenCursor.swift`, and the anchor registration
in `EventStreamComposer.swift` and `EventComposer.swift`. This makes 10
additional official YAML test suite cases pass: 26DV, 6KGN, 74H7, 87E4, 9KAX,
E76Z, G9HC, HMQ5, LQZ7, and ZWK4 (G9HC is an invalid-anchor case that is now
correctly rejected). 2SXE now parses but still mismatches its expected JSON (a
separate compact block-mapping structural gap) and is re-recorded as
`json-mismatch`.
### Added
- Track conformance against the official YAML test suite
(github.com/yaml/yaml-test-suite, `data` branch). The 182 cases PureYAML does
not yet handle correctly are vendored under `Tests/Fixtures/yaml-test-suite/`
with a machine-readable manifest (`Tests/Fixtures/yaml-test-suite-gaps.yaml`)
recording each gap's category and current behavior. A new opt-in sweep
(`bash scripts/check-yaml-test-suite.sh`, gated behind
`PUREYAML_RUN_YAML_TEST_SUITE=1`) reproduces every recorded gap so future fixes
must update the manifest. `docs/YAML_TEST_SUITE_CONFORMANCE.md` documents the
measured result (220/402 passing) and the ordered plan for closing the gaps.
### Added
- Opt-in located parse: `parseStreamLocated` returns a source-span projection of every document,
each node carrying its `SourceRange` (line/column start and end) through `Model.LocatedValue` and
`Model.LocatedPair`. It runs alongside the plain parse without changing it, for callers that need
exact source positions (diagnostics, surgical fixes, language servers).
- `%TAG` directives are now scoped per document rather than globally, and a tag shorthand whose
handle is not defined in its own document is diagnosed instead of silently resolved.
- Canonical event-stream conformance gate: the parser is checked against the official yaml-test-suite
event (yeast) format, the strongest oracle, comparing anchors, tags, scalar styles, flow vs block,
and aliases for every valid case including those with no `in.json`. It ships an explicit recorded-gap
set (never a silent skip) and a CI workflow that fetches the suite and runs it on every push.
- The tagged validator now enforces mapping-key uniqueness, the invariant the coverage manifest
already claimed for `Tagged.Mapping` and `Tagged.Node` but no check enforced. `Tagged.Mapping`
keeps every parsed pair, so a duplicate key survives into the tree; `Tagged.BuiltinCheck`
`.mappingKeysAreUnique` (a `Check<Mapping, Node>` on the one framework) reports each duplicate at
its key path, exactly as the plain model does. It ships failing, near-miss, and nested fixtures,
and the tagged validator's two built-in checks are pinned by description. The `Model.Value`
coverage reason is corrected to note that merges and aliases are resolved at parse time, so they
are not a validation-phase invariant: every `validated` manifest entry now names an invariant a
check actually enforces.
- A finding-coverage meta-test makes validation coverage a tested law, now across both validating
layers: every default check, in the model validator and in the tagged validator, must ship a
failing fixture and a near-miss (just-valid) fixture. The meta-test asserts the fixture set covers
exactly each layer's default checks, naming any default check without fixtures or any fixture for a
check that no longer ships. Each check is exercised in isolation, in a validator holding only it, so
a fixture cannot lean on a sibling check to fail and the law stays precise as a layer grows past one
check. A planted orphan fixture was confirmed to fail both the set-equality and the isolation tests.
- Type-directed validation, the core of the validation-rules idiom: a `Check<Subject, Document>`
carries a positive description, a `when` predicate, and a Bool or multi-error check, and fires only
on values of its `Subject` type wherever they occur in the tree. `AnyCheck` erases the subject type
and dispatches by runtime type, guarding against an optional or a wrong type matching. The
validator gains a `checks` list and a `validating(_:)` overload for checks, and the traversal
offers each node the values it may specialise on (the value, and for a mapping the mapping and
its keys). `BuiltinCheck.mappingKeysAreUnique` is the type-directed counterpart of the legacy
rule, firing on `Model.Mapping` directly. The machinery has its own negative tests (wrong type,
optional, and false predicate yield nothing; the same type twice yields two errors) and the
builtin ships a failing and a near-miss test.
- The validator is now configurable by rule identity, per the validation-rules idiom:
`withoutValidating(_ description:)` removes a rule by its description (the removal identity, so
descriptions must be unique and stable), and `validationDescriptions` lists the active rules in
application order. A configuration-pin test asserts the exact descriptions of the blank and
default validators, so adding, removing, or rewording a rule is a deliberate change.
- A validation-coverage gate (`scripts/check-validation-coverage.sh`, wired into
`scripts/check-all.sh` and CI) enforcing the validation-rules discipline that every public type is
either validated or excluded. It enumerates the public surface from the compiler symbol graph (the
authoritative source, not a grep heuristic) and fails the build, by name, if any public type is
missing from `scripts/validation-coverage.tsv` or any manifest entry is no longer public. All 58
public types are classified: the value models carrying a runtime invariant (mapping-key
uniqueness) are `validated`, and the caseless namespaces, closed sum types, data-transfer values,
error types, and engine/machinery are `excluded` with a one-line reason each. This is the first
phase of adopting the OpenAPIKit-style validation idiom; the type-directed `Validation<Subject>`
framework that backs the `validated` entries follows.
- An opt-in conformance harness against the official YAML test-suite, the canonical
YAML 1.2 corpus (`Tests/YAMLTestSuiteConformanceTests.swift`,
`scripts/fetch-yaml-test-suite.sh`). Gated behind `PUREYAML_RUN_YAML_TEST_SUITE=1`
and a separate fetch, so the data is never vendored. It reports two levels:
parse-level (a should-parse case must parse; a should-error case must be
rejected; failures histogrammed by cause) and the stricter, honest value-level,
comparing the parsed tree to each case's `in.json`. Value-level baseline:
130/170 (76.5%) of comparable single-document cases, exposing 40 documents that
parse without error but to the wrong tree (e.g. double-quoted line folding,
scalar resolution, block-scalar indentation) which parse-level cannot see. This
turns "best YAML library" from an assertion into a tracked number.
### Changed
- The validation framework is now a single value type across the whole package. The legacy `Rule`
and its `Context` are removed: a rule was exactly a `Check<Model.Value, Model.Value>` (it fired on
every node, switched on the value itself, and carried the root), so it was a second copy of the one
check machinery (validation-rules.md, principle 10). The model validator now holds only `checks`,
its default is the type-directed `BuiltinCheck.mappingKeysAreUnique`, and `validating(_:)` takes a
`Check`. Custom whole-tree rules are written as `Check<Model.Value, Model.Value>` with the former
`context.root` now `context.document`; behaviour is byte-identical (same issues, paths, and order),
so the public `validate`/`collect`/`blank` surface is unchanged.
- The tagged-node validator no longer carries its own copy of the validation machinery. The
type-directed `Check`, its erasure `AnyCheck`, and its `SubjectContext` are now generic over both
the subject and the document (the validated tree's root type), so one framework serves both layers:
the model tree validates with a `Model.Value` document, the tagged tree with a `Tagged.Node`
document. `Tagged.Rule` and the tagged validation `Context` are removed; the tagged validator keeps
only its own document type, its built-in check (`Tagged.BuiltinCheck`), and its traversal. This is
the one-framework, no-per-layer-duplication discipline of the validation-rules idiom (principle 10);
the tagged validator's public surface (`validate`, `collect`, `blank`, `validating`) is unchanged
except that `validating` now takes a `Check<Tagged.Node, Tagged.Node>` instead of a `Tagged.Rule`.
### Fixed
- Tag-handle diagnostics gathered before a scan-aborting error are now reported rather than dropped.
The diagnostic accessor ran the scan through `try?`, which discarded the whole scanner state, and
the tag-handle issues collected on it, on any error; a document with an undeclared `!handle!` and a
later fatal error reported nothing. The scanner now returns its partial state alongside the error so
the diagnostics survive the abort.
- A repeated document-end marker (`...` immediately followed by another `...`) is now accepted.
- A root-level block scalar (`|`/`>` as the whole document) now ends at a following document marker
(`---`/`...`) rather than folding the marker into its content.
- An empty explicit key in a flow mapping now parses.
- An explicit mapping key that is only a node property (`? &e` then `: &a`) now parses as an empty
key carrying that property, completing the empty-property-key work for the explicit-key form. The
block parser rejected the `:` after a property-only explicit key with an `expectedNode` error;
`parseNode` now treats a `:` in node position as an empty property-carrying key, the same node
`emitEmptyNodeOrThrow` already produced for a property at end of input. yaml-test-suite PW8X now
passes (parse-level 314/333 to 315/333); should-error rejections and value-level matches are
unchanged.
- A block mapping key consisting only of a node property (`&a : v`, `!!str : v`) is now parsed as an
empty implicit key carrying that property, the block analogue of the flow `{ !!str : v }` case. The
scanner synthesized an empty mapping key only for a bare `: v` line; a line whose sole content
before the `:` was an anchor or tag emitted no key marker, so the parser rejected the `:` with an
`expectedNode` error. The scanner now tracks non-property content per line and synthesizes the empty
key after a property, including directly after a block entry (`- !!str : v`). This makes
yaml-test-suite FH7J pass; PW8X still needs the empty-anchored explicit-value form (`: &a`).
Parse-level conformance rises from 313/333 to 314/333; should-error rejections and value-level
matches are unchanged.
- A line beginning with `%` inside a block scalar is now kept as literal content rather than being
misread as a directive. A zero-indent literal or folded scalar sits at column 0, where the
directive scan runs, so a content line such as `%!PS-Adobe-2.0` was rejected as an unsupported
directive. The directive scan now runs only outside a block scalar; a real `%YAML`/`%TAG`
directive and the rejection of a stray `%` in other content are unchanged. This is groundwork for
yaml-test-suite W4TN, which additionally needs block-scalar termination by `...` and per-document
directives, so conformance counts are unchanged.
- A block sequence value carrying an inline property (`sequence: !!seq` then `- entry` at the key's
column) is now correctly terminated by a sibling mapping key. Such a zero-indent sequence has its
entries at the mapping key's own indentation, so a following key ends it. The untagged form already
stopped at the sibling key, but the property-bearing form was routed through the generic node path,
which read the sibling key as a mixed sequence-and-mapping entry and rejected the document. The
zero-indent sequence is now handled before that path, with or without a property. Fixes
yaml-test-suite 57H4; raises parse-level conformance to 313/333 (94.0%) and value-level to 206/206
(100.0%), with the should-error rate unchanged.
- Node properties that span their own indented lines above a further-indented node now attach to
that node (`key: &anchor` then ` !!map` then ` a: b`). An inline anchor with a tag on the next
indented line, above a mapping indented deeper still, left the node introduced by another indent
token, which the parser could not read and rejected as a missing node. The indented mapping value
now descends through that deeper indent, carrying the accumulated properties to the node. Fixes
yaml-test-suite BU8L; raises parse-level conformance to 312/333 (93.7%) and value-level to 205/205
(100.0%), with the should-error rate unchanged.
- A node property written on its own indented line above a zero-indent block sequence (`seq:` then
` &anchor` then `- a` back at the key's column) now binds to that sequence. The grammar lets a
block-in-block sequence sit at its parent's indentation while its property sits one level deeper,
so the indent's matching dedent falls between the property and the entries. The parser had read
the property as an empty anchored value and then rejected the entries as unexpected; it now
consumes that dedent and attaches the property to the sequence. Fixes yaml-test-suite SKE5; raises
parse-level conformance to 311/333 (93.4%) and value-level to 204/204 (100.0%), with the
should-error rate unchanged.
- An alias node may now serve as an implicit mapping key (`*a : value`). The scanner emits the
implicit-key indicator before a scalar key followed by a `:` but did not for an alias, so the
parser, which already accepts an alias node in key position, never entered the pair and rejected
the alias as an unexpected token. The scanner now emits the same indicator before an alias that a
`:` (after optional spaces) follows, in block context. A `:` written directly against the alias
stays part of its name, since `:` is a valid anchor-name character. Fixes yaml-test-suite E76Z and
26DV; raises parse-level conformance to 310/333 (93.1%) and value-level to 203/203 (100.0%), with
the should-error rate unchanged.
- A property written inline before a block mapping's first key (`&a key: value`) now binds to that
key rather than to the whole mapping. The YAML grammar requires a block collection's own
properties to be followed by a line break, so a property on the first key's line belongs to the
key, not the collection. The parser had attached it to the mapping, where an anchor registered
only at the mapping's end, too late for an alias inside one of the mapping's values to resolve. A
property on its own line above the key still binds to the mapping. Fixes yaml-test-suite 2SXE
(anchor names include a trailing `:`, so `&a:` and `&a` are distinct anchors, already handled by
the scanner) and HMQ5 (a key carrying both a tag and an anchor); raises parse-level conformance to
308/333 (92.5%) and value-level to 201/201 (100.0%), with the should-error rate unchanged.
- An anchor on an empty mapping value (`a: &anchor`) now registers that value's implicit null node,
so a later alias resolves to null instead of failing as an undefined alias. Two bugs compounded:
the parser dispatched the anchor-only value position to a nested block mapping, absorbing the
sibling key (`b:`) as the anchored node's child instead of emitting the empty anchored node; and
the stream composer registered an anchor by its raw scalar text rather than the resolved node, so
even a registered anchor on an empty scalar came back as `""` and a typed scalar (`&a 42`) came
back as text. The value position now emits an empty anchored node when only a sibling mapping key
follows, and an anchor registers the composed value. Fixes yaml-test-suite 6KGN and a latent
typed-scalar anchor bug; raises parse-level conformance to 306/333 (91.9%) and value-level to
199/199 (100.0%), with the should-error rate unchanged.
- A `%YAML` directive with a higher minor version than 1.2 is now accepted rather than rejected. The
spec calls for processing a document with a higher minor version, with a warning, so `%YAML 1.3` is
read as a YAML 1.x document instead of erroring. The version check now accepts any well-formed
`1.minor` version and rejects only a different major version (`2.0`) or a malformed one. Three tests
that asserted `%YAML 1.3` was an error were corrected to use `2.0`. Raises yaml-test-suite parse-level
conformance to 305/333 (91.6%), with the should-error rate unchanged.
- A plain scalar that is an inline block sequence item value (`: - b`) now stops folding at a later
block entry that returns to the sequence's own indent, so the dashes form the rest of the sequence
rather than folding into the first item. A block entry only recorded its inline node's indent as a
valid implicit level, not the dash's own indent, so a continuation could not tell a sibling entry
from content, and `? a` / `: - b` / ` - - c` / ` - d` folded into one scalar. The entry's indent
is now recorded as a sibling level too (except under an explicit key, whose sequence key drives
implicit indentation itself), while a block entry more indented than its container stays plain
content. Brings yaml-test-suite value-level conformance to 197/197 (100.0%), with the parse and
should-error rates unchanged.
- A leading blank line of a block scalar now keeps any whitespace past the content indentation as
content, as a blank line within the scalar already did. The leading blank lines (those before the
first content line) were only counted, so an otherwise-blank `␣\t` line under a one-space content
indent lost its tab, and a folded scalar that began with such a line dropped the leading tab. The
leading blanks now record their whitespace and emit the surplus once the content indentation is
known, so the tab survives. Raises yaml-test-suite value-level conformance to 196/197 (99.5%), with
the parse and should-error rates unchanged.
- A flow mapping entry may now omit its `:` value indicator, mapping its key to null: `{ a }` and
`{ a, b }` are flow sets. The parser required a `:` after every flow mapping key and rejected these,
and a multi-line key with no value (`{ multi`/`line, a: b}`) with it. A value now parses only when a
`:` follows the key; otherwise the key maps to null. A test that asserted `{ a, }` was an error was
corrected, since it is a valid set. Raises yaml-test-suite parse-level conformance to 304/333
(91.3%) and value-level to 195/197, with the should-error rate unchanged.
- A plain scalar key in a flow collection may now span lines. A flow key, unlike a block implicit
key, is allowed to be multi-line, so `{ multi` / ` line: v }` is the key `multi line`. The scalar
continuation stopped at the `:` value indicator in every non-explicit case, dropping the second
line, so the parser mis-split the entry. Folding across the `:` now also applies in flow, while a
block implicit key still stops before such a `:`. Raises yaml-test-suite parse-level conformance to
301/333 (90.4%) and value-level to 193/195, with the should-error rate unchanged.
- An explicit key (`?`) inside a flow collection is now parsed correctly. The flow `?` did not mark an
explicit key, so the following scalar synthesized its own implicit key: `{? foo : bar}` came out as
`{{foo: bar}: null}` rather than `{foo: bar}`, and `[? foo : bar]` likewise. The explicit-key state
is now tracked per flow level on the flow context, so the key scalar is not turned into an implicit
key, a `:` at that level is the pair's value indicator, a `,` ends the key, and a nested collection's
`:` does not affect an outer key (`{? {a: b} : c}`). The key may also span lines, folding to one key,
which restores the multi-line flow case `[? foo\n bar : baz]`. Raises yaml-test-suite value-level
conformance to 192/194 (99.0%), with the parse and should-error rates unchanged.
- An explicit key (`? foo`) whose plain scalar spans lines now folds a continuation line that ends at
the value `:`. `? foo` then `bar : baz` is the key `foo bar`, not an error: the `:` is the key's
value indicator, so the line stays key content and folds in, where before the scalar dropped it and
the parser stalled. An implicit key is unchanged, since there a continuation line ending at `:` is a
new mapping key and the scalar still stops before it. The block test-suite rates are unchanged (no
comparable case isolates this), and the flow form needs separate explicit-key work, but the parser
now folds block explicit keys correctly.
- An explicit block scalar indentation indicator (`|2`, `>1`) on an inline mapping value now measures
from the mapping's indentation. A value like `- aaa: |2` has the inline mapping as its parent, but
that mapping sits at an implicit indentation off the main stack, so the indicator was added to the
stack's level (0) instead of the mapping's (2), making the content indentation too small and pulling
the next key into the scalar. The deepest applicable implicit indentation is now folded in for an
inline mapping value, while a direct sequence item (`- |1`) keeps the sequence's own level. Raises
yaml-test-suite value-level conformance to 191/194 (98.5%), with the parse and should-error rates
unchanged.
- Compact nested block sequences (`- - x`) now separate their outer items correctly. Such a sequence
carries no indent/dedent tokens, only block entries at their columns, but the sibling-entry loop
collected every following block entry regardless of column, so `- - x` then `- - y` became one
outer item holding both rather than two. A sequence now collects siblings only at its own column, so
an entry at a smaller column ends it and belongs to the enclosing sequence. Raises yaml-test-suite
value-level conformance to 190/194 (97.9%), with the parse and should-error rates unchanged.
- A block scalar whose only lines are blank now carries those blank lines through chomping at stream
end. Such a scalar never establishes a content indentation, so its blank lines stayed buffered and
were dropped, making `keep: |+` over a blank line resolve to the empty string instead of a line
break. The buffered blanks are now flushed as content at stream end, so keep (`|+`/`>+`) preserves
them while clip and strip trim them: `keep: |+` over one blank line is `"\n"`. Raises yaml-test-suite
value-level conformance to 189/194 (97.4%), with the parse and should-error rates unchanged.
- A scalar carrying an explicit non-core tag now stays a string instead of being implicitly resolved.
A node with a tag is not subject to plain-scalar resolution, so a local tag (`!foo`), the
non-specific `!`, or an unknown verbatim URI fixes the node as the string of its content: `! 12` is
the string `12`, not the integer, and `!Thing false` is the string `false`, not a boolean.
Recognized core tags (`!!int`, `!!bool`, ...) and untagged scalars still resolve. Two compatibility
tests that asserted the previous behavior were corrected. Raises yaml-test-suite value-level
conformance to 188/194 (96.9%), with the parse and should-error rates unchanged.
- A tab before a `#` now ends a plain scalar and begins a trailing comment, as a space already did,
since a comment opens after any white space. `c: d\t#X` is the value `d` with a comment, not the
literal `d\t#X`. A tab inside a plain scalar with no `#` after it stays part of the value, and a `#`
glued to content with no white space is still content. Raises yaml-test-suite value-level
conformance to 187/194 (96.4%), with the parse and should-error rates unchanged.
- A blank line next to a more-indented line in a folded block scalar (`>`) now keeps both line
breaks. Between two non-more-indented lines a blank run of N folds to N breaks (the fold is
absorbed into the first), but next to a more-indented line nothing folds, so the absorbed break
survives and the run yields N+1 breaks. The folder counted N breaks in every case, collapsing the
paragraph spacing around indented blocks. Raises yaml-test-suite value-level conformance to 186/194
(95.9%), with the parse and should-error rates unchanged.
- Plain scalar line folding now strips tabs, not only spaces, around continuation lines. A tab is not
indentation, so a continuation line's leading tab (`\t3rd`) is folding white space and folds away
rather than entering the value, and a line of only white space that holds a tab (`␣␣\t`) is an empty
line that folds to a line break, not an extra space. Raises yaml-test-suite value-level conformance
to 183/194 (94.3%), with the parse and should-error rates unchanged.
- A tab in block scalar content past the block's indentation is now kept verbatim instead of being
re-expanded to spaces. A block scalar is indented with spaces only, so a line like `␣\ttext` under a
one-space indent strips the space and keeps the tab, yielding `\ttext`; the scanner measured the
leading whitespace as a tab-expanded width and rebuilt the surplus as spaces, turning the tab into
three spaces. The surplus is now reconstructed from the actual leading characters, so a tab stays a
tab while surplus spaces stay spaces. Raises yaml-test-suite value-level conformance to 180/194
(92.8%), with the parse and should-error rates unchanged.
- Mid-line separation white space between tokens now includes tabs, not only spaces, completing the
tab-as-separation handling. An indicator followed by a tab no longer leaks the tab into the next
node: `?\tkey` is the key `key`, not `\tkey`, and a run of tabs between a `:` and its value
(`a:\t\tb`) is separation. This closes the post-indicator tab leak noted when the indicator boundary
itself was fixed. Tabs as indentation are still handled separately by the line-start scan, and the
yaml-test-suite rates are unchanged (no comparable case isolates this), but the parser now matches
the spec on tab separation.
- A tab now follows the `:` mapping-value indicator the way a space does, in both block and flow
context, since a tab is separation white space in YAML. `foo:\tbar` is the mapping `{foo: bar}`
rather than the plain scalar `foo:\tbar`. A `:` with no following white space at all (`foo:bar`)
stays plain scalar content. Raises yaml-test-suite parse-level conformance to 300/333 (90.1%), with
the should-error rejection rate unchanged.
- A tab now ends a leading indicator the way a space does, since a tab is separation white space in
YAML (only indentation forbids a tab). A document marker, block entry, or explicit key followed by a
tab (`---\tscalar`, `-\titem`, `?\tkey`) is recognized as the indicator rather than folded into a
plain scalar. Raises yaml-test-suite value-level conformance to 175/192 (91.1%), with the parse and
should-error rates unchanged.
- A flow sequence element that begins with `:` is now parsed as a single-pair mapping with an empty
(null) key, e.g. `[ : value ]` is `[{null: value}]`. The parser raised "expected a YAML node" for
the leading `:`; it now emits the empty-key mapping, mirroring the way `[ a: b ]` (a scalar key
followed by `:`) already becomes a mapping element. Plain elements and scalar-key mapping elements
are unchanged. Raises yaml-test-suite parse-level conformance to 298/333 (89.5%), with the
should-error rejection rate unchanged.
- A flow mapping entry may now begin with its `:` indicator, giving an empty (null) key, e.g.
`{ : value }` or `{ a: b, : c }`, or carry only properties before the `:`, e.g. `{ !!str : value }`.
The parser required a node before every flow mapping `:` and raised "expected a YAML node"; it now
emits a null key in that position, the same way an empty value after the `:` is already handled. The
allowance is local to flow mapping key position, so an entry with no `:` at all (`{ a, }`) stays an
error. Raises yaml-test-suite parse-level conformance to 297/333 (89.2%) and value-level to 174/192,
with the should-error rejection rate unchanged.
- A flow collection followed by a `: ` indicator on the same line is now parsed as an implicit block
mapping key, e.g. `[a, b]: value` or `{a: 1}: value`. The scanner looks ahead from the flow opener
to its matching close (skipping quoted scalars so their brackets and colons do not miscount) and,
when a same-line `:` follows, emits the implicit-key marker a scalar key would get, so the flow
node is parsed as the key through the ordinary mapping path. A block implicit key must lie on one
line, so a flow collection spanning a line break before its `:` (`[23 / ]: 42`) stays an error, and
a flow node that no `:` follows, or that is a mapping value, is unaffected. Raises yaml-test-suite
parse-level conformance to 294/333 (88.3%) with no change to the should-error rejection rate.
- A block scalar that is the document's own root node now folds or keeps content that begins at
column 0, e.g. `>` or `--- |` followed by unindented lines. The document node's indentation is -1,
so its content at column 0 is still more indented than its parent, but the scanner read the base
level as 0 and required content to be strictly deeper, leaving the lines to be mis-scanned as a
stray scalar after the document. The scanner now recognizes a root block scalar (one with no
enclosing mapping value, explicit key, block-entry indent, or flow context on its header line) and
establishes its content indentation at column 0. A non-root block scalar keeps its parent's indent
as the floor, so an under-indented sequence item or mapping value is still rejected. Raises
yaml-test-suite parse-level conformance to 291/333 (87.4%) and value-level to 173/191 (90.6%).
- Content after a document-end marker (`...`) now begins a new bare document. A stream such as
`--- scalar / ... / key: value` or `a / ... / b` is two documents, but the scanner rejected the
content and the parser expected the document to end at the marker. The scanner now reopens the
stream after `...`, and the document loop returns at a `...` so the outer loop parses the next
document. A trailing `...` with nothing after it still leaves a single document. Lifts official
test-suite parse-level conformance from 288/333 (86.5%) to 289/333 (86.8%), with no regression.
- An anchor or tag with no node after it now denotes an empty node. A property always attaches to a
node and an empty node is allowed, so `&anchor` alone, `key: &a`, or a `!!str` with nothing after
it (`x: !!str` is the empty string, `x: !!null` is null) is the property applied to a null node
rather than a missing-node error. A real tagged or anchored value is unaffected. Lifts official
test-suite parse-level conformance from 287/333 (86.2%) to 288/333 (86.5%) and value-level by one.
- An implicit null mapping key is now parsed. A `:` that opens a line with no key before it
(`: value`, a bare `:`, or a null key among other entries) is an entry with a null key, which
the parser previously rejected as a missing node. The scanner synthesizes the empty key, tracking
a pending explicit `?` key so that an explicit key's value indicator on the next line, including
after a flow complex key (`? {a: b}` then `: c`), is not mistaken for a null key. Lifts official
test-suite parse-level conformance from 283/333 (85.0%) to 287/333 (86.2%), with no regression.
- A directive after a document-end marker (`...`) now begins the next document instead of being
rejected. A multi-document stream where each document carries its own directives
(`%TAG ...` / `--- doc` / `...` / `%TAG ...` / `--- doc`) is now accepted, since a directive
following `...` opens the next document's directive section. A directive after document content
within the same document is still rejected, and a directive after `...` still requires its own
`---`. Lifts official test-suite parse-level conformance from 278/333 (83.5%) to 283/333
(85.0%), with no regression.
- A block sequence entry can no longer follow a flow collection that closed earlier on the same
line (`- { y: z }- invalid`, `- [a] - b`); the entry would begin a sequence inline after a
complete node. The scanner now tracks a top-level flow close on the current line and rejects a
block entry after it, while a nested block entry (`- -`, which has no preceding flow close), a
flow collection on its own line, and nested flow collections are unaffected. Lifts official
test-suite parse-level conformance from 276/333 (82.9%) to 277/333 (83.2%), with no regression.
- A mapping value's anchor or tag on the line below the key must now be indented past the key. A
property at the key's own indentation is ambiguous with a sibling and is rejected, whether it is
the only property (`seq:` then `&anchor` in column 1) or a later one after an inline first
(`key: &x` then a `!!tag` in column 1). An inline property, a property indented under the key, and
a block sequence at the key's indent (which carries no property) stay valid. Lifts official
test-suite parse-level conformance from 275/333 (82.6%) to 278/333 (83.5%), with no regression.
- A `-` immediately before a flow indicator is no longer treated as a scalar. In a flow collection
a `-` that begins a plain scalar must be followed by a non-indicator character, so `[-]`,
`[-, -]`, and `{a: -}` are now rejected. A dash followed by ordinary content is unaffected:
negative numbers (`[-1, -2]`), a dash inside a scalar (`a-b`), a dash starting a word (`-x`), and
a quoted dash all stay valid. Lifts official test-suite parse-level conformance from 273/333
(82.0%) to 275/333 (82.6%), with no regression.
- A block scalar leading blank line that is more indented than the content is now rejected. The
spec forbids a leading empty line from containing more spaces than the first non-empty line, so a
block scalar whose blank lines are wider than its content (`|` then a 5-space blank then 2-space
content) no longer parses. Indentation is counted in spaces, not tab-expanded width, so a tab on
a blank line (`␣\t`) is not treated as a deeper indent. Leading blanks within the content indent
are unaffected. Lifts official test-suite parse-level conformance from 270/333 (81.1%) to
273/333 (82.0%), with no regression.
- A document marker line (`---` or `...`) inside a multi-line quoted scalar is now rejected. Such
a line is ambiguous with a document boundary, so the spec forbids it within a scalar; the scanner
previously folded it into the value. The marker must begin a line and be followed by a boundary
to be rejected, so inline dashes (`"a --- b"`), a `---` with no following boundary (`---b`), and
an indented `---` remain ordinary content. Lifts official test-suite parse-level conformance from
268/333 (80.5%) to 270/333 (81.1%), with no regression.
- A block sequence can no longer start on the same line as its anchor or tag (`&a - x`). A block
collection must begin on the line after its node properties, so an anchor or tag immediately
before the first block entry is now rejected. An anchor or tag on its own line above the
sequence, inside a sequence item (`- &a x`), before a flow sequence (`&a [x]`), or on a scalar
is unaffected. Lifts official test-suite parse-level conformance from 267/333 (80.2%) to 268/333
(80.5%), with no regression.
- An alias node can no longer carry an anchor or tag. An alias is a bare reference, so `&b *a`
(an anchor on an alias) and a tag on an alias are now rejected; the parser previously dropped the
property and accepted the document. A bare alias and an anchor or tag on a real node are
unaffected. Lifts official test-suite parse-level conformance from 266/333 (79.9%) to 267/333
(80.2%), with no regression.
- A comment can no longer interrupt a multi-line plain scalar. A comment terminates a plain
scalar, so content that tries to continue it on a following line (`key: word1` then a `#` line
then ` word2`, or `a` then `b # ...` then `c`) is now rejected instead of being silently merged
back into the value. A genuine multi-line plain scalar and a trailing comment after a value are
unaffected. The parser path that re-merged the scanner-split scalar is removed, since the scanner
only splits a plain scalar this way when a comment broke its folding. Lifts official test-suite
parse-level conformance from 264/333 (79.3%) to 266/333 (79.9%), with no regression.
- A quoted scalar that spans a line break can no longer be a block implicit key (`"c⏎ d": 1`).
An implicit key in block context must lie on a single line, so a quoted scalar whose open and
close quotes sit on different lines is now rejected when used as such a key. The rule is
block-only: a flow mapping key may span lines (`{ "a⏎ b": 1 }`), and a multiline quoted value, a
single-line quoted key, and an explicit key (`? "a⏎ b"`) are all unaffected. Lifts official
test-suite parse-level conformance from 261/333 (78.4%) to 264/333 (79.3%), with no regression.
- A block sequence can no longer be an implicit key's value on the same line as its `:`
(`key: - a`); the value must begin on a following line, or use flow style (`key: [a]`). Like the
inline nested mapping rule, this is specific to an implicit key: an explicit- or null-key `:`
opens its line, so a compact sequence after it stays valid, as do a sequence on the next line, a
nested sequence (`- - a`), a flow sequence, and a negative number (`key: -1`). Lifts official
test-suite parse-level conformance from 260/333 (78.1%) to 261/333 (78.4%), with no regression.
- An inline nested block mapping is now rejected. Block style allows only one mapping pair per
line, so a scalar followed by `:` after a block-level value on the same line (`a: b: c`,
`a: 'b': c`, `x: { y: z }in: valid`, or trailing content after a quoted value) no longer parses
as a nested mapping. The scanner suppresses the implicit key it would otherwise synthesize once
the line already holds a pair. A value indicator that opens its line is an explicit- or null-key
indicator, whose value may legitimately be a compact mapping (`? k` then `: a: 1`), so that, flow
mappings (`{a: b, c: d}`), next-line nesting, and colons inside scalars are unaffected. Lifts
official test-suite parse-level conformance from 256/333 (76.9%) to 260/333 (78.1%), with no
regression.
- A `...` document-end marker no longer accepts content on its own line. Only a comment or blank
may follow the marker, so `... invalid` is now rejected; the scanner previously skipped the rest
of the line. A bare `...`, a trailing comment (`... # done`), and a `...` not followed by a
boundary (`...junk`, which is an ordinary plain scalar, not a marker) are unaffected. Lifts
official test-suite parse-level conformance from 255/333 (76.6%) to 256/333 (76.9%), with no
regression.
- A directive must now be followed by an explicit `---` document start. A stream that ends, or
reaches a `...` document-end marker, while a directive is still awaiting its document
(`%YAML 1.2` alone, or `%YAML 1.2` then `...`) is now rejected, as the YAML grammar requires a
directive to head an explicit document. This applies to any directive, including an unrecognized
one, which is still otherwise ignored. A directive followed by `---` (with content, inline, or
empty) is unaffected. Lifts official test-suite parse-level conformance from 253/333 (76.0%) to
255/333 (76.6%), with no regression.
- The `%YAML` directive is now validated. It takes exactly one parameter, the version, optionally
followed by a comment, and at most one may appear per document; an extra parameter
(`%YAML 1.2 foo`) or a second `%YAML` directive is now rejected. Previously any tokens after the
version were ignored and duplicate directives were accepted. A well-formed directive, with or
without a trailing comment, is unaffected. Lifts official test-suite parse-level conformance
from 251/333 (75.4%) to 253/333 (76.0%), with no regression.
- A flow collection no longer accepts an empty entry. A comma separates entries, so one must
precede it; a leading comma (`[ , a ]`), a repeated comma (`[ a, , b ]`), or the same in a flow
mapping is now rejected, while a single trailing comma (`[ a, b, ]`) and empty collections
(`[]`, `{}`) stay valid. The parser previously skipped a comma without checking that an entry
came before it. Lifts official test-suite parse-level conformance from 249/333 (74.8%) to
251/333 (75.4%), with no regression.
- Double-quoted escape sequences now follow the full YAML 1.2 escape table. Previously the
decoder recognized only a subset (`\" \\ \/ \b \f \n \r \t \u`) and silently passed any other
character through, so invalid escapes such as `\.` and `\'` were accepted while valid ones such
as `\x`, `\v`, `\e`, `\0`, `\N`, `\_`, `\L`, `\P`, `\U`, and backslash-space were mangled
(`\x0d` became the text `x0d`). The decoder now maps every defined escape to its character,
handles `\x` (8-bit) and `\U` (32-bit) hex escapes alongside `\u`, treats a backslash before a
line break as an escaped line break that joins the lines with no separator, and rejects an
unknown or truncated escape. Lifts official test-suite parse-level conformance from 247/333
(74.2%) to 249/333 (74.8%) and value-level from 166 to 170 of comparable cases (90.4%), with
no regression.
- A `#` glued to earlier content on its line (e.g. `"value"# c`, `[ a, b ]#c`, `>#c`) is now
rejected: the spec requires a comment to be preceded by whitespace or to open a line, and such
input was previously accepted as a comment. A `#` that opens a line, follows a space, or sits
inside a plain scalar (`a#b`, where it is literal content) is unaffected. The check tracks
whether real content has been scanned on the current line, rather than comparing token line
numbers, because the streaming tokenizer drains its buffer and a block scalar's blank line
emits an empty scalar attributed to the next line. Lifts official test-suite parse-level
conformance from 243/333 (73.0%) to 247/333 (74.2%); four wrongly-accepted invalid documents
are now correctly rejected with no valid document regressing.
- A block scalar followed by a less-indented comment line no longer fails with a spurious
"unexpected indentation". When a comment ends a block scalar, its own column was processed as
a dedent and, landing on a width no indentation level held, raised an error; a comment's
indentation is now left out of block structure. Relatedly, a whitespace-only line indented
past a block scalar's content level now keeps its surplus spaces as content instead of
collapsing to empty, so such a line contributes the correct value. Lifts official test-suite
parse-level conformance from 240/333 (72.1%) to 243/333 (73.0%) and value-level matches.
- A quoted mapping key may now have a space before its `:` (`"key" : value`). A plain key
consumes the space before the colon into the scalar, but a quoted key stops at its closing
quote, leaving the space, and the implicit-key check looked only at the immediate next
character. It now looks past spaces for the colon, as the spec allows. Quoted keys with no
space, quoted values, and plain keys are unaffected. Lifts official test-suite parse-level
conformance from 238/333 (71.5%) to 240/333 (72.1%).
- A block mapping pair after the first may now carry an anchor or tag on its key
(`k: 1` / `&b key2: 2`). The first pair's key properties were consumed when the mapping was
entered, but later pairs were not, so an anchored key past the first failed with
"expected mapping key". The pair parser now consumes leading key properties for every pair,
and the mapping continues across an anchor or tag that begins a key. Lifts official test-suite
parse-level conformance from 235/333 (70.6%) to 238/333 (71.5%).
- An explicit mapping key (`? key`) with no following `:` now maps to null instead of failing to
parse, so several `? key` entries in a row (a set, for instance) each map to null. The parser
required a `:` after every explicit key; it now treats a missing one as an implicit null value.
Explicit keys with values, and complex keys generally, are unaffected. Lifts official test-suite
parse-level conformance from 229/333 (68.8%) to 235/333 (70.6%).
- An unrecognized directive (e.g. `%FOO`) before a document is now ignored, as the YAML spec
requires (a warning, otherwise no effect), instead of being rejected with an error. `%YAML`
and `%TAG` are unchanged, and a directive after content (where one cannot appear) is still
rejected. Lifts official test-suite parse-level conformance from 227/333 (68.2%) to 229/333
(68.8%).
- A plain scalar at the document root now folds across lines that sit at column 1, so a bare
multi-line value such as `a` / `b` / `c` becomes `"a b c"` instead of failing to parse. The
continuation check required a continuation to be more indented than the scalar, but a root
scalar has nothing enclosing it, so its enclosing-block indent is -1 and a continuation may
start at column 1. The continuation now stops at a document marker (`---` / `...`), so a
scalar never folds across a document boundary. Lifts official test-suite parse-level
conformance from 221/333 (66.4%) to 227/333 (68.2%). (A stream test that asserted two bare
plain lines were rejected as extra content was corrected: that input is a valid multi-line
scalar; genuine extra content, like a flow node followed by a scalar, is still rejected.)
- A multi-line single-quoted scalar now folds its line breaks, like the double-quoted fix
below; both share the same flow line folding, so the fold logic is now one helper. Lifts
official test-suite value-level conformance from 143/169 (84.6%) to 147/169 (87.0%), with
no regression.
- A multi-line double-quoted scalar now folds its line breaks instead of keeping them raw.
The decoder appended real line breaks and continuation-line indentation verbatim, so
`"a\n b"` came back as `"a\n b"` rather than `"a b"`. It now applies flow folding: a
single break folds to a space, a blank line to one newline, and whitespace adjacent to a
break (a line's trailing, the next line's leading) is stripped, while first-line leading
and last-line trailing whitespace and `\n` escapes are preserved. Lifts official test-suite
value-level conformance from 135/169 (79.9%) to 143/169 (84.6%), with no regression;
parse-level is unchanged because these documents already parsed, just to the wrong tree.
- A root node that begins on the document-marker line (`--- >`, `--- |`, `--- foo`,
`--- {a: 1}`) is no longer discarded. The marker scanner skipped the rest of the
`---` line unconditionally, so a block-scalar header was dropped and its body was
mis-read as a plain scalar (folded to spaces, no trailing newline) and a same-line
plain or flow node was lost entirely. The document start now keeps any real
same-line content for the parser, skipping only trailing spaces or a comment.
Lifts official test-suite value-level conformance from 130/170 (76.5%) to
135/169 (79.9%), with no regression; parse-level is unchanged because these
documents already parsed, just to the wrong tree, which only the value-level
metric could see.
- A multi-line plain scalar whose value begins on its own line under a mapping key
now folds its continuation lines instead of stopping after the first. The
continuation check compared the next line's indent against the value's own pushed
indent, so a continuation at the same indent failed the strict `>`; it now
compares against the parent block's indent, matching the spec (continuation must
exceed the mapping's indentation, not the value's). Lifts official test-suite
parse-level conformance from 211/333 (63.4%) to 221/333 (66.4%), with no
regression across the existing suite.
### Changed
- Split multi-type source files so each file declares exactly one top-level type
(`docs/rules/code-style.md`): the Tagged node component types, the validation
report types, `TokenKind`, and the event-stream composer helpers each move to
their own file. Public type names are unchanged. `scripts/check-namespacing.sh`
is now namespace-aware and detects multiple sibling types declared inside
`extension` blocks, which the previous column-anchored scan missed.
- Merge-key (`<<`) override resolution now lives in a single internal helper,
`Parsing.MergeResolution`, shared by the buffered and streaming event
composers. Both paths previously carried an identical copy of the
inherited-versus-local pair merge, so the subtle precedence rules (local keys
win, earlier sources win, inherited-then-local ordering) now have one source
of truth. No behavior change.
### Removed
- Remove three `Parsing.ParseError` cases that were never thrown: `emptyDocument`
(empty and comment-only documents parse to `null`), `expectedMappingKey`
(superseded by `unexpectedToken` carrying an "expected mapping key" message),
and `tabIndentation` (tabs in indentation are reported by the validation
layer's `PreflightScanner`, not the parser). No reachable behavior changes.
### Fixed
- Preserve blank lines in block scalars. The scanner dropped every blank line
within and after a block scalar, so a literal `|` lost its internal blank
lines, a folded `>` turned a blank line into a space instead of a newline, and
the `|+` / `>+` keep indicator could not preserve trailing blank lines. Blank
lines are now captured as empty content in both the batch and streaming scan
paths (which had duplicated the loop), and `blockScalarValue` applies literal
versus folded line semantics and clip/strip/keep chomping correctly. A block
scalar without blank lines is unchanged.
- Keep a more-indented line inside a folded `>` block literal, with its line
breaks preserved, rather than folding it into the surrounding text.
- Preserve leading blank lines between a block scalar header and its first
content line. They precede the indent token, so they are buffered and emitted
as empty content once the content indentation is established.
### Documentation
- Correct `SECURITY.md`: fix the reporting contact address and replace the
scope section, which described an unrelated static-site generator, with the
YAML parsing, validation, and emitting surface that is actually in scope.
## [0.1.5] - 2026-06-23
### Documentation
- Document why `maxNestingDepth` is the necessary and sufficient safety bound
for deeply nested input: a `Model.Value` is a recursive tree whose every
operation (compose, emit, compare, hash, and especially deallocate) recurses
with nesting, and Swift cannot deallocate a recursive value type iteratively,
so bounding the tree's depth at parse time is the only complete protection.
Records the measured headroom, why the bound cannot be raised without an
iterative parser (and would be pointless past the ~5,000 `deinit` limit), and
the caveat that trees built by encoding a deep `Codable` value bypass it.
## [0.1.4] - 2026-06-23
### Fixed
- Reject documents nested beyond a fixed depth (`maxNestingDepth`) with a
`nestingTooDeep` parse error instead of overflowing the recursive-descent
parser's stack. A pathologically nested input (for example hundreds of nested
`[[[...]]]` or deeply indented mappings) previously crashed the process; it now
fails cleanly. The limit is well above the nesting of real-world specs, which
are unaffected.
## [0.1.3] - 2026-06-23
### Fixed
- Parse a leading `#` in block scalar content (`>`, `>-`, `|`, `|-`) as literal
text instead of a comment. A block scalar whose first content line began with
`#`, such as an OpenAPI `$ref: >-` folded onto `#/components/...`, was
previously dropped to an empty string.
## [0.1.2] - 2026-06-06
### Added
- Document the real-world YAML corpus, fuzz/property gate, and CI artifact
contract for production hardening.
- Add machine-readable parser diagnostic codes to validation report payloads and
cover malformed production-shaped YAML with 10+ structured diagnostics.
- Vendor a 114-file real-world YAML corpus with a machine-readable manifest,
default metadata checks, and an opt-in full parse/validate/round-trip gate.
- Add large public APIs.guru Stripe and Zoom OpenAPI YAML specs to the opt-in
corpus gate.
- Add a reduced synthetic ACME Zephyr fixture set for Stitcher-style multi-file
OpenAPI inputs and bundled output.
- Add a full renamed Birch-style ACME fixture catalog with 594 YAML files,
preserving module/file structure while replacing private organization names,
domains, and documentation URLs.
- Add YAML 1.2 JSON-subset compatibility for compact flow mappings such as
`{"openapi":"3.0.0"}`, including regression coverage for colons inside flow
mapping values.
- Add JSON compatibility tests and artifacts that compare PureYAML against
Swift's JSON decoder across JSON roots, escapes, Unicode surrogate pairs,
exponent numbers, nested values, and whitespace.
- Preserve round-trip fidelity for mapping keys that require quotes, including
trailing-space keys found in the large Zoom OpenAPI corpus seed.
- Add an opt-in Yams differential script that compares parse behavior for the
checked-in real corpus without adding Yams to the PureYAML package manifest.
- Add an opt-in Yams diagnostic comparison script that records PureYAML
structured malformed-input diagnostics alongside Yams parser errors.
- Add an opt-in Yams validation comparison script proving structured duplicate
key validation on YAML that both parsers can load.
- Add an opt-in Yams throughput script and scheduled/manual CI artifact for
release-mode parser performance comparison on representative corpus seeds.
- Add an opt-in PureYAML phase profiler that times scanner, event parser,
composer, lazy parse, validation, and dumping phases on slow corpus seeds.
- Compose normal parser output from events as they are emitted, avoiding a full
intermediate event array while keeping `parseEvents` available for explicit
event materialization.
- Lazily scan tokens for normal `parse` and `parseStream` calls, avoiding a
retained full token array on the main parser path.
- Track reader positions with a UTF-8 cursor so ASCII-heavy scanner advancement
avoids per-character string allocation while preserving byte-accurate marks.
- Add deterministic generated corpus properties for valid YAML round trips and
malformed real-seed mutations with structured diagnostics.
- Write corpus run artifacts under `.build/pureyaml-artifacts/`, including real
seed summaries, generated mutation summaries, seed metadata, and environment
details.
- Add a scheduled/manual CI corpus job that runs the corpus and Yams
differential gates and uploads the generated artifact bundle.
## [0.1.1] - 2026-06-06
### Added
- Add platform CI badges and an active production-hardening roadmap to the
README.
### Changed
- Add release-mode build and test gates to the macOS, Linux, Windows, and WASM
verification scripts.
- Update GitHub Actions checkout steps to `actions/checkout@v6` so CI uses the
current Node 24 action runtime.
### Fixed
- Treat more-indented plain scalar continuation lines that start with `-` as
text unless they appear at a known sequence-entry indentation. This fixes
OpenAPI-style `allOf` descriptions while preserving nested block sequences
such as `- - 123`.
- Parse explicit scalar keys whose `:` value starts a same-line mapping and
continues with indented mapping siblings, matching large OpenAPI schema keys.
## [0.1.0] - 2026-06-05
### Added
- Bootstrap PureYAML as a root Swift package with one `PureYAML` library product,
no external SwiftPM dependencies, no C sources, no generated parser targets,
and no Foundation requirement in the library target.
- Add the core pure-Swift YAML model with ordered mappings, sequences, scalars,
duplicate-key preservation, and first-class complex mapping keys through
`Model.Pair.keyNode`.
- Add parser support for block mappings, block sequences, comments, flow
collections, quoted strings, literal and folded block scalars, anchors,
aliases, YAML directives, document markers, merge keys, complex mapping keys,
multi-document streams, and selected explicit built-in scalar tags.
- Add deterministic YAML dumping with block output by default, optional flow
collections, conservative plain scalars, conservative literal block scalars,
complex mapping key dumping, and multi-document stream dumping.
- Add tag-preserving parsing with `PureYAML.parseTagged(_:)` and
`PureYAML.parseTaggedStream(_:)`, plus tagged validation rules for unsupported
built-in tags and tags applied to the wrong node kind.
- Add `PureYAML.Tagged.Constructor` for caller-owned tagged construction with
exact missing-handler and kind-mismatch errors, recursive path context, and a
model-value fallback that preserves mapping order and duplicate keys.
- Add path-aware validation with duplicate-key checks, custom rules, rule
predicates, strict and non-strict behavior, warning collection, stream
document indexes, deterministic validation paths, and direct value-tree
validation for states ordinary loaders may collapse.
- Add diagnostic-first validation reports, raw-source preflight scanning,
`Validation.ReportError`, and Markdown, YAML, and JSON report rendering for
application-owned files or error bodies.
- Add scalar, keyed, unkeyed, nested, dynamic-key, dictionary-like, and
super-coder `Codable` conversion APIs with exact path-aware errors.
- Add fixture-backed compatibility coverage for scalar spellings, explicit tags,
collections, anchors, unsupported YAML gaps, literal block emission, downstream
documents, and deliberately failing validation fixtures.
- Add a real-world YAML fixture corpus with representative short, medium, large,
and very large documents, including OpenAPI, Kubernetes, GitHub Actions, Docker
Compose, Prometheus, and cert-manager YAML.
- Add macOS, Linux, Windows, and WASM verification gates, hosted CI, a full
local verification script, a pre-push hook, usage documentation, migration
boundaries, attribution, and release notes.
### Changed
- Lower the package tools version to Swift 6.1 for hosted macOS CI compatibility.
- Harden duplicate-key validation to use set membership while preserving exact
diagnostics.
- Make validation rules immutable after construction.
- Preserve keyed super-encoder field order even when delayed child encoders are
written in a different order than they were requested.
- Make keyed default `superEncoder()` and `superDecoder()` use the standard
`super` mapping key instead of the current mapping.
- Resolve Yams-compatible plain scalar spellings for `yes` and `no` booleans,
radix-prefixed integers, numeric separators, and `.inf` and `.nan` floats.
- Broaden opt-in literal block emission while preserving exact parser
round-trips for every emitted block scalar.
- Harden block mapping, multiline plain-scalar parsing, merge expansion, tagged
parsing, lower-column mapping siblings, indented root fragments, JSON-style
flow mapping pairs, tab-indented corpus files, and empty collection emission.