The double-quoted escape decoder recognized only a subset of escapes
(" \ \/ \b \f \n \r \t \u) and silently passed any other character
through. That accepted invalid escapes such as \. and \', and mangled
valid ones the subset omitted: \x and \U hex escapes, \v \e \0, the named
\N \_ \L \P, and a backslash before a space, were emitted as their literal
letters (\x0d became the text x0d).
The decoder now maps every escape in the YAML 1.2 table to its character,
reads \x (8-bit) and \U (32-bit) hex escapes alongside \u, treats a
backslash before a line break as an escaped line break that joins the
lines with no separator, and rejects an unknown or truncated escape with
a diagnostic. Single-character escapes move to a lookup table so the
decoder stays within the complexity limit.
Measured against the official YAML test-suite, parse-level conformance
rises from 247/333 (74.2%) to 249/333 (74.8%) as two invalid documents
are now rejected, and value-level from 166 to 170 of comparable cases
(90.4%) as the previously-mangled escapes decode correctly, with no
regression.
19 KiB
19 KiB
Changelog
All notable changes to PureYAML are documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[Unreleased]
Added
- An opt-in conformance harness against the official YAML test-suite, the canonical
YAML 1.2 corpus (
Tests/YAMLTestSuiteConformanceTests.swift,scripts/fetch-yaml-test-suite.sh). Gated behindPUREYAML_RUN_YAML_TEST_SUITE=1and a separate fetch, so the data is never vendored. It reports two levels: parse-level (a should-parse case must parse; a should-error case must be rejected; failures histogrammed by cause) and the stricter, honest value-level, comparing the parsed tree to each case'sin.json. Value-level baseline: 130/170 (76.5%) of comparable single-document cases, exposing 40 documents that parse without error but to the wrong tree (e.g. double-quoted line folding, scalar resolution, block-scalar indentation) which parse-level cannot see. This turns "best YAML library" from an assertion into a tracked number.
Fixed
- Double-quoted escape sequences now follow the full YAML 1.2 escape table. Previously the
decoder recognized only a subset (
\" \\ \/ \b \f \n \r \t \u) and silently passed any other character through, so invalid escapes such as\.and\'were accepted while valid ones such as\x,\v,\e,\0,\N,\_,\L,\P,\U, and backslash-space were mangled (\x0dbecame the textx0d). The decoder now maps every defined escape to its character, handles\x(8-bit) and\U(32-bit) hex escapes alongside\u, treats a backslash before a line break as an escaped line break that joins the lines with no separator, and rejects an unknown or truncated escape. Lifts official test-suite parse-level conformance from 247/333 (74.2%) to 249/333 (74.8%) and value-level from 166 to 170 of comparable cases (90.4%), with no regression. - A
#glued to earlier content on its line (e.g."value"# c,[ a, b ]#c,>#c) is now rejected: the spec requires a comment to be preceded by whitespace or to open a line, and such input was previously accepted as a comment. A#that opens a line, follows a space, or sits inside a plain scalar (a#b, where it is literal content) is unaffected. The check tracks whether real content has been scanned on the current line, rather than comparing token line numbers, because the streaming tokenizer drains its buffer and a block scalar's blank line emits an empty scalar attributed to the next line. Lifts official test-suite parse-level conformance from 243/333 (73.0%) to 247/333 (74.2%); four wrongly-accepted invalid documents are now correctly rejected with no valid document regressing. - A block scalar followed by a less-indented comment line no longer fails with a spurious "unexpected indentation". When a comment ends a block scalar, its own column was processed as a dedent and, landing on a width no indentation level held, raised an error; a comment's indentation is now left out of block structure. Relatedly, a whitespace-only line indented past a block scalar's content level now keeps its surplus spaces as content instead of collapsing to empty, so such a line contributes the correct value. Lifts official test-suite parse-level conformance from 240/333 (72.1%) to 243/333 (73.0%) and value-level matches.
- A quoted mapping key may now have a space before its
:("key" : value). A plain key consumes the space before the colon into the scalar, but a quoted key stops at its closing quote, leaving the space, and the implicit-key check looked only at the immediate next character. It now looks past spaces for the colon, as the spec allows. Quoted keys with no space, quoted values, and plain keys are unaffected. Lifts official test-suite parse-level conformance from 238/333 (71.5%) to 240/333 (72.1%). - A block mapping pair after the first may now carry an anchor or tag on its key
(
k: 1/&b key2: 2). The first pair's key properties were consumed when the mapping was entered, but later pairs were not, so an anchored key past the first failed with "expected mapping key". The pair parser now consumes leading key properties for every pair, and the mapping continues across an anchor or tag that begins a key. Lifts official test-suite parse-level conformance from 235/333 (70.6%) to 238/333 (71.5%). - An explicit mapping key (
? key) with no following:now maps to null instead of failing to parse, so several? keyentries in a row (a set, for instance) each map to null. The parser required a:after every explicit key; it now treats a missing one as an implicit null value. Explicit keys with values, and complex keys generally, are unaffected. Lifts official test-suite parse-level conformance from 229/333 (68.8%) to 235/333 (70.6%). - An unrecognized directive (e.g.
%FOO) before a document is now ignored, as the YAML spec requires (a warning, otherwise no effect), instead of being rejected with an error.%YAMLand%TAGare unchanged, and a directive after content (where one cannot appear) is still rejected. Lifts official test-suite parse-level conformance from 227/333 (68.2%) to 229/333 (68.8%). - A plain scalar at the document root now folds across lines that sit at column 1, so a bare
multi-line value such as
a/b/cbecomes"a b c"instead of failing to parse. The continuation check required a continuation to be more indented than the scalar, but a root scalar has nothing enclosing it, so its enclosing-block indent is -1 and a continuation may start at column 1. The continuation now stops at a document marker (---/...), so a scalar never folds across a document boundary. Lifts official test-suite parse-level conformance from 221/333 (66.4%) to 227/333 (68.2%). (A stream test that asserted two bare plain lines were rejected as extra content was corrected: that input is a valid multi-line scalar; genuine extra content, like a flow node followed by a scalar, is still rejected.) - A multi-line single-quoted scalar now folds its line breaks, like the double-quoted fix below; both share the same flow line folding, so the fold logic is now one helper. Lifts official test-suite value-level conformance from 143/169 (84.6%) to 147/169 (87.0%), with no regression.
- A multi-line double-quoted scalar now folds its line breaks instead of keeping them raw.
The decoder appended real line breaks and continuation-line indentation verbatim, so
"a\n b"came back as"a\n b"rather than"a b". It now applies flow folding: a single break folds to a space, a blank line to one newline, and whitespace adjacent to a break (a line's trailing, the next line's leading) is stripped, while first-line leading and last-line trailing whitespace and\nescapes are preserved. Lifts official test-suite value-level conformance from 135/169 (79.9%) to 143/169 (84.6%), with no regression; parse-level is unchanged because these documents already parsed, just to the wrong tree. - A root node that begins on the document-marker line (
--- >,--- |,--- foo,--- {a: 1}) is no longer discarded. The marker scanner skipped the rest of the---line unconditionally, so a block-scalar header was dropped and its body was mis-read as a plain scalar (folded to spaces, no trailing newline) and a same-line plain or flow node was lost entirely. The document start now keeps any real same-line content for the parser, skipping only trailing spaces or a comment. Lifts official test-suite value-level conformance from 130/170 (76.5%) to 135/169 (79.9%), with no regression; parse-level is unchanged because these documents already parsed, just to the wrong tree, which only the value-level metric could see. - A multi-line plain scalar whose value begins on its own line under a mapping key
now folds its continuation lines instead of stopping after the first. The
continuation check compared the next line's indent against the value's own pushed
indent, so a continuation at the same indent failed the strict
>; it now compares against the parent block's indent, matching the spec (continuation must exceed the mapping's indentation, not the value's). Lifts official test-suite parse-level conformance from 211/333 (63.4%) to 221/333 (66.4%), with no regression across the existing suite.
Changed
- Split multi-type source files so each file declares exactly one top-level type
(
docs/rules/code-style.md): the Tagged node component types, the validation report types,TokenKind, and the event-stream composer helpers each move to their own file. Public type names are unchanged.scripts/check-namespacing.shis now namespace-aware and detects multiple sibling types declared insideextensionblocks, which the previous column-anchored scan missed. - Merge-key (
<<) override resolution now lives in a single internal helper,Parsing.MergeResolution, shared by the buffered and streaming event composers. Both paths previously carried an identical copy of the inherited-versus-local pair merge, so the subtle precedence rules (local keys win, earlier sources win, inherited-then-local ordering) now have one source of truth. No behavior change.
Removed
- Remove three
Parsing.ParseErrorcases that were never thrown:emptyDocument(empty and comment-only documents parse tonull),expectedMappingKey(superseded byunexpectedTokencarrying an "expected mapping key" message), andtabIndentation(tabs in indentation are reported by the validation layer'sPreflightScanner, not the parser). No reachable behavior changes.
Fixed
- Preserve blank lines in block scalars. The scanner dropped every blank line
within and after a block scalar, so a literal
|lost its internal blank lines, a folded>turned a blank line into a space instead of a newline, and the|+/>+keep indicator could not preserve trailing blank lines. Blank lines are now captured as empty content in both the batch and streaming scan paths (which had duplicated the loop), andblockScalarValueapplies literal versus folded line semantics and clip/strip/keep chomping correctly. A block scalar without blank lines is unchanged. - Keep a more-indented line inside a folded
>block literal, with its line breaks preserved, rather than folding it into the surrounding text. - Preserve leading blank lines between a block scalar header and its first content line. They precede the indent token, so they are buffered and emitted as empty content once the content indentation is established.
Documentation
- Correct
SECURITY.md: fix the reporting contact address and replace the scope section, which described an unrelated static-site generator, with the YAML parsing, validation, and emitting surface that is actually in scope.
[0.1.5] - 2026-06-23
Documentation
- Document why
maxNestingDepthis the necessary and sufficient safety bound for deeply nested input: aModel.Valueis a recursive tree whose every operation (compose, emit, compare, hash, and especially deallocate) recurses with nesting, and Swift cannot deallocate a recursive value type iteratively, so bounding the tree's depth at parse time is the only complete protection. Records the measured headroom, why the bound cannot be raised without an iterative parser (and would be pointless past the ~5,000deinitlimit), and the caveat that trees built by encoding a deepCodablevalue bypass it.
[0.1.4] - 2026-06-23
Fixed
- Reject documents nested beyond a fixed depth (
maxNestingDepth) with anestingTooDeepparse error instead of overflowing the recursive-descent parser's stack. A pathologically nested input (for example hundreds of nested[[[...]]]or deeply indented mappings) previously crashed the process; it now fails cleanly. The limit is well above the nesting of real-world specs, which are unaffected.
[0.1.3] - 2026-06-23
Fixed
- Parse a leading
#in block scalar content (>,>-,|,|-) as literal text instead of a comment. A block scalar whose first content line began with#, such as an OpenAPI$ref: >-folded onto#/components/..., was previously dropped to an empty string.
[0.1.2] - 2026-06-06
Added
- Document the real-world YAML corpus, fuzz/property gate, and CI artifact contract for production hardening.
- Add machine-readable parser diagnostic codes to validation report payloads and cover malformed production-shaped YAML with 10+ structured diagnostics.
- Vendor a 114-file real-world YAML corpus with a machine-readable manifest, default metadata checks, and an opt-in full parse/validate/round-trip gate.
- Add large public APIs.guru Stripe and Zoom OpenAPI YAML specs to the opt-in corpus gate.
- Add a reduced synthetic ACME Zephyr fixture set for Stitcher-style multi-file OpenAPI inputs and bundled output.
- Add a full renamed Birch-style ACME fixture catalog with 594 YAML files, preserving module/file structure while replacing private organization names, domains, and documentation URLs.
- Add YAML 1.2 JSON-subset compatibility for compact flow mappings such as
{"openapi":"3.0.0"}, including regression coverage for colons inside flow mapping values. - Add JSON compatibility tests and artifacts that compare PureYAML against Swift's JSON decoder across JSON roots, escapes, Unicode surrogate pairs, exponent numbers, nested values, and whitespace.
- Preserve round-trip fidelity for mapping keys that require quotes, including trailing-space keys found in the large Zoom OpenAPI corpus seed.
- Add an opt-in Yams differential script that compares parse behavior for the checked-in real corpus without adding Yams to the PureYAML package manifest.
- Add an opt-in Yams diagnostic comparison script that records PureYAML structured malformed-input diagnostics alongside Yams parser errors.
- Add an opt-in Yams validation comparison script proving structured duplicate key validation on YAML that both parsers can load.
- Add an opt-in Yams throughput script and scheduled/manual CI artifact for release-mode parser performance comparison on representative corpus seeds.
- Add an opt-in PureYAML phase profiler that times scanner, event parser, composer, lazy parse, validation, and dumping phases on slow corpus seeds.
- Compose normal parser output from events as they are emitted, avoiding a full
intermediate event array while keeping
parseEventsavailable for explicit event materialization. - Lazily scan tokens for normal
parseandparseStreamcalls, avoiding a retained full token array on the main parser path. - Track reader positions with a UTF-8 cursor so ASCII-heavy scanner advancement avoids per-character string allocation while preserving byte-accurate marks.
- Add deterministic generated corpus properties for valid YAML round trips and malformed real-seed mutations with structured diagnostics.
- Write corpus run artifacts under
.build/pureyaml-artifacts/, including real seed summaries, generated mutation summaries, seed metadata, and environment details. - Add a scheduled/manual CI corpus job that runs the corpus and Yams differential gates and uploads the generated artifact bundle.
[0.1.1] - 2026-06-06
Added
- Add platform CI badges and an active production-hardening roadmap to the README.
Changed
- Add release-mode build and test gates to the macOS, Linux, Windows, and WASM verification scripts.
- Update GitHub Actions checkout steps to
actions/checkout@v6so CI uses the current Node 24 action runtime.
Fixed
- Treat more-indented plain scalar continuation lines that start with
-as text unless they appear at a known sequence-entry indentation. This fixes OpenAPI-styleallOfdescriptions while preserving nested block sequences such as- - 123. - Parse explicit scalar keys whose
:value starts a same-line mapping and continues with indented mapping siblings, matching large OpenAPI schema keys.
[0.1.0] - 2026-06-05
Added
- Bootstrap PureYAML as a root Swift package with one
PureYAMLlibrary product, no external SwiftPM dependencies, no C sources, no generated parser targets, and no Foundation requirement in the library target. - Add the core pure-Swift YAML model with ordered mappings, sequences, scalars,
duplicate-key preservation, and first-class complex mapping keys through
Model.Pair.keyNode. - Add parser support for block mappings, block sequences, comments, flow collections, quoted strings, literal and folded block scalars, anchors, aliases, YAML directives, document markers, merge keys, complex mapping keys, multi-document streams, and selected explicit built-in scalar tags.
- Add deterministic YAML dumping with block output by default, optional flow collections, conservative plain scalars, conservative literal block scalars, complex mapping key dumping, and multi-document stream dumping.
- Add tag-preserving parsing with
PureYAML.parseTagged(_:)andPureYAML.parseTaggedStream(_:), plus tagged validation rules for unsupported built-in tags and tags applied to the wrong node kind. - Add
PureYAML.Tagged.Constructorfor caller-owned tagged construction with exact missing-handler and kind-mismatch errors, recursive path context, and a model-value fallback that preserves mapping order and duplicate keys. - Add path-aware validation with duplicate-key checks, custom rules, rule predicates, strict and non-strict behavior, warning collection, stream document indexes, deterministic validation paths, and direct value-tree validation for states ordinary loaders may collapse.
- Add diagnostic-first validation reports, raw-source preflight scanning,
Validation.ReportError, and Markdown, YAML, and JSON report rendering for application-owned files or error bodies. - Add scalar, keyed, unkeyed, nested, dynamic-key, dictionary-like, and
super-coder
Codableconversion APIs with exact path-aware errors. - Add fixture-backed compatibility coverage for scalar spellings, explicit tags, collections, anchors, unsupported YAML gaps, literal block emission, downstream documents, and deliberately failing validation fixtures.
- Add a real-world YAML fixture corpus with representative short, medium, large, and very large documents, including OpenAPI, Kubernetes, GitHub Actions, Docker Compose, Prometheus, and cert-manager YAML.
- Add macOS, Linux, Windows, and WASM verification gates, hosted CI, a full local verification script, a pre-push hook, usage documentation, migration boundaries, attribution, and release notes.
Changed
- Lower the package tools version to Swift 6.1 for hosted macOS CI compatibility.
- Harden duplicate-key validation to use set membership while preserving exact diagnostics.
- Make validation rules immutable after construction.
- Preserve keyed super-encoder field order even when delayed child encoders are written in a different order than they were requested.
- Make keyed default
superEncoder()andsuperDecoder()use the standardsupermapping key instead of the current mapping. - Resolve Yams-compatible plain scalar spellings for
yesandnobooleans, radix-prefixed integers, numeric separators, and.infand.nanfloats. - Broaden opt-in literal block emission while preserving exact parser round-trips for every emitted block scalar.
- Harden block mapping, multiline plain-scalar parsing, merge expansion, tagged parsing, lower-column mapping siblings, indented root fragments, JSON-style flow mapping pairs, tab-indented corpus files, and empty collection emission.