In a folded block scalar, a blank run of N lines between two non-more-indented
lines folds to N line breaks: the fold absorbs the first break. Next to a
more-indented line nothing folds, so that break survives and the run yields
N+1 breaks. The folder used N in every case, so paragraph spacing around an
indented block collapsed: a blank line before a more-indented block came out
as one break instead of two.
Track whether the line before the blank run, or the one after it, is
more-indented and add the extra break in that case. Folding between plain
lines and runs with no blank lines are unchanged.
Raises yaml-test-suite value-level conformance to 186/194 (95.9%) with the
parse-level and should-error rejection rates unchanged. 444 tests pass.
Plain scalar line folding strips white space around continuation lines, but
the scanner stripped only spaces, so tabs leaked in. A tab is not indentation,
so the indentation scan left a continuation line's leading tab in place and it
joined the value (`\t3rd` folded as `\t3rd`, not `3rd`), and a line of only
white space holding a tab (` \t`) was treated as a non-blank continuation,
folding to a space rather than the line break an empty line gives.
Skip leading white space, tabs included, before a continuation line's content,
and treat a whitespace-only line that holds a tab as blank when counting the
folded line breaks.
Raises yaml-test-suite value-level conformance to 183/194 (94.3%) with the
parse-level and should-error rejection rates unchanged. 441 tests pass.
A block scalar is indented with spaces only, so a tab past the content
indentation is literal content. The indentation scan measured the leading
whitespace as a tab-expanded width and rebuilt the surplus beyond the
indentation as spaces, so a line like `␣\ttext` under a one-space indent came
out ` text` rather than `\ttext`: the tab became three spaces.
Record the literal leading-whitespace characters during the indentation scan
and reconstruct the surplus from them, walking columns the way the width scan
does and keeping the remainder verbatim. A tab stays a tab; surplus spaces
stay spaces; the blank-line surplus path uses the same reconstruction.
Raises yaml-test-suite value-level conformance to 180/194 (92.8%) with the
parse-level and should-error rejection rates unchanged. 438 tests pass.
skipSeparation consumed only spaces between tokens, so a tab after an
indicator leaked into the following node: an explicit key written `?\tkey`
became the key `\tkey` rather than `key`. A tab is separation white space in
YAML, forbidden only as indentation, which the line-start scan already handles
before this point.
Skip tabs there too. This closes the post-indicator tab leak left open when
the indicator boundary itself was changed, and handles a run of tabs between a
`:` and its value. The test-suite rates are unchanged, since no comparable
case isolates this, but the parser now matches the spec on tab separation.
435 tests pass.
A tab is separation white space in YAML, so it follows the `:` mapping-value
indicator exactly as a space does. isMappingValueBoundary (flow) and
isBlockMappingValueBoundary (block) accepted only a space, line break, comma,
or closing bracket, so `foo:\tbar` folded into the plain scalar `foo:\tbar`
instead of the mapping `{foo: bar}`.
Accept a tab in both. A `:` with no following white space (`foo:bar`) is still
plain scalar content, and the symmetric counterpart for a tab after a leading
indicator landed in the previous change.
Raises yaml-test-suite parse-level to 300/333 (90.1%) with the should-error
rejection rate unchanged. 433 tests pass.
A tab is separation white space in YAML, forbidden only for indentation, so it
ends a leading indicator exactly as a space does. isIndicatorBoundary treated
only a space, line break, or end of input as a boundary, so a document marker,
block entry, or explicit key followed by a tab was folded into a plain scalar:
`---\tscalar` parsed to the string `---\tscalar` rather than a document whose
root scalar is `scalar`.
Accept a tab there too. The document-marker and block-entry forms also strip
the tab as separation, so the content is clean; the explicit-key form is now
recognized as well, with post-indicator tab stripping left for a later change.
Raises yaml-test-suite value-level conformance to 175/192 (91.1%) with the
parse-level and should-error rejection rates unchanged. 429 tests pass.
A flow sequence element that begins with `:` is a single-pair mapping with an
empty key, e.g. `[ : value ]` is `[{null: value}]`, but parseFlowSequence ran
the element through parseNode, which found the leading `:` and raised
"expected a YAML node".
Detect the leading mapping-value indicator and emit a single-pair flow mapping
with a null key and the following value (itself possibly empty), the sequence
counterpart of the scalar-key element `[ a: b ]` that already becomes a
mapping through its implicit-key marker. Plain elements and scalar-key mapping
elements are untouched, and the allowance is confined to flow sequence element
position.
Raises yaml-test-suite parse-level to 298/333 (89.5%) with the should-error
rejection rate unchanged. 426 tests pass.
A flow mapping entry may begin with its `:` indicator for an empty key
(`{ : value }`) or carry only properties before the `:` (`{ !!str : value }`).
Both resolve to a null key, but parseFlowMappingPair required a node before
the `:` and raised "expected a YAML node".
Emit a null key in that position instead, mirroring the empty-value handling
already present after the `:`. The change consumes the key's properties first
so a property-only key keeps its tag or anchor, and is confined to flow
mapping key position, where a null key is unambiguously valid; an entry with
no `:` at all (`{ a, }`) still errors.
Raises yaml-test-suite parse-level to 297/333 (89.2%) and value-level to
174/192, with the should-error rejection rate unchanged (no invalid document
newly accepted). 422 tests pass.
A flow collection followed by a same-line `: ` indicator is an implicit block
mapping key, e.g. `[a, b]: value` or `{a: 1}: value`, but the parser took the
flow node as the document's root value and then rejected the trailing `:` with
"expected document boundary".
The marker that tells the parser a node is a key is emitted before the node,
so for a flow key it must precede the flow tokens, which are already drained
by the time the `:` is seen (the streaming cursor drains after each token).
Resolve it in the scanner with a character lookahead from the flow opener to
its matching close, skipping quoted scalars so their brackets and colons do
not miscount the nesting; when a same-line `:` boundary follows, emit the same
implicit-key marker a scalar key gets, so the flow node parses as the key
through the ordinary mapping path with no parser change.
A block implicit key must lie on one line, so a flow collection spanning a
line break before its `:` (`[23 / ]: 42`) is rejected, which a first cut
wrongly accepted until the lookahead was made to stop at a line break. A flow
node that no `:` follows, or that is a mapping value, is unaffected.
Raises yaml-test-suite parse-level to 294/333 (88.3%) with the should-error
rejection rate unchanged. 416 tests pass.
A block scalar that is the document's own root node (`>`, `|`, or after
`---` on the same line) may begin its content at column 0: the document node's
indentation is -1, so column-0 content is still more indented than its parent.
The scanner read the base level as 0 and required content strictly deeper than
it, so the lines fell through indentation handling and were mis-scanned as a
stray plain scalar after the document, raising "expected document boundary".
A root block scalar and an under-indented nested one (`- |` then unindented
content) present identical indentation state at the first content line, so the
two cannot be told apart there. Capture the distinction at header-scan time
instead: a root block scalar has the base indentation and no mapping value,
explicit key, block-entry implicit indent, or flow context introducing it on
its line. Only then is its parent the document; a sequence item or mapping
value keeps its parent's indent as the floor, so its under-indented content is
still rejected, unchanged.
Raises yaml-test-suite parse-level to 291/333 (87.4%) and value-level to
173/191 (90.6%). A regression suite covers the root cases (including a leading
'#' as literal content) and the nested under-indented cases that must error.
410 tests pass.
The coverage manifest claimed Tagged.Mapping and Tagged.Node validate key
uniqueness, but the tagged validator's only default check was the built-in
tag check: the claim was false. Tagged.Mapping keeps every parsed pair, so a
duplicate key survives into the tree and is as invalid as in the plain model.
Add Tagged.BuiltinCheck.mappingKeysAreUnique, a Check<Mapping, Node> on the
one validation framework, to the tagged validator's defaults; the traversal
now offers each mapping to the checks. It reports each duplicate at its key
path, with failing, near-miss, and nested fixtures, and the two default
tagged checks are pinned by description.
Correct the Model.Value coverage reason: merges and aliases resolve at parse
time, so they are not a validation-phase invariant. Every validated manifest
entry now names an invariant a check actually enforces. 403 tests pass.
A Rule was exactly a Check<Model.Value, Model.Value>: it fired on every
node, switched on the value itself, and carried the root. It was therefore a
second copy of the one check machinery, the duplication the validation idiom
exists to prevent (principle 10), now also resolved for the tagged layer.
Remove Rule and its Context. The model validator holds only checks; its
default is the type-directed BuiltinCheck.mappingKeysAreUnique; validating
takes a Check. Custom whole-tree rules become Check<Model.Value, Model.Value>
with the former context.root now context.document.
Behaviour is byte-identical: the mapping-key builtin reports the same issues
at the same paths in the same DFS order, so the public validate/collect/blank
surface is unchanged. All 399 tests pass; the coverage gate reconciles to 60
public types.
The tagged-node validator carried its own copy of the validation machinery:
a Tagged.Rule cloning Validation's check type and a Tagged.Context cloning
its context, differing only in subject type. That is the per-layer
duplication the validation idiom exists to prevent (principle 10).
Generalise the type-directed framework over both the subject and the
document: SubjectContext, Check, and AnyCheck now take a Document type
parameter for the validated tree's root. The model layer validates with a
Model.Value document, the tagged layer with a Tagged.Node document, each
truthful rather than forcing one root type to stand in for the other.
The tagged layer keeps only what is genuinely its own: the Node document
type, its built-in check (now Tagged.BuiltinCheck, a Check<Node, Node>), and
its traversal. Tagged.Rule and the tagged Context are removed. The validator
surface is unchanged but validating now takes a Check<Node, Node>.
All 399 tests pass; the coverage gate reconciles to 62 public types.
Add the declarative combinators the validation idiom composes rules from,
as free functions returning closures: comparison operators (==, !=, <, <=,
>, >=) lifting a key path and a literal into a predicate; take to dig to a
value and run arbitrary logic; all to apply many checks to one context; lift
to run child-typed checks against a value reached by key path; and && / ||
to compose two checks (concatenating their errors, or short-circuiting so
errors surface only when both fail). Both the context-rooted and
subject-rooted key-path forms are provided so an author can pin the subject
type in either clause.
A key path is immutable and read-safe but not declared Sendable, so a
@Sendable predicate captures it through a small @unchecked Sendable box.
lookup is omitted: it resolves references against a component store the
generic model does not carry; a consumer with references supplies it.
Each combinator has a pure-function test against a hand-built context, per
the idiom's requirement to unit test the operators in isolation.
Introduce the core of the validation-rules idiom: a Check<Subject> with a
positive description, a when predicate, and a Bool or multi-error check
that fires only on values of its Subject type wherever they occur. AnyCheck
erases the subject and dispatches by runtime type, guarding an optional or
a wrong type from matching. The validator gains a checks list, a
validating(_:) overload, and a traversal that offers each node the values
it may specialise on. BuiltinCheck.mappingKeysAreUnique is the
type-directed counterpart of the legacy rule.
The machinery has its own negative tests (wrong type, optional, false
predicate yield nothing; same type twice yields two errors) and the builtin
ships a failing and a near-miss test. The coverage gate catches the five
new public types, which are classified as validation machinery. The default
validator is unchanged, so this is additive; retiring the legacy Rule,
moving the default onto the check, the combinator algebra, the error
rendering, and de-duplicating the Tagged validator are the remaining steps.
Move the validator toward the validation-rules idiom's identity-based
configuration. withoutValidating(_:) removes a rule by its description, the
removal identity the idiom uses, so descriptions must be unique and stable.
validationDescriptions lists the active rules in application order.
A configuration-pin test asserts the exact descriptions of the blank and
default validators and that the default descriptions are unique, so adding,
removing, or rewording a rule is a deliberate change that a test catches.
Both additions are non-breaking; the type-directed Validation<Subject>
dispatch and the error-rendering change remain the larger migration steps.
Content after a '...' document-end marker begins a new bare document, but
the scanner rejected it as an unsupported multi-document stream and the
parser expected the document to end at the marker. A stream like
'--- scalar / ... / key: value' or 'a / ... / b' is two documents.
The scanner now reopens the stream when content follows '...', and
parseDocument returns at a '...' so the outer stream loop parses the next
document, bare or marked. A trailing '...' with nothing after it still
leaves a single document. The single-document composer still rejects a
multi-document stream, so 'parse' is unchanged. Two tests that asserted
content after '...' is an error are corrected, since the construct is valid
per the grammar and the official test-suite.
Measured against the official YAML test-suite, parse-level conformance
rises from 288/333 (86.5%) to 289/333 (86.8%) with no regression.
A node property always attaches to a node, and an empty node is allowed, so
an anchor or tag with nothing after it (a bare '&anchor', a 'key: &a', or a
'!!str'/'!!null' with no value) denotes that property applied to a null
node. parseNode rejected it as a missing node.
parseNode now emits a null scalar carrying the properties when a node
position holds only an anchor or tag, via a small helper to stay within the
complexity limit; with no properties, a missing node is still an error. A
'!!str' empty node resolves to the empty string and '!!null' to null. The
error-path test that asserted a dangling property throws is corrected, as
the construct is valid per the grammar and the official test-suite.
Measured against the official YAML test-suite, parse-level conformance
rises from 287/333 (86.2%) to 288/333 (86.5%), with value-level up one and
no regression.
A ':' that opens a line with no key before it is a mapping entry with a
null key (': value', a bare ':', or a null-keyed entry among others). The
scanner emitted the ':' with no preceding key, so the parser rejected it
as a missing node.
The scanner now synthesizes the empty key and its node for such a ':'. To
avoid mistaking an explicit key's value indicator for a null key, it tracks
a pending block-level '?' key: a ':' opening a line is that key's value
when one is pending, and only a synthesized null key otherwise. The flag
is set and cleared only at flow depth zero, so a ':' inside a flow complex
key ('? {a: b}' then ': c') does not disturb the pending block key.
Measured against the official YAML test-suite, parse-level conformance
rises from 283/333 (85.0%) to 287/333 (86.2%) with no regression.
A directive following a document-end marker ('...') begins the next
document's directive section, but the scanner rejected it: the
isDocumentClosed flag set by '...' tripped the directive guard, so a
multi-document stream where each document carries its own directives
('%TAG ...' / '--- doc' / '...' / '%TAG ...' / '--- doc') failed.
A directive now reopens the stream when it follows '...', clearing the
closed flag, while a directive after document content within the same
document stays rejected via the content guard. The directive still
requires its own '---', so a directive after '...' with no document start
is still an error. Multi-document streams already composed through the
stream mode; this removes the scanner-side block on per-document
directives.
Measured against the official YAML test-suite, parse-level conformance
rises from 278/333 (83.5%) to 283/333 (85.0%) with no regression.
The value-property indentation check used only the first property's mark,
so it caught a sole under-indented property ('seq:' then '&anchor') but not
a second one after an inline first ('key: &x' then a '!!tag' in column 1),
which still parsed.
The last consumed property is cursor.previous after consumeProperties, so
the check now uses it instead of the first mark, catching an under-indented
property whether it is the only one or a later one. Inline properties, a
property indented under the key, and a properly indented anchor-and-tag pair
are unaffected.
Measured against the official YAML test-suite, parse-level conformance
rises from 277/333 (83.2%) to 278/333 (83.5%) with no regression.
A block sequence entry cannot follow a flow collection that closed earlier
on the same line ('- { y: z }- invalid', '- [a] - b'): the entry would
begin a sequence inline after a complete node. The scanner accepted it,
producing extra sequence items.
The scanner now records when a flow collection closes at the top level on
the current line and rejects a following block entry. The flag resets at
each line break, so a flow collection on its own line is unaffected, and a
nested block entry ('- -') has no preceding flow close, so it stays valid.
Measured against the official YAML test-suite, parse-level conformance
rises from 276/333 (82.9%) to 277/333 (83.2%) with no regression.
A mapping value's anchor or tag on the line below the key must be indented
past the key. The parser consumed such a property regardless of its column,
so 'seq:' then '&anchor' in column 1, anchoring a sequence at the document
level, parsed instead of failing.
When the value's properties are consumed before the indent check, a
properly indented one would have been preceded by an indent token and
consumed inside the indented value; reaching that point with a property on
a later line therefore means it sits at the key's indentation, which is
rejected. The indented-value branch and the new guard move to their own
helpers so parseMappingPair stays within the complexity and length limits.
An inline property, an indented property, and a block sequence at the key's
indent are unaffected.
Measured against the official YAML test-suite, parse-level conformance
rises from 275/333 (82.6%) to 276/333 (82.9%) with no regression.
In a flow collection a '-' that begins a plain scalar must be followed by
a non-indicator character. Immediately before a flow indicator ('[-]',
'[-, -]', '{a: -}') the dash is not a valid scalar, but the scanner
accepted it as the scalar '-'.
scanToken now rejects a '-' in flow context when the next character is a
flow indicator, before falling through to plain-scalar scanning. A dash
followed by ordinary content stays valid: negative numbers, a dash inside
a scalar, a dash starting a word, and a quoted dash are unaffected.
Measured against the official YAML test-suite, parse-level conformance
rises from 273/333 (82.0%) to 275/333 (82.6%) with no regression.
A block scalar's leading empty lines must not be more indented than its
first non-empty line, so a blank line wider than the content (a 5-space
blank above 2-space content, or 2,3-space blanks above 1-space content) is
malformed. The scanner buffered the leading blanks without comparing their
indentation to the content, folding such input into the value.
The scanner now records the widest leading blank and rejects the scalar
when it exceeds the content indentation, established for both auto-detected
and explicit indicators. Indentation is counted in leading spaces, not
tab-expanded width, so a tab on a blank line ('␣\t') is one space rather
than a jump to the next tab stop, keeping such valid block scalars intact.
Measured against the official YAML test-suite, parse-level conformance
rises from 270/333 (81.1%) to 273/333 (82.0%) with no regression.
A line that begins with a document marker ('---' or '...') followed by a
boundary is forbidden inside a multi-line scalar, since it is ambiguous
with a document boundary. A multi-line quoted scalar folded such a line
into its value instead of rejecting it.
scanQuotedScalar now checks, on crossing each line break, whether the next
line begins a document marker, and rejects the scalar if so. The check
requires the marker at the line start with a following boundary, so inline
dashes ('a --- b'), '---' with no boundary ('---b'), an indented '---',
and dots not at a line start stay ordinary content.
Measured against the official YAML test-suite, parse-level conformance
rises from 268/333 (80.5%) to 270/333 (81.1%) with no regression.
A block collection must begin on the line after its node properties, so a
block sequence whose first entry sits on the same line as its anchor or tag
('&a - x', '!!seq - x') is malformed. The parser accepted it, anchoring or
tagging the inline sequence.
parseBlockSequence now rejects a sequence whose first block entry shares a
line with the anchor or tag that introduced it, via a small helper to stay
within the complexity and brace-style limits. An anchor or tag on its own
line above the sequence, inside a sequence item ('- &a x'), before a flow
sequence ('&a [x]'), or on a scalar is unaffected; the mapping form is
left alone, since '&a key: value' is a valid anchored key.
Measured against the official YAML test-suite, parse-level conformance
rises from 267/333 (80.2%) to 268/333 (80.5%) with no regression.
An alias is a bare reference and cannot carry node properties, so '&b *a'
(an anchor on an alias) and a tag on an alias are malformed. parseNode
ignored any pending anchor or tag when it reached an alias token, dropping
the property and accepting the document.
The alias case now rejects a pending anchor or tag, and moves to its own
parseAlias helper so parseNode stays within the complexity limit. A bare
alias and an anchor or tag on a real node are unaffected.
Measured against the official YAML test-suite, parse-level conformance
rises from 266/333 (79.9%) to 267/333 (80.2%) with no regression.
A comment terminates a plain scalar, so a multi-line plain scalar cannot
resume after one. The parser accepted 'key: word1' then a '#' line then
' word2', and 'a' then 'b # ...' then 'c', merging the trailing content
back into the value ('word1 word2', 'a b c').
The scanner folds a genuine multi-line plain scalar into a single token
and splits it into a value-line scalar, an indent, and a further scalar
only when a comment broke its continuation folding. That token shape
therefore always denotes a comment-interrupted scalar, so the parser now
rejects it rather than re-merging. The re-merge path and its helper are
removed as they only ever served this malformed shape. A real multi-line
plain scalar and a trailing comment after a value stay valid.
Measured against the official YAML test-suite, parse-level conformance
rises from 264/333 (79.3%) to 266/333 (79.9%) with no regression.
A block implicit key must lie on a single line, so a quoted scalar whose
opening and closing quotes sit on different lines cannot be one. The
scanner synthesized the implicit key regardless, so '"c\n d": 1' and the
single-quoted form parsed as a mapping with a folded key.
When synthesizing the implicit key for a quoted scalar followed by ':',
the scanner now rejects the construct if the scalar spanned a line break.
The restriction is block-only: a flow mapping key may span lines, so the
check is gated on block context, and a multiline quoted value, a single-
line quoted key, and an explicit key (which carries no ':' directly after
the scalar) are all unaffected.
Measured against the official YAML test-suite, parse-level conformance
rises from 261/333 (78.4%) to 264/333 (79.3%) with no regression.
A block sequence cannot be an implicit key's value on the same line as its
':' (key: - a); the value must begin on a following line, or use flow
style. The scanner accepted the inline block entry and parsed it as the
key's sequence value.
The block-entry case now rejects a '-' that starts a sequence when the
line already holds an implicit-key mapping value, reusing the same
same-line-pair flag as the inline nested mapping rule. As there, an
explicit- or null-key ':' opens its line and does not set the flag, so a
compact sequence value after it stays valid, as do a sequence on the next
line, a nested sequence, a flow sequence, and a negative number (key: -1).
Measured against the official YAML test-suite, parse-level conformance
rises from 260/333 (78.1%) to 261/333 (78.4%) with no regression.
Block style permits only one mapping pair per line, so a scalar followed
by ':' after a block-level value on the same line (a: b: c, a: 'b': c,
x: { y: z }in: valid, or trailing content after a quoted value) is not a
new key but a forbidden inline nested mapping. The scanner synthesized an
implicit key for any scalar followed by ':', regardless of position, so
these parsed as nested mappings instead of failing.
The scanner now records when a line already holds a block-level mapping
pair and suppresses a further implicit key on that line, in plain and
quoted scalars alike. The flag is set only for an implicit-key ':' (one
preceded by its key on the same line): a ':' that opens its line is an
explicit- or null-key value indicator, whose value may be a compact
mapping (? k then : a: 1), so that, flow mappings, next-line nesting, and
colons inside scalars stay valid.
Measured against the official YAML test-suite, parse-level conformance
rises from 256/333 (76.9%) to 260/333 (78.1%) with no regression.
A document-end marker takes no node on its own line: only a comment or
blank may follow. The scanner skipped the rest of the marker's line, so
content such as the 'invalid' in '... invalid' was silently dropped and
the document accepted.
scanDocumentMarker now distinguishes the two markers after skipping spaces:
a document start may carry the root node on its line and leaves it for the
main loop, while a document end rejects any real content (a comment or
blank still passes). A '...' not followed by an indicator boundary is not
a marker at all and remains an ordinary plain scalar, so '...junk' still
parses as content.
Measured against the official YAML test-suite, parse-level conformance
rises from 255/333 (76.6%) to 256/333 (76.9%) with no regression.
The YAML grammar requires every directive to head an explicit document,
so a directive must be followed by a '---' marker. The scanner accepted a
directive with no document at all: '%YAML 1.2' alone, or '%YAML 1.2' then
a '...' document-end marker, parsed without error.
The scanner now records the line of a directive awaiting its document,
clears it at a document start, and rejects a still-pending directive at a
document-end marker and at stream end. Both scan drivers finalize through
the same stream-end path, so the check lives in one helper called from
each. The rule applies to any directive, including an unrecognized one
(which is otherwise still ignored); the directive test that relied on an
unrecognized directive parsing without a '---' is corrected to include
one, since that leniency is not permitted by the grammar.
Measured against the official YAML test-suite, parse-level conformance
rises from 253/333 (76.0%) to 255/333 (76.6%) with no regression.
The %YAML directive takes exactly one parameter, the version, and at most
one may appear per document. The scanner checked only that a valid version
was present: it ignored any further tokens, so '%YAML 1.2 foo' was
accepted, and it did not track that a %YAML directive had already been
seen, so a repeated directive was accepted too. It now rejects an extra
parameter (allowing only a trailing comment) and a second %YAML directive,
resetting the per-document flag at each document boundary. A well-formed
directive, with or without a trailing comment, is unaffected.
Measured against the official YAML test-suite, parse-level conformance
rises from 251/333 (75.4%) to 253/333 (76.0%) with no regression.
In a flow sequence or mapping a comma separates entries, so one must
precede it. The parser advanced past a comma without checking that an
entry came before, accepting a leading comma ([ , a ]) or a repeated comma
([ a, , b ]) as empty entries. Each flow loop now tracks whether an entry
precedes the comma and rejects it otherwise. A single trailing comma
([ a, b, ]) and empty collections ([], {}) remain valid, since the closing
bracket is handled before that check.
Measured against the official YAML test-suite, parse-level conformance
rises from 249/333 (74.8%) to 251/333 (75.4%) with no regression.
The double-quoted escape decoder recognized only a subset of escapes
(" \ \/ \b \f \n \r \t \u) and silently passed any other character
through. That accepted invalid escapes such as \. and \', and mangled
valid ones the subset omitted: \x and \U hex escapes, \v \e \0, the named
\N \_ \L \P, and a backslash before a space, were emitted as their literal
letters (\x0d became the text x0d).
The decoder now maps every escape in the YAML 1.2 table to its character,
reads \x (8-bit) and \U (32-bit) hex escapes alongside \u, treats a
backslash before a line break as an escaped line break that joins the
lines with no separator, and rejects an unknown or truncated escape with
a diagnostic. Single-character escapes move to a lookup table so the
decoder stays within the complexity limit.
Measured against the official YAML test-suite, parse-level conformance
rises from 247/333 (74.2%) to 249/333 (74.8%) as two invalid documents
are now rejected, and value-level from 166 to 170 of comparable cases
(90.4%) as the previously-mangled escapes decode correctly, with no
regression.
A '#' that begins a comment must be preceded by whitespace or open the
line; glued to earlier same-line content (e.g. "value"# c, [ a, b ]#c,
>#c, a flow entry then ,#c) it is neither a comment nor, in those
positions, plain-scalar content, so the document is malformed. Such input
was previously accepted as a comment.
A '#' that opens a line, follows a space, or sits inside a plain scalar
(a#b, where it stays literal content and never reaches this check) is
unaffected. The scanner tracks whether a real token has been scanned on
the current physical line, reset on each line break, rather than comparing
token line numbers: the streaming tokenizer drains its token buffer
between scans, and a block scalar's blank line emits an empty scalar
attributed to the next line, either of which would misread as content.
Measured against the official YAML test-suite, parse-level conformance
rises from 243/333 (73.0%) to 247/333 (74.2%): four wrongly-accepted
invalid documents are now rejected, with no valid document regressing.
The scalar-scanning routines move to Scanner.Scalars.swift to keep the
file within the length limit.
A block scalar terminated by a less-indented comment line failed with a
spurious 'unexpected indentation': once the smaller indentation ended the
scalar, the comment's own column was processed as a dedent, and because no
indentation level held that width the dedent raised an error. A comment's
indentation does not participate in block structure, so a comment that ends
a block scalar is now left for the main loop to scan, with the block
scalar's own first content line (which may itself begin with '#') still
treated as content.
Separately, a whitespace-only line indented past a block scalar's content
level lost its surplus spaces because the width was consumed before the
line was recognized as blank; the surplus is now preserved as the line's
content, so an over-indented blank contributes the correct value.
Measured against the official YAML test-suite, parse-level conformance
rises from 240/333 (72.1%) to 243/333 (73.0%) with matching value-level
gains, and no regression.
A quoted mapping key followed by a space and then ':' ('"key" : value')
failed to parse: the scanner synthesizes the implicit mapping-key token by
checking the character immediately after the key, but a quoted scalar
stops at its closing quote, leaving the separating space, whereas a plain
scalar consumes that space into itself and ends at the colon. The quoted
path now looks past spaces for the colon, which the spec allows between an
implicit key and its value indicator.
Quoted keys with no space, quoted values, and plain keys (which already
worked with a space) are unaffected. Measured against the official YAML
test-suite, parse-level conformance rises from 238/333 (71.5%) to
240/333 (72.1%), with no regression.
A block mapping pair after the first could not carry an anchor or tag on
its key (k: 1 then &b key2: 2): the first pair's key properties are
consumed when the mapping is entered, but subsequent pairs were not, so an
anchored key past the first failed with 'expected mapping key, found
anchor'. The pair parser now consumes leading key properties for every
pair, attaching them to the key node, and a block mapping continues across
an anchor or tag token that begins a key.
Normal mappings, anchored values, sequences, and explicit keys are
unaffected. Measured against the official YAML test-suite, parse-level
conformance rises from 235/333 (70.6%) to 238/333 (71.5%), with no
regression.
The block mapping parser required a ':' after every explicit key ('? key'),
so a key with no value, including several '? key' entries in a row (a set),
failed to parse with 'expected mapping value'. An explicit key with no
following ':' now takes an implicit null value, shared with the existing
'key:'-with-nothing-after path via one emitNullMappingValue helper.
Explicit keys with values and complex keys in general are unaffected. A
test that asserted '? [a, b]' followed by a block entry is a missing-value
error was corrected: the key is a valid null entry, and the trailing
sequence is now rejected as extra content (a second root node), which is
the more accurate diagnosis.
Measured against the official YAML test-suite, parse-level conformance
rises from 229/333 (68.8%) to 235/333 (70.6%), with several of the newly
parsed documents also value-correct and no regression.
A directive the processor does not recognize (e.g. %FOO) before a
document is now ignored, matching the YAML spec (emit a warning, otherwise
no effect), rather than failing the whole document with an
unsupportedDirective error. %YAML and %TAG handling is unchanged, and a
directive after content, where a directive cannot legally appear, is still
rejected via the existing guard.
Two tests and a fixture that pinned the old reject-everything behavior are
updated: the unknown-directive case moves from the unsupported-gaps list,
and the scanner-error test now exercises a directive-after-content case
(still an error) rather than an unknown directive before content.
Measured against the official YAML test-suite, parse-level conformance
rises from 227/333 (68.2%) to 229/333 (68.8%), with the two newly-parsed
documents also value-correct and no regression.
A plain scalar at the document root failed to parse when it spanned
multiple lines at column 1 (`a\nb\nc`): the continuation check required a
continuation to be indented more than the scalar, but a root scalar has
nothing enclosing it, so its enclosing-block indent is -1 and a
continuation may start at column 1. The base indentation now returns -1 at
the root, and the continuation stops at a document marker (--- / ...) so a
scalar never folds across a document boundary.
A stream test that asserted two bare plain lines (first / second) were
rejected as extra content was corrected: that input is a valid multi-line
scalar; genuine extra content (a flow node followed by a scalar) is still
rejected, which the test now uses.
Measured against the official YAML test-suite, parse-level conformance
rises from 221/333 (66.4%) to 227/333 (68.2%), with four more documents
also now value-correct and no regression across the existing suite.
Single-quoted scalars share the same flow line folding as double-quoted,
which the previous commit added: a single break folds to a space, a blank
line to one newline, whitespace adjacent to a break is stripped, and the
first-line leading and last-line trailing whitespace are kept. The
single-quoted decoder kept its raw breaks; it now reuses the shared
foldFlowScalarLineBreaks helper, while doubled quotes ('') still escape to
one quote.
Measured against the official YAML test-suite, value-level conformance
rises from 143/169 (84.6%) to 147/169 (87.0%) with no regression.
Parse-level is unchanged: these documents already parsed, just to the
wrong tree.
The double-quoted decoder appended real line breaks and continuation-line
indentation verbatim, so a multi-line "..." scalar kept its raw newlines
and leading whitespace instead of folding. It now applies YAML flow
folding at each unescaped break: a single break folds to a space, a blank
line to one newline, and whitespace adjacent to a break (the line's
trailing, the next line's leading) is stripped. First-line leading and
last-line trailing whitespace stay (they sit next to a quote, not a
break), and \n escapes are still decoded to literal newlines.
Measured against the official YAML test-suite, value-level conformance
rises from 135/169 (79.9%) to 143/169 (84.6%) with no regression.
Parse-level is unchanged: these documents already parsed, just to the
wrong tree, which only the value-level metric exposes.
A document-start marker scan skipped the rest of the '---' line
unconditionally, so a root node written on that same line was discarded:
a block-scalar header ('--- >' / '--- |') was dropped and its body
mis-read as a plain scalar (folded to spaces, no trailing newline), and a
same-line plain or flow node was lost entirely. The document start now
skips only trailing spaces and leaves any real same-line content for the
parser; an empty rest-of-line or a comment is still consumed, and a node
on the line below the marker is unaffected. Document end ('...') still
takes no same-line node.
Measured against the official YAML test-suite, value-level conformance
rises from 130/170 (76.5%) to 135/169 (79.9%) with no regression.
Parse-level is unchanged: these documents already parsed, just to the
wrong tree, which only the value-level metric exposes.
A multi-line plain scalar whose value sits one line below its mapping key
stopped after the first line: the continuation check compared the next
line's indent against the value's own indent, which had been pushed onto
the indentation stack, so a continuation at the same indent failed the
strict '>'. It now compares against the parent block's indent (the level
below the value's, when the scalar begins exactly at the indentation top),
matching the spec: a continuation must exceed the mapping's indentation,
not the value's own. Inline values are unaffected, since their top-of-
stack already is the parent block's indent.
Measured against the official YAML test-suite, parse-level conformance
rises from 211/333 (63.4%) to 221/333 (66.4%) with no regression across
the existing 306 tests. The +10 covers the multi-line plain scalar family
the harness flagged, not just the case investigated.
Blank lines between a block scalar header and its first content line come
before the indent token, so the earlier blank-line capture skipped them.
Buffer leading blanks while the block indentation is still pending and
flush them as empty content once the indent is established, in the shared
indentation path used by both scan loops. Also fix folded bodies to emit
a break for a leading blank rather than folding it into a leading space.
This closes the last block-scalar blank-line edge from #57.
emptyDocument, expectedMappingKey, and tabIndentation were never thrown
anywhere in the parser. Empty and comment-only documents parse to null,
the mapping-key path throws unexpectedToken with an "expected mapping
key" message, and tabs in indentation are reported by the validation
layer's PreflightScanner rather than the parser. Drop the dead cases and
their switch arms and description test assertions. No reachable behavior
changes; the PropertyMutation.tabIndentation test mutation is unrelated
and untouched.
