Int(_, radix:) accepts a sign, so 0-3 (a CPU range, a port range) was read as the integer -3, and 0x-1, 0b-1 and 0o-1 as negative numbers. After a radix prefix only digits may follow. Octal, hex, binary and signed integers read as before.
The emitter kept its own list of reserved words and number shapes,
apart from the parser's, and the two disagreed: the string yes was
written plain and read back as a boolean, as were 0x1F, 1_000 and
.inf. The parser's reading of a plain scalar is now one type that
both use, and the emitter quotes what does not read back as a string.
Also quoted: the merge key, the document end marker, and a string
with a CR+LF break, which was written raw.
enforceNestingLimit (Parser.Events.swift) bounds PHYSICAL collection
depth in the raw event stream, since it counts sequenceStart/
mappingStart events directly. It does nothing against an
anchor-expansion bomb: a handful of shallow, small anchors each
aliased many times, each alias nested inside the next, keeps raw
event depth at a constant 2 per line while the LOGICAL tree an alias
resolves to grows exponentially in the number of alias levels. A
25-level, fan-out-9 document (~1.5KB) expands past 9^25 logical
nodes; before this fix, parse accepted it in under 2ms, and any
full-tree walk downstream (JSON representability checking, JSON
writing, validation) never terminated.
Added a total expanded-node budget, charged incrementally as the
document streams through the composer rather than by walking
anything: every scalar charges 1, every completed container charges 1
for its own wrapper (children already charged their own cost as they
completed), and every alias resolution charges the target's CACHED
expanded size -- computed once when the anchor was defined, looked up
in O(1). This makes a nested bomb's true compounded size visible
arithmetically within a few alias levels, without ever materializing
or walking the exponential tree it would otherwise produce. A
document with genuinely wide but non-exponential alias reuse (the
same small anchor referenced thousands of times at one level, no
nesting) stays comfortably inside the budget and parses normally.
Verified via the full existing test suite (674 tests, all passing,
zero regressions -- including the YAML Test Suite conformance corpus
and the tagged round-trip corpus), two new tests proving the bomb is
rejected (in under 2ms) and that wide non-exponential fan-out still
parses, and swiftformat/swiftlint --strict clean on all five touched
files.
PureYAML.Parsing.EventComposer and Tagged.EventComposer (two other,
separate composer implementations in this package, used by the
diagnostic-validation and tagged-parsing entry points respectively)
share the same architectural gap and would benefit from the same
fix, but are not reachable from the confirmed exploit path this
change closes -- disclosed, not fixed here.
Seven correctness fixes to the recently landed anchors/styles, canonical,
timestamp/binary, and full-node-key work, each reproduced and covered by a
regression test.
- Stream dump doubled the `---` marker under explicitDocumentStart or
canonical: the single-document body now renders marker-free and the
stream path owns marker emission, so a two-document stream reparses as
two documents, not four.
- Tagged flow-mapping keys used block-context safety, so a key like `a,b`
emitted bare and reparsed as several entries; flow keys now use the
flow-aware plain-scalar check.
- Plain scalars ending in `:` reparse as an empty-valued mapping key, so
both the model and tagged emit paths now fall back to quoting them.
- An alias mapping key pointing at a defined non-scalar anchor raised the
misleading undefinedAlias; it now raises expectedScalarKey, and a truly
undefined alias key still raises undefinedAlias.
- Tagged duplicate-key detection keyed on source spelling: it now keys on
the resolved scalar value plus effective tag, so 0x1 and 1 collide while
1 and "1" do not and custom-tagged keys stay distinct.
- Dumping a tree nested past the emitter limit overflowed the stack.
- Timestamp now accepts a leap-second value of 60, counted as 60 elapsed
seconds in the epoch math.
- Tagged.Dumper honors sortKeys in its block and canonical paths for
all-plain-string-key mappings, matching the model dumper.
Nesting-guard design: dump is public and non-throwing across 90+ call
sites, so making it throwing would be a breaking change. A tree nested
deeper than the parser's own maxNestingDepth cannot round-trip through
PureYAML (the parser rejects it), so no faithful emission exists. Each
dump entry point now measures depth iteratively (no recursion, so the
guard itself cannot overflow) and, past the limit, returns a documented,
reparseable marker document that resolves to null. Under the limit the
existing recursive renderers run byte-identically, so parsed trees and
every realistic hand-built tree are unaffected. This mirrors the parser's
enforceNestingLimit, which bounds depth before any recursive consumer.
Closes#12 and closes#13. Tagged.Pair stores keyNode instead of a
bare string: alias mapping keys parse on the Tagged path (composer
resolves them through the anchors table, dumper re-emits the reference
with the mandatory space before the colon since a colon is a legal
anchor character), key anchors register for later aliases and re-emit,
and a key's presentation style survives dump and reparse. key and
keyTag remain as read-through computed accessors and the string-key
convenience init keeps most call sites source compatible. A style-less
hand-built key keeps the plain-when-safe emission convention. The
corpus round-trip law compares key nodes structurally; alias-key
documents move from rejected to round-tripping.
The validation module already implemented the composable-rule idiom under
its own names (Check/Issue/SubjectContext) and a value-type Validator with
duplicated traversal machinery per layer. Renamed the core types to the
canonical names (Validation/ValidationError/ValidationErrorCollection/
ValidationContext/BuiltinValidation/AnyValidation), converted Validator to
a reference-type class with the two-tier default-rule split, and genericized
it over both Subject and Document via a new Traversable protocol so the
model and tagged layers share one walk engine instead of two copies. Wired
the JSON-representability rule set into the same shape and closed the two
missing pieces (unwrap combinator, JSON coverage/config-pin tests).
Completes issue #11. Tagged.Scalar.timestampValue parses a !!timestamp
scalar under the YAML 1.1 grammar into the pure-Swift Tagged.Timestamp
(calendar components, timezone offset, secondsSinceEpoch via the
days-from-civil closed form; epoch test expectations measured against
Python's independent implementation). Tagged.Scalar.binaryValue base64
decodes a !!binary scalar into bytes, accepting unpadded and block
scalar wrapped content and rejecting corrupt input whole.
Yams hands back Foundation Date/Data; this package is mechanically
Foundation-free, so the bridges are caller-owned one-liners:
Date(timeIntervalSince1970:) and Data(_:). !!set and !!omap stay
skipped since Model.Mapping already preserves order and duplicates.
Closes#11
Completes issue #10. Canonical mode follows libyaml's canonical layout
on both the Model and Tagged dump paths (tags explicit, scalars double
quoted, flow one entry per line, anchors and aliases preserved on the
Tagged path). explicitDocumentStart/End wrap the body in --- and ...
markers. lineWidth wraps overflowing flow collections one entry per
line and folds plain block scalars at single-space boundaries, vetoing
any fold that would manufacture syntax, so values always reparse
unchanged.
Closes#10
swiftformat --lint failed on these pre-existing files and the pre-push
hook checks the whole tree, blocking every push until they conform.
The wrapped if bodies pushed YAMLTestSuiteGapsTests over the 400-line
lint limit, so its private JSONStreamScanner helper moves to its own
support file.
The 22-case TokenType mapping was split across three initializers to
stay under the complexity lint, leaving a preconditionFailure default
the types could not prove unreachable, which also tripped the
forbidden-patterns gate on every push. One exhaustive switch makes a
new TokenType case a compile error instead of a runtime trap; the lint
now ignores case statements when counting complexity, since exhaustive
closed-enum switches are the point, not a smell.
Tagged.Scalar/Sequence/Mapping gain anchor and style fields
(Tagged.ScalarStyle, Tagged.CollectionStyle), Tagged.Node gains an
alias(anchor:node:) case carrying both the reference and its resolution,
and parseTagged populates all of it from the parse events instead of
discarding anchors and expanding aliases into copies.
PureYAML.dump(_ node: Tagged.Node, options:) re-emits anchors, alias
references, tags, and per-node styles over the document-wide policy.
Plain scalars re-emit their source spelling (NO stays NO, an empty null
stays empty); literal blocks cover keep chomping, single-line strip
chomping, leading blank lines, and whitespace-led first lines via an
explicit indentation indicator; folded scalars re-emit as > when the
folding rules invert exactly and degrade to a literal block otherwise.
The round-trip law parseTagged(dump(parseTagged(d))) == parseTagged(d)
is corpus-tested over every real-world fixture, with one documented
degradation: single-quoted content containing line breaks re-emits
double-quoted, value unchanged.
Closes#9
Closes two of the four emitter gaps a feature-parity audit against Yams
found (PureYAMLYamsComparison's FEATURE-PARITY.md): no configurable
indentation and no sorted-key output.
indentWidth replaces every hardcoded indent + 2 in Dumper.swift/
Dumper.Mapping.swift, verified with round-trip tests since indentation is
purely a rendering choice and must never change what a document means.
sortKeys sorts a mapping's pairs by their string key in both block and flow
style, but only when every key in that mapping is a plain string; a mapping
with any complex (mapping/sequence) key stays in insertion order, since
there is no well-defined ordering across mixed key kinds. Off by default,
since insertion order is what makes a duplicate-key-preserving Mapping
round-trip meaningfully in the first place.
width-based line wrapping and canonical mode are each a correctness surface
of their own and are deliberately left for later, not folded into this.
PureYAML.parse() silently accepted raw C0 control characters anywhere in
the input; Yams's underlying libyaml rejects them unconditionally at the
reader level. Found by comparing parse() directly against Yams on a
real-world OpenAPI corpus. The printable-character predicate this needed
already existed (CharacterClass.isPrintable, matching libyaml's
c-printable set) but was only ever wired into anchor-name validation, and
the separate PreflightScanner control-character check is opt-in via
validate(), not part of what parse() itself runs.
Reader now records the index of the first non-printable scalar as it
buffers input (covering both batch and streaming), and fetchNextToken()
throws once scanning has actually reached that position, not merely once
one exists, so an earlier real syntax error is not hidden behind a later
control character.
Verified with 5 new tests and a full re-run of the official yaml-test-suite
conformance (333/333 parse-level, 208/208 value-level, unchanged) and the
whole-suite streaming differential (still clean), confirming no regression.
Plain scalars now suspend and resume mid-token instead of rolling back to a
checkpoint and rescanning from the token's start on every chunk boundary
inside them: scanPlainScalar's loop is an explicit PlainScalarPhase state
machine that throws PlainScalarSuspended the moment a lookahead runs dry, and
scanStep() resumes it directly from Scanner.suspendedPlainScalar. Getting
this right took four rounds of bugs, each caught by the existing whole-suite
streaming differential test: checking ranDry only once at the loop's end
instead of after every peek-dependent decision, resuming a dry-interrupted
content or break run through a pre-check meant only for a fresh entry point,
trusting dispatchToken's own dry-fooled document-marker check, and a nested
blankLineAhead() lookahead independently going dry before an outer check
could catch it.
A synthetic long-scalar test (needed since the real-world benchmark fixture's
scalars are too short to exercise this) then caught a fifth, genuine bug: the
first working resumePlainScalar took the whole suspended value by parameter,
which kept a reference to its accumulated string alive for the entire resume
call and defeated copy-on-write, copying the whole scalar on every resume.
That is O(N^2) in the scalar's length, and measured 29x slower than not
suspending at all at small chunk sizes. Passing state inout directly, with
the alias broken at the call site before the call, fixed it to 3-4x, verified
genuinely linear (not just a smaller quadratic constant) by a permanent test
scaling scalar length 16x at a fixed chunk size.
Also exposes this scanner as public API for the first time: scanStep() and
everything around it were internal-only, so no consumer outside this module,
including the separate PureYAMLStreaming package that actually wants a
resumable token scanner, could call it. PureYAML.Streaming.Scanner wraps
scanStep() directly with no independent logic, translating its tokens into
public Streaming.Token/Kind/ScalarStyle types that don't leak the libyaml
port's internal naming. Proven separately from the internal scanner with its
own differential test, since a wrong case mapping in the translation layer
wouldn't be caught by a proof that never looks at the public types.
Stops short of rewiring PureYAMLStreaming's own document-boundary scanner
onto this; that is separate, cross-repo work with its own build and CI cycle.
Add StreamingScannerPerformanceTests (issue #4), gated behind
PUREYAML_STREAMING_PERF_MEASURE=1, measuring scanStep() against scanAll() on
the largest real-world fixture (9.5MB) across a spread of chunk sizes.
Small chunk sizes (16-256 scalars) show a real, chunk-size-dependent rescan
cost, up to 6x at 16, consistent with the checkpoint-and-retry design
rescanning a fetch attempt once per chunk it spans. A control run with
isFinal: true from construction, where ranDry can never be set and no
rollback ever happens, still cost ~1.5x over scanAll(), proving a separate,
chunk-size-independent cost dominates at any realistic chunk size (4KB+): the
checkpoint was being copied on every fetch even when it could never be used.
Fixed by skipping the checkpoint entirely when reader.isFinal is true.
Re-measured rather than assumed fixed: the isolated control improved from
1.49x to 1.38x, a real but modest gain, while the large-chunk-size ratios
were unchanged within measurement noise. The remaining floor's cause is not
identified and is left open on issue #4 rather than patched further on
speculation; closing it needs a profiler, not another guess.
Whole-suite conformance (402/402) and the chunk-size differential proof both
still pass unchanged.
Add Scanner.scanStep(), the first increment of true chunked input streaming
(#4): Reader gains isFinal/append(_:)/markFinal(), and peek(_:) flags ranDry
instead of asserting end of input when the buffer runs out mid-fetch and more
chunks may still arrive. scanStep() snapshots the scanner before each
fetchNextToken() and restores it whenever a fetch runs dry, so a construct
spanning a chunk boundary is retried after more input lands rather than
misread as an early end of input. Every existing call site keeps the default
isFinal: true, so scanAll() and batch parsing are unchanged; ranDry can never
be set when isFinal is true, by construction. A token already queued is not
always safe to release, since libyaml's simple-key mechanism can still splice
a KEY token in front of it, so scanStep() ports libyaml's
yaml_parser_fetch_more_tokens release condition.
Proof: StreamingScannerDifferentialTests.swift drives scanStep() at several
chunk sizes over representative cases always, and over the entire
yaml-test-suite manifest (402 cases) behind PUREYAML_RUN_YAML_TEST_SUITE=1,
asserting the token sequence matches scanAll() exactly. That proof caught a
real bug: lastTokenForbidsInlineSequence() read indentEngine.tokens.last,
which goes empty once tokens are released incrementally, silently breaking
the key: - a rejection. Fixed by tracking IndentEngine.lastToken explicitly,
kept in sync with tokens.last so batch behavior (which never dequeues) is
unchanged.
Not yet wired into PureYAMLStreaming; scanStep() and its supporting types stay
internal until that integration is designed.
Also broaden the real-seed mutation property suite (#6) with the three
mutation classes it named but didn't yet cover: invalidControlCharacter,
mixedTabSpaceIndentation, and truncatedFlowCollection. The test now asserts
diagnostic quality, not just pass/fail: line must fall within the mutated
document, and column, when present, must be 1-based (some ParseError cases,
like tabIndentation, are line-only by design).
The ported engine wins every conflict: main's post-fork patches to the
hand-rolled Scanner and TokenEventParser land on deleted files and are
dropped; their behaviors are carried by the port and guarded by the
whole-suite conformance gate (402/402 passing).
Merge adaptations, all gates green (style, namespacing, forbidden
patterns, changelog, roadmap, swiftformat, swiftlint strict, build,
full test suite, whole-suite conformance sweep):
- PureYAMLJSON representability rules move onto the Check algebra,
replacing the retired legacy Rule type.
- The permissive tag-handle pre-pass is removed; the port rejects an
undefined handle at parse time (suite case QLJ7) and the validation
report carries it as a parse diagnostic.
- EventStreamComposer's nested types split one-per-file per the
namespacing gate.
- Tests pinned to old-engine behavior were re-verified against the
reference and updated: libyaml rejects an explicit-key continuation
ending at a value colon, and suite case QB6E requires rejecting a
quoted continuation at column 0.
- CHANGELOG unreleased entries from both lines are preserved in two
groups with a provenance note.
Closes#1, closes#3
The diagnostic accessor ran the scan through `try?`, which discarded the whole
scanner state, and the tag-handle issues collected on it, on any error: the
comment promised the issues found before a scan-aborting error were returned,
but the code returned none. Split the run into a state-capturing `collect` that
returns the partial state with the aborting error, so `scan` still throws while
`tagHandleIssues` reports the diagnostics gathered before the abort. A
regression test pins an undeclared handle followed by an unterminated quote.
In a flow mapping the value is a flow node and may sit on any line; flow is line
insensitive. parseMappingPair gated value parsing on hasNodeOnSameLine, a
block-only rule, so a value after `:` on the next line was treated as null and
left unconsumed, failing the whole flow mapping. Thread an inFlow flag and, in
flow, parse the value as a flow node line-insensitively (only a flow terminator
leaves it null), bypassing the block-only same-line and indentation branches.
Pretty-printed JSON (a YAML 1.2 superset) relies on this. Against the JSON Schema
Test Suite corpus this fixes all 27 files that previously failed to parse
(432/432 now); the YAML test-suite (318/333, value-level 206/206) and the full
suite stay green.
Fix three should-parse conformance gaps: a repeated document-end marker (M7A3),
a root block scalar ended by a document marker (W4TN), and an empty explicit key
in a flow mapping (DFF7). Threaded the located parse end mark through the new
explicit-key emit. Parse-level suite conformance reaches 318/333.
Per-document %TAG scoping and an undefinedTagHandle diagnostic via
validationReport, with Scanner.State and the tag helpers in their own files.
Resolved against main's Scanner.State by carrying its sawNonPropertyContentOnLine
field into the extracted State.
Opt-in located parse (parseLocated / parseStreamLocated) on the hand-rolled
engine, with exact scalar end positions. Resolved against main's extracted
parseScalarNode by threading the token end mark through it.
An explicit key (`?`) may have no key node: `? : v`, a trailing `? }` / `? ]` /
`? ,`, or `?` at the end of a flow mapping all denote a null key. The parser now
emits an empty key in those positions instead of failing to find a node before
the value indicator or terminator.
Fixes suite case DFF7 (parse-level conformance 316 -> 317); adds tests.
A `---` or `...` at column 0 ends an active block scalar rather than being
consumed as its content. The scanner now closes the scalar's indentation at the
marker, dedenting back to the document level, so the token stream stays balanced:
the dedent precedes the documentEnd. Previously the scalar's indent lingered past
the marker and surfaced as a stray late dedent plus a phantom empty document, and
a following directive was misread as a scalar.
Fixes suite case W4TN (parse-level conformance 315 -> 316); adds tests.
An explicit key that is only a node property (`? &e` then `: &a`) parsed to an
`expectedNode` error. Unlike the implicit form (`&a : v`), the scanner emits no
synthesized empty scalar for an explicit key, so the block parser read the key
via parseNode, consumed the property, then hit the `:` value indicator with no
node and threw.
parseNode now treats a `:` in node position as an empty key carrying any
properties, reusing emitEmptyNodeOrThrow (the node it already produced for a
property at end of input) without consuming the `:`, so the mapping machinery
reads the `:` as the value indicator. To stay within the cyclomatic-complexity
limit, the scalar case is extracted to parseScalarNode.
yaml-test-suite PW8X now passes (parse-level 314/333 to 315/333). should-error
rejections and value-level matches are unchanged, and a value-level test pins
the produced tree for the anchored and tagged property-only explicit keys.
A `...` after a document is already closed (only comments or blank lines since
the last `...`) ends nothing: a repeated document-end marker is valid YAML, so
consume it as a no-op rather than rejecting the stream as an unsupported
multi-document case. A second documentEnd is not emitted, so no phantom null
document is composed, and real content after the marker stays an error.
Fixes suite case M7A3 (parse-level conformance 313 -> 315); adds focused tests.
A block key that is only a node property (`&a : v`, `!!str : v`) parsed to an
`expectedNode` error. The scanner synthesized an empty mapping key for a bare
`: v` line but not when the line's sole content before the `:` was an anchor or
tag, so no key marker was emitted and the parser rejected the `:`.
Track non-property content per line (a leading `&`/`!` does not count, and a
block entry opens a fresh node position) and synthesize the empty key whenever
only properties precede the `:`. The property tokens already bind to the
synthesized empty scalar through the existing mapping machinery, so no parser
change is needed. Relocate the synthesis helper to Scanner.Helpers to keep
Scanner under the file-length limit.
yaml-test-suite FH7J now passes (parse-level 313/333 to 314/333); PW8X still
needs the empty-anchored explicit-value form. should-error rejections and
value-level matches are unchanged, and a value-level regression test pins the
produced tree for each property-only-key shape.
A tag shorthand using a named handle (`!handle!suffix`) must be declared by a
`%TAG` directive in the document that uses it; the handle does not carry across
documents (YAML 1.2). The parser was permissive but silent here, so an
undeclared handle passed unnoticed.
- Scope tag handles per document: a `%TAG` handle now applies only to the
document it precedes (merged onto the defaults at the document start, cleared
at the boundary) instead of accumulating across the whole stream. A directive
repeated for each document keeps working, as the suite requires.
- Detect the named-handle form before the primary-handle fallback (which
previously masked `!prefix!x` as a local tag) and record an undeclared handle.
The parse stays permissive: it still yields a value, and
`PureYAML.validationReport` surfaces the issue as an `undefinedTagHandle`
error. The check reads the scanner's tokens, so it is exact: a `!x!` inside a
scalar or comment is never flagged.
Extracts `Scanner.State` and the tag helpers into their own files to keep each
source file within the 400-line limit. Adds focused tests; the full suite and
strict lint stay green.
Port the located value tree onto the hand-rolled engine, reusing the
engine-agnostic located layer (Model.LocatedValue, LocatedPair,
SourcePosition, SourceRange, and the located composer helper) authored
on the conformance branch. The streaming composer now builds a parallel
located tree behind an opt-in tracksLocations flag, exposed as
parseLocated / parseStreamLocated; the plain Value path stays
byte-identical, gated behind the flag.
The scalar event carries the scanner token's end mark, so a located
scalar spans the whole token (end exclusive, one past the last
character) rather than being start-anchored. The hand-rolled scanner
already tracked token end marks, so this surfaces information the engine
was already computing.
Also stop emitting a spurious null document for a comment-only or empty
stream: YAML 1.2 gives such a stream zero documents, while the
single-document value API still maps it to null, so parse("") is
unchanged.
A whole-suite test proves the located projection equals a plain parse
for every passing case; the value, event-dump, and conformance suites
stay green (507 tests).
A zero-indent literal or folded block scalar sits at column 0, where the
directive scan runs, so a content line beginning with `%` (`%!PS-Adobe-2.0`)
was misread as a directive and rejected as unsupported.
The directive scan now runs only outside a block scalar, so a leading `%` in
block scalar content is literal text. A real `%YAML` or `%TAG` directive still
parses, and a stray `%` in other content is still rejected rather than silently
accepted, so the should-error behaviour is unchanged.
This is groundwork for yaml-test-suite W4TN, which additionally needs a block
scalar to be terminated by `...` and per-document directives across a multi-
document stream, so conformance counts are unchanged this step.
A block sequence value carrying an inline property (`sequence: !!seq` then
`- entry` at the key's column) is a zero-indent sequence: its entries sit at the
mapping key's own indentation, so a following sibling key ends it.
The untagged form was already parsed as a zero-indent sequence that stops at a
sibling mapping key, but the property-bearing form fell through to the generic
node path, which parsed the sequence without that terminator and read the
sibling key as a mixed sequence-and-mapping entry. The zero-indent sequence is
now handled before the generic path, carrying any property to the sequence.
Fixes yaml-test-suite 57H4; raises parse-level conformance to 313/333 (94.0%)
and value-level to 206/206 (100.0%), with the should-error rate unchanged.
An inline anchor with a tag on the next indented line, above a mapping indented
deeper still (`key: &anchor` / ` !!map` / ` a: b`), left the node introduced by
a further indent token after the properties were consumed. The parser could not
read a node from that indent and rejected it as a missing node.
The indented mapping value now descends through the deeper indent, carrying the
accumulated anchor and tag to the node it introduces, so the properties attach
to the mapping rather than to an empty value.
Fixes yaml-test-suite BU8L; raises parse-level conformance to 312/333 (93.7%)
and value-level to 205/205 (100.0%), with the should-error rate unchanged.
A node property written on its own indented line can sit above a block sequence
whose entries return to the mapping key's own indentation (`seq:` then
` &anchor` then `- a` at the key's column). The grammar allows a block-in-block
sequence at its parent's indent n while the property sits at n+1, so the
indent's matching dedent falls between the property and the entries.
The parser consumed the property as an empty anchored value and then rejected
the entries as an unexpected block entry. It now recognises the property-then-
dedent-then-entry shape, consumes that dedent, and binds the property to the
block sequence that follows.
Fixes yaml-test-suite SKE5; raises parse-level conformance to 311/333 (93.4%)
and value-level to 204/204 (100.0%), with the should-error rate unchanged.
An alias may stand where a mapping key is expected (`*a : value`). The scanner
emits the implicit-key indicator before a scalar key that a `:` follows, but
not before an alias, so the parser, which already accepts an alias node in key
position, never entered the pair and rejected the alias as an unexpected token.
The scanner now emits the same indicator before an alias that a `:` (after
optional spaces) follows, in block context. A `:` written directly against the
alias stays part of its name, since `:` is a valid anchor-name character, so
only a space-separated `:` marks the alias as a key.
Fixes yaml-test-suite E76Z and 26DV; raises parse-level conformance to 310/333
(93.1%) and value-level to 203/203 (100.0%), with the should-error rate
unchanged.
A property written inline before a block mapping's first key (`&a key: v`) was
attached to the whole mapping, so an anchor registered only when the mapping
ended, too late for an alias inside one of the mapping's values to resolve.
The YAML grammar requires a block collection's own properties to be followed by
a line break (`s-l+block-collection`), so a property on the first key's line
belongs to the key, not the collection. The mapping now keeps only a property
written on its own line above the first key; an inline one is threaded back to
the first pair, where the key parses it and an anchor registers with the key.
Fixes yaml-test-suite 2SXE (`&a:` and `&a` are distinct anchors, since a name
includes a trailing `:`) and HMQ5 (a key carrying both a tag and an anchor).
Raises parse-level conformance to 308/333 (92.5%) and value-level to 201/201
(100.0%), with the should-error rate unchanged.
An anchor on an empty mapping value (`a: &anchor`) failed to register, so a
later `*anchor` errored as an undefined alias. Two bugs compounded:
- The parser dispatched the anchor-only value position to a nested block
mapping, absorbing the sibling key as the anchored node's child instead of
emitting the empty anchored node. It now emits an empty anchored node when
only a sibling mapping key follows, since the indented and inline-nested
cases are already handled by earlier branches.
- The stream composer registered an anchor by its raw scalar text, so an
anchor on an empty scalar came back as "" and a typed scalar (`&a 42`) as
text. It now registers the composed value, fixing a latent typed-anchor bug.
Fixes yaml-test-suite 6KGN; raises parse-level conformance to 306/333 (91.9%)
and value-level to 199/199 (100.0%), with the should-error rate unchanged.
The %YAML version check accepted only 1.1 and 1.2, rejecting anything else,
so `%YAML 1.3` errored. The spec calls for processing a document whose minor
version is higher than the processor's, with a warning, rather than refusing
it: only a different major version is incompatible.
Accept any well-formed `1.minor` version and reject only a different major
(`2.0`) or a malformed version. Three tests that asserted `%YAML 1.3` was an
error are corrected to use `2.0`, the genuinely incompatible case, and a test
covers that 1.3 now parses.
Raises yaml-test-suite parse-level conformance to 305/333 (91.6%) with the
value-level (100%) and should-error rejection rates unchanged. 486 tests pass.
A block entry with an inline node recorded only the node's indent (one past
the dash) as a valid implicit level, not the dash's own indent. So a plain
scalar that is an inline sequence item value could not tell a sibling entry
that returns to the sequence indent from its own content, and a compact nested
sequence as an explicit key's value folded into one scalar: `? a` / `: - b` /
` - - c` / ` - d` became `{a: [b - - c - d]}` instead of `{a: [b, [c, d]]}`.
Record the entry's own indent as a sibling level too, so the continuation
stops there. Skip it under a pending explicit key, whose own sequence key
drives the implicit indentation differently (an explicit key whose key is a
sequence, `? - x`, must keep parsing). A block entry more indented than its
container is still plain content, so a wrapped value beginning a line with
`- ` folds in. The block-entry recording moved to a helper to keep the file
within its length limit.
Brings yaml-test-suite value-level conformance to 197/197 (100.0%) with the
parse-level and should-error rejection rates unchanged. 485 tests pass.
A blank line within a block scalar keeps any whitespace past the content
indentation as content, but the leading blank lines, those before the first
content line, were only counted and emitted as empty. So an otherwise-blank
`tab` line under a one-space content indent lost its tab, and a folded scalar
beginning with such a line dropped the leading tab.
Record each leading blank line's whitespace verbatim instead of a count, and
once the content indentation is established emit the surplus past it, the same
way an interior blank line does. A leading blank with no surplus still emits an
empty line, and an empty block scalar (no content indentation) still emits
empty lines for its blanks.
Raises yaml-test-suite value-level conformance to 196/197 (99.5%) with the
parse-level and should-error rejection rates unchanged. 482 tests pass.
A flow mapping entry may omit its `:` value indicator, in which case its key
maps to null: `{ a }` and `{ a, b }` are flow sets. parseFlowMappingPair
required a `:` after every key, so it rejected these entries, and rejected a
valueless key that spanned lines (`{ multi\n line, a: b}`) with them.
Parse a value only when a `:` follows the key; otherwise emit a null value for
the key. A unit test that asserted `{ a, }` was an error is corrected to expect
the set `{a: null}`, the behaviour the yaml-test-suite cases 8KB6 and 9BXH
require.
Raises yaml-test-suite parse-level conformance to 304/333 (91.3%) and
value-level to 195/197, with the should-error rejection rate unchanged.
480 tests pass.
A plain scalar continuation stops before a line ending at `:`, since a block
implicit key may not span lines and the next line is a new key. A flow key may
span lines, so the same rule wrongly split `{ multi\n line: v }`: the scalar
returned just `multi` and the parser mis-read `line: v` as a separate entry.
Fold across the `:` in flow as well, leaving the `:` to be scanned as the
value indicator, so the key folds to `multi line`. A block implicit key keeps
the previous behaviour and still stops before the `:`.
Raises yaml-test-suite parse-level conformance to 301/333 (90.4%) and
value-level to 193/195, with the should-error rejection rate unchanged.
476 tests pass.
A flow `?` did not mark an explicit key, so the scalar after it synthesized
its own implicit key: `{? foo : bar}` parsed as `{{foo: bar}: null}` instead of
`{foo: bar}`, and `[? foo : bar]` the same way.
Track the explicit-key state per flow level on the flow context: set it at a
flow `?`, clear it at that level's `:` or a `,`, and discard it with the
context when the collection closes. The key scalar then is not turned into an
implicit key, the pair's `:` is its value indicator, and a nested collection's
`:` does not affect an outer key (`{? {a: b} : c}`). The continuation fold for
a multi-line explicit key now also consults the flow context, restoring
`[? foo\n bar : baz]` to the single key `foo bar`.
The block explicit-key flag is left untouched, since it is not cleared between
valueless `? a` / `? b` entries and folding it into this check regressed
block sets; only the flow path is changed here.
Raises yaml-test-suite value-level conformance to 192/194 (99.0%) with the
parse-level and should-error rejection rates unchanged. 472 tests pass.
A plain scalar continuation that ends at a `:` is normally a new implicit
mapping key, so the scalar stops before it. Under an explicit key (`? foo`
then `bar :`) the `:` is the key's own value indicator instead, so the line is
still key content. The scalar dropped that line and the parser stalled, so
`? foo` then `bar : baz` failed rather than yielding the key `foo bar`.
When the explicit-key marker is pending, fold the continuation line in and
stop, leaving the `:` to be scanned as the value indicator. An implicit key
keeps the previous behaviour, so a continuation ending at `:` still begins a
new key.
The test-suite rates are unchanged, since no comparable case isolates the
block form and the flow form needs separate explicit-key work, but block
explicit keys spanning lines now fold correctly. 466 tests pass.
An explicit block scalar indentation indicator is the content indentation
relative to the block scalar's parent node. For an inline mapping value
(`- aaa: |2`) the parent is the mapping, but a compact inline mapping never
pushes its indentation onto the main stack, so the indicator was added to the
stack level (0) rather than the mapping's (2). The content indentation then
came out too small and the next mapping key was pulled into the scalar.
Capture at the header whether the block scalar is an inline mapping value
(a mapping value was seen on its line). When it is, fold the deepest
applicable implicit indentation into the indicator's base; a direct sequence
item (`- |1`), which has no mapping value on its line, keeps the sequence's
own level, so the explicit indicators on sequence items stay correct.
Raises yaml-test-suite value-level conformance to 191/194 (98.5%) with the
parse-level and should-error rejection rates unchanged. 463 tests pass.
A compact nested block sequence (`- - x`) carries no indent or dedent tokens,
only block entries at their source columns. The sibling-entry loop in
parseBlockSequence collected every following block entry by kind alone, so the
inner sequence absorbed an entry that belonged to an enclosing one: `- - x`
then `- - y` parsed as a single outer item `[[x, [y]]]` instead of two items
`[[x], [y]]`.
Record the first entry's column as the sequence's column and continue the
sibling loop only while the next entry sits at it. An entry at a smaller
column ends this sequence and is handled by the enclosing one; entries at the
same column still group, and flat sequences are unchanged.
Raises yaml-test-suite value-level conformance to 190/194 (97.9%) with the
parse-level and should-error rejection rates unchanged. 460 tests pass.
A block scalar whose only lines are blank never establishes a content
indentation, so its blank lines stayed in the leading-blank buffer and were
discarded when the scalar ended. `keep: |+` over a blank line then resolved to
the empty string rather than the line break keep chomping should preserve.
At stream end, flush the buffered blank lines as content wrapped in an
indent/dedent pair, so the parser builds a block scalar of those lines and
chomping applies: keep preserves the breaks, clip and strip trim them. Only
the stream-end case is handled, where no following line constrains token
ordering; an empty keep scalar before a sibling key is left for later.
Raises yaml-test-suite value-level conformance to 189/194 (97.4%) with the
parse-level and should-error rejection rates unchanged. 456 tests pass.
A node with an explicit tag is not subject to implicit plain-scalar
resolution, but composeScalarValue fell through to it whenever the tag was not
a recognized core type. So a local tag (`!foo`), the non-specific `!`, or an
unknown verbatim URI still let `! 12` resolve to the integer 12 and
`!Thing false` to a boolean.
When a tag is present but not a core type, fix the node as the string of its
content instead of resolving it. Recognized core tags and untagged scalars
are unchanged. Two compatibility tests asserted the old behavior (a custom-
tagged `true` as a boolean); they are corrected to expect the string, the
behavior the yaml-test-suite case S4JQ requires.
Raises yaml-test-suite value-level conformance to 188/194 (96.9%) with the
parse-level and should-error rejection rates unchanged. 452 tests pass.
A comment opens after white space, a tab as well as a space, but the plain
scalar terminator checked only for a space before `#`. So `c: d\t#X` kept the
tab and hash as content, giving the value `d\t#X` rather than `d` with a
trailing comment.
Treat a tab before `#` as a space there, in both the lookahead from a white
space character and the check on the accumulated value's last character. A tab
inside a plain scalar with no `#` after it stays content, and a `#` glued to
content with no white space is still content.
Raises yaml-test-suite value-level conformance to 187/194 (96.4%) with the
parse-level and should-error rejection rates unchanged. 448 tests pass.
In a folded block scalar, a blank run of N lines between two non-more-indented
lines folds to N line breaks: the fold absorbs the first break. Next to a
more-indented line nothing folds, so that break survives and the run yields
N+1 breaks. The folder used N in every case, so paragraph spacing around an
indented block collapsed: a blank line before a more-indented block came out
as one break instead of two.
Track whether the line before the blank run, or the one after it, is
more-indented and add the extra break in that case. Folding between plain
lines and runs with no blank lines are unchanged.
Raises yaml-test-suite value-level conformance to 186/194 (95.9%) with the
parse-level and should-error rejection rates unchanged. 444 tests pass.
Plain scalar line folding strips white space around continuation lines, but
the scanner stripped only spaces, so tabs leaked in. A tab is not indentation,
so the indentation scan left a continuation line's leading tab in place and it
joined the value (`\t3rd` folded as `\t3rd`, not `3rd`), and a line of only
white space holding a tab (` \t`) was treated as a non-blank continuation,
folding to a space rather than the line break an empty line gives.
Skip leading white space, tabs included, before a continuation line's content,
and treat a whitespace-only line that holds a tab as blank when counting the
folded line breaks.
Raises yaml-test-suite value-level conformance to 183/194 (94.3%) with the
parse-level and should-error rejection rates unchanged. 441 tests pass.
