NodetteHQ/PureYAML
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(parsing): fold a document-root plain scalar across column-1 lines

Browse Source 
A plain scalar at the document root failed to parse when it spanned
multiple lines at column 1 (`a\nb\nc`): the continuation check required a
continuation to be indented more than the scalar, but a root scalar has
nothing enclosing it, so its enclosing-block indent is -1 and a
continuation may start at column 1. The base indentation now returns -1 at
the root, and the continuation stops at a document marker (--- / ...) so a
scalar never folds across a document boundary.

A stream test that asserted two bare plain lines (first / second) were
rejected as extra content was corrected: that input is a valid multi-line
scalar; genuine extra content (a flow node followed by a scalar) is still
rejected, which the test now uses.

Measured against the official YAML test-suite, parse-level conformance
rises from 221/333 (66.4%) to 227/333 (68.2%), with four more documents
also now value-correct and no regression across the existing suite.
This commit is contained in:
Mihaela M 2026-06-26 21:30:49 +02:00
parent 27e8defac6
commit 688f0d5c12
4 changed files with 66 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
CHANGELOG.md
View File

@ -23,6 +23,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- A plain scalar at the document root now folds across lines that sit at column 1, so a bare
multi-line value such as `a` / `b` / `c` becomes `"a b c"` instead of failing to parse. The
continuation check required a continuation to be more indented than the scalar, but a root
scalar has nothing enclosing it, so its enclosing-block indent is -1 and a continuation may
start at column 1. The continuation now stops at a document marker (`---` / `...`), so a
scalar never folds across a document boundary. Lifts official test-suite parse-level
conformance from 221/333 (66.4%) to 227/333 (68.2%). (A stream test that asserted two bare
plain lines were rejected as extra content was corrected: that input is a valid multi-line
scalar; genuine extra content, like a flow node followed by a scalar, is still rejected.)
- A multi-line single-quoted scalar now folds its line breaks, like the double-quoted fix
below; both share the same flow line folding, so the fold logic is now one helper. Lifts
official test-suite value-level conformance from 143/169 (84.6%) to 147/169 (87.0%), with

56
Sources/Parsing/Scanner.Helpers.swift
View File

@ -135,19 +135,7 @@ extension PureYAML.Parsing.Scanner {
}
probe.advance()
var blankLineCount = 0
while true {
var blankProbe = probe
_ = blankProbe.consume { character in
character == " "
}
guard isLineBreak(blankProbe.peek()) else {
break
}
blankProbe.advance()
probe = blankProbe
blankLineCount += 1
}
let blankLineCount = consumeBlankLines(&probe)
_ = probe.consume { character in
character == " "
@ -158,6 +146,12 @@ extension PureYAML.Parsing.Scanner {
return value
}
// A plain scalar never spans a document marker (`---` / `...`); stop the continuation at
// one so it begins (or ends) the document rather than folding into the scalar's value.
if indentationWidth == 0, isAtDocumentMarker(probe) {
return value
}
let baseIndentation = plainScalarContinuationBaseIndentation(state: state, firstLineIndent: firstLineIndent)
guard indentationWidth > baseIndentation else {
return value
@ -183,14 +177,40 @@ extension PureYAML.Parsing.Scanner {
}
}
/// Advance `probe` over a run of blank lines (lines that are empty or only spaces), returning
/// how many were consumed; the probe is left at the start of the next non-blank line.
func consumeBlankLines(_ probe: inout PureYAML.Parsing.Reader) -> Int {
var count = 0
while true {
var blankProbe = probe
_ = blankProbe.consume { character in
character == " "
}
guard isLineBreak(blankProbe.peek()) else {
break
}
blankProbe.advance()
probe = blankProbe
count += 1
}
return count
}
/// Whether the reader is positioned at a document marker (`---` or `...`) followed by an
/// indicator boundary, i.e. the marker that starts or ends a document.
func isAtDocumentMarker(_ reader: PureYAML.Parsing.Reader) -> Bool {
(startsWith("---", reader) || startsWith("...", reader)) && isIndicatorBoundary(reader.peek(offset: 3))
}
func plainScalarContinuationBaseIndentation(state: State, firstLineIndent: Int) -> Int {
// When the scalar begins exactly at the current indentation top, it is a block value on its
// own line, so its own indent was pushed onto the stack; its continuation lines sit at that
// same indent, and the level they must exceed is the parent block's (the level below), not
// the scalar's own. For an inline value the top of the stack already is the parent block's
// indent, so it is used directly.
if state.indentation.last == firstLineIndent, state.indentation.count >= 2 {
return state.indentation[state.indentation.count - 2]
// same indent, and the level they must exceed is the enclosing block's: the indent below on
// the stack, or -1 at the document root, where nothing encloses the scalar and a
// continuation may sit at column 1. For an inline value the top of the stack already is the
// enclosing block's indent, so it is used directly.
if state.indentation.last == firstLineIndent {
return state.indentation.count >= 2 ? state.indentation[state.indentation.count - 2] : -1
}
return state.indentation.last ?? 0
}

12
Tests/DocumentMarkerNodeTests.swift
View File

@ -17,4 +17,16 @@ struct DocumentMarkerNodeTests {
// A node still on its own line below the marker keeps working.
#expect(try PureYAML.parse("---\nplain text\n") == .string("plain text"))
}
@Test("A root plain scalar folds across lines, even at column 1")
func test_rootPlainScalarFoldsAcrossLines() throws {
// A document-root scalar has nothing enclosing it, so a continuation may sit at column 1.
#expect(try PureYAML.parse("a\nb \n c\nd\n\ne\n") == .string("a b c d\ne"))
}
@Test("A root plain scalar stops at a document marker, not folding across it")
func test_rootPlainScalarStopsAtDocumentMarker() throws {
let documents = try PureYAML.parseStream("document 1\n---\ndocument 2\n")
#expect(documents.map(\.value) == [.string("document 1"), .string("document 2")])
}
}

11
Tests/StreamParsingTests.swift
View File

@ -155,18 +155,21 @@ struct StreamParsingTests {
@Test("Stream parser rejects extra root content without a document start")
func test_streamParserRejectsExtraRootContentWithoutDocumentStart() {
// A complete flow node followed by a bare scalar is extra content and needs an explicit
// `---` to start a new document. (Two bare plain lines such as `first` / `second` are not
// extra content; they correctly fold into a single multi-line plain scalar.)
expectStreamParseError(
"""
first
second
[1, 2]
extra
""",
.unexpectedToken(
expected: "document boundary or stream end",
actual: "scalar value=\"second\" style=plain",
actual: "scalar value=\"extra\" style=plain",
line: 2,
column: 1,
),
description: "expected document boundary or stream end at line 2, column 1, found scalar value=\"second\" style=plain",
description: "expected document boundary or stream end at line 2, column 1, found scalar value=\"extra\" style=plain",
)
}