NodetteHQ/PureYAML
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(parsing): apply flow folding to quoted scalars

Browse Source 
Single (') and double (") quoted multi-line scalars now apply YAML 1.2 flow folding (sections 6.5, 7.3.1, 7.3.2): trailing whitespace before a break is stripped, leading whitespace on continuation lines is stripped, a single break between non-empty lines folds to one space, and N consecutive breaks yield N-1 literal line feeds (first and last lines keep their own leading/trailing spaces as content). A double-quoted backslash at end of line suppresses the break entirely.

Complete the double-quoted escape set: add \xXX and \UXXXXXXXX hex escapes plus the previously missing \0 \a \v \e, literal backslash-space, and named breaks \N \_ \L \P (existing \uXXXX surrogate-pair handling kept). The folding state machine mirrors scanner.c yaml_parser_scan_flow_scalar; implemented in Sources/Parsing/Scalar.swift.

Flips 30 official YAML test suite cases to passing (250/402). Updates the gap manifest, fixtures, count assertions, and roadmap.
This commit is contained in:
Mihaela Mihaljevic 2026-06-23 20:17:56 +02:00
parent 3db3e87d39
commit 88b67ff0b2
65 changed files with 263 additions and 601 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
CHANGELOG.md
View File

@ -24,6 +24,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
cases pass: 4QFQ, 5LLU, 6VJK, 7T8X, 96NN/00, 96NN/01, DWX9, F6MC, F8F9, H2RW,
JEF9/00, JEF9/01, K527, K858, L24T/00, L24T/01, M29M, M9B4, MJS9, P2AD, R4YG,
TS54, W9L4, and Y79Y/001.
- Implemented YAML 1.2 flow (quoted) scalar line folding for both single (`'`)
and double (`"`) quoted scalars, mirroring the reference scanner's
`yaml_parser_scan_flow_scalar`. Multi-line quoted scalars now fold correctly:
trailing whitespace before a break is stripped, leading whitespace on a
continuation line is stripped, a single break between non-empty lines folds to
one space, and N consecutive breaks yield N-1 literal line feeds (the first and
last lines keep their own leading and trailing spaces as content). A double
quoted backslash at end of line now suppresses the break entirely (no space, no
newline). Completed the double quoted escape set with `\xXX` (2 hex digits) and
`\UXXXXXXXX` (8 hex digits), and added the previously missing `\0 \a \v \e`,
literal `\<space>`, and the named breaks `\N \_ \L \P`, alongside the existing
`\uXXXX` (including surrogate pairs). This makes 30 additional official YAML
test suite cases pass: 3RLN/00, 3RLN/01, 3RLN/02, 3RLN/03, 3RLN/04, 3RLN/05,
4ZYM, 565N, 5GBF, 6WPF, 7A4E, 8UDB, 9SA2, 9TFX, DE56/00, DE56/01, DE56/02,
DE56/03, DE56/04, DE56/05, DK95/02, DK95/08, G4RS, NAT4, NP9H, PRH3, Q8AD,
T4YY, TL85, and XV9V.
### Added

313
Sources/Parsing/Scalar.swift
View File

@ -110,22 +110,8 @@ extension PureYAML.Parsing.Parser {
guard text.count >= 2, text.last == "'" else {
throw PureYAML.Parsing.ParseError.unterminatedQuotedString(line: line)
}
let inner = text.dropFirst().dropLast()
var output = ""
var iterator = inner.makeIterator()
while let character = iterator.next() {
if character == "'", let next = iterator.next() {
if next == "'" {
output.append("'")
} else {
output.append(character)
output.append(next)
}
} else {
output.append(character)
}
}
return output
let inner = Array(text.dropFirst().dropLast())
return foldFlowScalar(inner, single: true)
}
func parseDoubleQuoted(
@ -135,94 +121,238 @@ extension PureYAML.Parsing.Parser {
guard text.count >= 2, text.last == "\"" else {
throw PureYAML.Parsing.ParseError.unterminatedQuotedString(line: line)
}
let inner = text.dropFirst().dropLast()
let inner = Array(text.dropFirst().dropLast())
return foldFlowScalar(inner, single: false)
}
/// Flow (quoted) scalar line folding, mirroring libyaml's
/// yaml_parser_scan_flow_scalar (single and double quoted). Trailing
/// whitespace before a break is dropped, leading whitespace on a
/// continuation line is dropped, a single break between non-empty lines
/// folds to one space, and N consecutive breaks yield N-1 line feeds.
/// For double quoted scalars a backslash at end of line suppresses the
/// break entirely.
func foldFlowScalar(
_ inner: [Character],
single: Bool,
) -> String {
var state = FlowFoldState()
let count = inner.count
while state.index < count {
state.leadingBlanks = false
consumeFlowContent(inner, single: single, state: &state)
consumeFlowBlanks(inner, state: &state)
joinFlowFold(state: &state)
}
return state.output
}
/// Mutable accumulators for flow scalar folding, matching the C reference's
/// string / leading_break / trailing_breaks / whitespaces buffers.
struct FlowFoldState {
var output = ""
var index = inner.startIndex
while index < inner.endIndex {
let character = inner[index]
if character == "\\" {
inner.formIndex(after: &index)
guard index < inner.endIndex else {
output.append("\\")
var whitespaces = ""
var leadingBreak = ""
var trailingBreaks = ""
var leadingBlanks = false
var index = 0
}
/// Consume the run of non-blank characters at `state.index`, decoding
/// escapes. A double quoted backslash at end of line sets `leadingBlanks`
/// and stops, so the following break is suppressed during folding.
func consumeFlowContent(
_ inner: [Character],
single: Bool,
state: inout FlowFoldState,
) {
let count = inner.count
while state.index < count, !isFlowBlank(inner[state.index]), !isFlowBreak(inner[state.index]) {
if single, inner[state.index] == "'" {
state.output.append("'")
let isEscapedQuote = state.index + 1 < count && inner[state.index + 1] == "'"
state.index += isEscapedQuote ? 2 : 1
continue
}
if !single, inner[state.index] == "\\" {
if state.index + 1 < count, isFlowBreak(inner[state.index + 1]) {
state.index += 2
state.leadingBlanks = true
break
}
appendDoubleQuotedEscape(in: inner, index: &index, to: &output)
} else {
output.append(character)
inner.formIndex(after: &index)
state.index += 1
appendDoubleQuotedEscape(inner, index: &state.index, to: &state.output)
continue
}
state.output.append(inner[state.index])
state.index += 1
}
return output
}
func appendDoubleQuotedEscape(
in text: Substring,
index: inout Substring.Index,
to output: inout String,
/// Consume the run of blanks and line breaks at `state.index`, sorting them
/// into the whitespace / leading-break / trailing-break buffers.
func consumeFlowBlanks(
_ inner: [Character],
state: inout FlowFoldState,
) {
let character = text[index]
switch character {
case "\"": output.append("\"")
case "\\": output.append("\\")
case "/": output.append("/")
case "b": output.append("\u{08}")
case "f": output.append("\u{0C}")
case "n": output.append("\n")
case "r": output.append("\r")
case "t": output.append("\t")
case "u":
appendUnicodeEscape(in: text, index: &index, to: &output)
default: output.append(character)
let count = inner.count
while state.index < count, isFlowBlank(inner[state.index]) || isFlowBreak(inner[state.index]) {
if isFlowBlank(inner[state.index]) {
if !state.leadingBlanks {
state.whitespaces.append(inner[state.index])
}
} else if !state.leadingBlanks {
state.whitespaces = ""
state.leadingBreak = "\n"
state.leadingBlanks = true
} else {
state.trailingBreaks.append("\n")
}
state.index += 1
}
text.formIndex(after: &index)
}
func appendUnicodeEscape(
in text: Substring,
index: inout Substring.Index,
to output: inout String,
) {
let escapeStart = index
guard let codeUnit = readUnicodeEscapeCodeUnit(in: text, index: &index) else {
output.append("u")
/// Join the captured whitespace or fold the captured line breaks into the
/// output, per YAML 1.2 flow folding.
func joinFlowFold(state: inout FlowFoldState) {
guard state.leadingBlanks else {
state.output.append(state.whitespaces)
state.whitespaces = ""
return
}
let surrogatePair = readSurrogatePairIfNeeded(high: codeUnit, in: text, index: &index)
if let scalar = surrogatePair {
if state.leadingBreak.first == "\n" {
if state.trailingBreaks.isEmpty {
state.output.append(" ")
} else {
state.output.append(state.trailingBreaks)
state.trailingBreaks = ""
}
state.leadingBreak = ""
} else {
state.output.append(state.leadingBreak)
state.output.append(state.trailingBreaks)
state.leadingBreak = ""
state.trailingBreaks = ""
}
}
func isFlowBlank(_ character: Character) -> Bool {
character == " " || character == "\t"
}
func isFlowBreak(_ character: Character) -> Bool {
character == "\n" || character == "\r" || character == "\r\n"
}
/// Decode a double quoted escape. On entry index points at the escape
/// character (the position just after the backslash). On return index has
/// advanced past every character the escape consumed. The escape set
/// mirrors libyaml's yaml_parser_scan_flow_scalar.
func appendDoubleQuotedEscape(
_ text: [Character],
index: inout Int,
to output: inout String,
) {
guard index < text.count else {
output.append("\\")
return
}
let character = text[index]
if let simple = Self.simpleDoubleQuotedEscapes[character] {
output.append(simple)
index += 1
return
}
switch character {
case "x":
index += 1
appendHexEscape(text, index: &index, digits: 2, marker: "x", to: &output)
case "u":
index += 1
appendUnicodeEscape(text, index: &index, to: &output)
case "U":
index += 1
appendHexEscape(text, index: &index, digits: 8, marker: "U", to: &output)
default:
output.append(character)
index += 1
}
}
/// Single character double quoted escapes, matching libyaml's escape table.
static let simpleDoubleQuotedEscapes: [Character: Character] = [
"0": "\u{00}",
"a": "\u{07}",
"b": "\u{08}",
"t": "\t",
"\t": "\t",
"n": "\n",
"v": "\u{0B}",
"f": "\u{0C}",
"r": "\r",
"e": "\u{1B}",
" ": " ",
"\"": "\"",
"/": "/",
"\\": "\\",
"N": "\u{85}",
"_": "\u{A0}",
"L": "\u{2028}",
"P": "\u{2029}",
]
/// Decode \xXX or \UXXXXXXXX. On entry index points at the first hex digit.
func appendHexEscape(
_ text: [Character],
index: inout Int,
digits: Int,
marker: Character,
to output: inout String,
) {
guard let value = readHex(text, index: &index, digits: digits) else {
output.append(marker)
return
}
if let scalar = UnicodeScalar(value) {
output.unicodeScalars.append(scalar)
} else {
output.append(marker)
}
}
/// Decode \uXXXX, including surrogate pairs. On entry index points at the
/// first hex digit.
func appendUnicodeEscape(
_ text: [Character],
index: inout Int,
to output: inout String,
) {
let start = index
guard let codeUnit = readHex(text, index: &index, digits: 4) else {
output.append("u")
index = start
return
}
if isHighSurrogate(codeUnit), let scalar = readSurrogatePair(high: codeUnit, text: text, index: &index) {
output.unicodeScalars.append(scalar)
} else if let scalar = UnicodeScalar(codeUnit) {
output.unicodeScalars.append(scalar)
} else {
output += text[escapeStart ... index]
output.append("u")
}
}
func readSurrogatePairIfNeeded(
high: UInt32,
in text: Substring,
index: inout Substring.Index,
) -> UnicodeScalar? {
guard isHighSurrogate(high) else {
return nil
}
return readSurrogatePair(high: high, in: text, index: &index)
}
func readSurrogatePair(
high: UInt32,
in text: Substring,
index: inout Substring.Index,
text: [Character],
index: inout Int,
) -> UnicodeScalar? {
var probe = index
text.formIndex(after: &probe)
guard probe < text.endIndex, text[probe] == "\\" else {
guard probe + 1 < text.count, text[probe] == "\\", text[probe + 1] == "u" else {
return nil
}
text.formIndex(after: &probe)
guard probe < text.endIndex, text[probe] == "u",
let low = readUnicodeEscapeCodeUnit(in: text, index: &probe),
isLowSurrogate(low)
else {
probe += 2
guard let low = readHex(text, index: &probe, digits: 4), isLowSurrogate(low) else {
return nil
}
index = probe
@ -230,22 +360,25 @@ extension PureYAML.Parsing.Parser {
return UnicodeScalar(value)
}
func readUnicodeEscapeCodeUnit(
in text: Substring,
index: inout Substring.Index,
/// Read exactly `digits` hex characters starting at index. On success index
/// advances past them, otherwise index is left unchanged.
func readHex(
_ text: [Character],
index: inout Int,
digits: Int,
) -> UInt32? {
var probe = index
text.formIndex(after: &probe)
var hex = ""
for _ in 0 ..< 4 {
guard probe < text.endIndex, text[probe].isHexDigit else {
guard index + digits <= text.count else {
return nil
}
var value: UInt32 = 0
for offset in 0 ..< digits {
guard let digit = text[index + offset].hexDigitValue else {
return nil
}
hex.append(text[probe])
text.formIndex(after: &probe)
value = (value << 4) + UInt32(digit)
}
index = text.index(before: probe)
return UInt32(hex, radix: 16)
index += digits
return value
}
func isHighSurrogate(_ value: UInt32) -> Bool {

270
Tests/Fixtures/yaml-test-suite-gaps.yaml
View File

@ -118,60 +118,6 @@ cases:
current: accepted
byteCount: 27
lineCount: 3
- id: "3RLN/00"
name: "Leading tabs in double quoted"
localPath: yaml-test-suite/3RLN/00/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 22
lineCount: 2
- id: "3RLN/01"
name: "Leading tabs in double quoted"
localPath: yaml-test-suite/3RLN/01/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 22
lineCount: 2
- id: "3RLN/02"
name: "Leading tabs in double quoted"
localPath: yaml-test-suite/3RLN/02/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 21
lineCount: 2
- id: "3RLN/03"
name: "Leading tabs in double quoted"
localPath: yaml-test-suite/3RLN/03/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 24
lineCount: 2
- id: "3RLN/04"
name: "Leading tabs in double quoted"
localPath: yaml-test-suite/3RLN/04/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 24
lineCount: 2
- id: "3RLN/05"
name: "Leading tabs in double quoted"
localPath: yaml-test-suite/3RLN/05/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 23
lineCount: 2
- id: "4CQQ"
name: "Spec Example 2.18. Multi-line Flow Scalars"
localPath: yaml-test-suite/4CQQ/in.yaml
@ -226,15 +172,6 @@ cases:
current: value-mismatch
byteCount: 35
lineCount: 4
- id: "4ZYM"
name: "Spec Example 6.4. Line Prefixes"
localPath: yaml-test-suite/4ZYM/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 70
lineCount: 7
- id: "55WF"
name: "Invalid escape in double quoted string"
localPath: yaml-test-suite/55WF/in.yaml
@ -244,15 +181,6 @@ cases:
current: accepted
byteCount: 9
lineCount: 2
- id: "565N"
name: "Construct Binary"
localPath: yaml-test-suite/565N/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 636
lineCount: 12
- id: "57H4"
name: "Spec Example 8.22. Block Collection Nodes"
localPath: yaml-test-suite/57H4/in.yaml
@ -262,15 +190,6 @@ cases:
current: parse-error
byteCount: 67
lineCount: 6
- id: "5GBF"
name: "Spec Example 6.5. Empty Lines"
localPath: yaml-test-suite/5GBF/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 83
lineCount: 8
- id: "5TRB"
name: "Invalid document-start marker in doublequoted tring"
localPath: yaml-test-suite/5TRB/in.yaml
@ -388,15 +307,6 @@ cases:
current: parse-error
byteCount: 86
lineCount: 8
- id: "6WPF"
name: "Spec Example 6.8. Flow Folding [1.3]"
localPath: yaml-test-suite/6WPF/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 32
lineCount: 8
- id: "6ZKB"
name: "Spec Example 9.6. Stream"
localPath: yaml-test-suite/6ZKB/in.yaml
@ -415,15 +325,6 @@ cases:
current: parse-error
byteCount: 62
lineCount: 5
- id: "7A4E"
name: "Spec Example 7.6. Double Quoted Lines"
localPath: yaml-test-suite/7A4E/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 50
lineCount: 4
- id: "7LBH"
name: "Multiline double quoted implicit keys"
localPath: yaml-test-suite/7LBH/in.yaml
@ -487,15 +388,6 @@ cases:
current: parse-error
byteCount: 51
lineCount: 4
- id: "8UDB"
name: "Spec Example 7.14. Flow Sequence Entries"
localPath: yaml-test-suite/8UDB/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 89
lineCount: 8
- id: "8XDJ"
name: "Comment in plain multiline value"
localPath: yaml-test-suite/8XDJ/in.yaml
@ -613,24 +505,6 @@ cases:
current: parse-error
byteCount: 15
lineCount: 3
- id: "9SA2"
name: "Multiline double quoted flow mapping key"
localPath: yaml-test-suite/9SA2/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 57
lineCount: 4
- id: "9TFX"
name: "Spec Example 7.6. Double Quoted Lines [1.3]"
localPath: yaml-test-suite/9TFX/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 54
lineCount: 5
- id: "9WXW"
name: "Spec Example 6.18. Primary Tag Handle"
localPath: yaml-test-suite/9WXW/in.yaml
@ -775,60 +649,6 @@ cases:
current: parse-error
byteCount: 26
lineCount: 4
- id: "DE56/00"
name: "Trailing tabs in double quoted"
localPath: yaml-test-suite/DE56/00/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 23
lineCount: 2
- id: "DE56/01"
name: "Trailing tabs in double quoted"
localPath: yaml-test-suite/DE56/01/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 25
lineCount: 2
- id: "DE56/02"
name: "Trailing tabs in double quoted"
localPath: yaml-test-suite/DE56/02/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 23
lineCount: 2
- id: "DE56/03"
name: "Trailing tabs in double quoted"
localPath: yaml-test-suite/DE56/03/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 25
lineCount: 2
- id: "DE56/04"
name: "Trailing tabs in double quoted"
localPath: yaml-test-suite/DE56/04/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 22
lineCount: 2
- id: "DE56/05"
name: "Trailing tabs in double quoted"
localPath: yaml-test-suite/DE56/05/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 24
lineCount: 2
- id: "DFF7"
name: "Spec Example 7.16. Flow Mapping Entries"
localPath: yaml-test-suite/DFF7/in.yaml
@ -856,24 +676,6 @@ cases:
current: accepted
byteCount: 16
lineCount: 2
- id: "DK95/02"
name: "Tabs that look like indentation"
localPath: yaml-test-suite/DK95/02/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 18
lineCount: 2
- id: "DK95/08"
name: "Tabs that look like indentation"
localPath: yaml-test-suite/DK95/08/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 25
lineCount: 2
- id: "E76Z"
name: "Aliases in Implicit Block Mapping"
localPath: yaml-test-suite/E76Z/in.yaml
@ -928,15 +730,6 @@ cases:
current: parse-error
byteCount: 24
lineCount: 4
- id: "G4RS"
name: "Spec Example 2.17. Quoted Scalars"
localPath: yaml-test-suite/G4RS/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 177
lineCount: 7
- id: "G5U8"
name: "Plain dashes in flow sequence"
localPath: yaml-test-suite/G5U8/in.yaml
@ -1162,15 +955,6 @@ cases:
current: parse-error
byteCount: 15
lineCount: 2
- id: "NAT4"
name: "Various empty or newline only quoted strings"
localPath: yaml-test-suite/NAT4/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 86
lineCount: 23
- id: "NB6Z"
name: "Multiline plain value with tabs on empty lines"
localPath: yaml-test-suite/NB6Z/in.yaml
@ -1207,15 +991,6 @@ cases:
current: parse-error
byteCount: 120
lineCount: 13
- id: "NP9H"
name: "Spec Example 7.5. Double Quoted Line Breaks"
localPath: yaml-test-suite/NP9H/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 63
lineCount: 5
- id: "P2EQ"
name: "Invalid sequene item on same line as previous item"
localPath: yaml-test-suite/P2EQ/in.yaml
@ -1225,15 +1000,6 @@ cases:
current: accepted
byteCount: 24
lineCount: 2
- id: "PRH3"
name: "Spec Example 7.9. Single Quoted Lines"
localPath: yaml-test-suite/PRH3/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 50
lineCount: 4
- id: "PW8X"
name: "Anchors on Empty Scalars"
localPath: yaml-test-suite/PW8X/in.yaml
@ -1243,15 +1009,6 @@ cases:
current: parse-error
byteCount: 65
lineCount: 12
- id: "Q8AD"
name: "Spec Example 7.5. Double Quoted Line Breaks [1.3]"
localPath: yaml-test-suite/Q8AD/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 66
lineCount: 6
- id: "Q9WF"
name: "Spec Example 6.12. Separation Spaces"
localPath: yaml-test-suite/Q9WF/in.yaml
@ -1396,15 +1153,6 @@ cases:
current: parse-error
byteCount: 47
lineCount: 9
- id: "T4YY"
name: "Spec Example 7.9. Single Quoted Lines [1.3]"
localPath: yaml-test-suite/T4YY/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 54
lineCount: 5
- id: "T5N4"
name: "Spec Example 8.7. Literal Scalar [1.3]"
localPath: yaml-test-suite/T5N4/in.yaml
@ -1414,15 +1162,6 @@ cases:
current: value-mismatch
byteCount: 24
lineCount: 5
- id: "TL85"
name: "Spec Example 6.8. Flow Folding"
localPath: yaml-test-suite/TL85/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 28
lineCount: 7
- id: "UGM3"
name: "Spec Example 2.27. Invoice"
localPath: yaml-test-suite/UGM3/in.yaml
@ -1522,15 +1261,6 @@ cases:
current: parse-error
byteCount: 21
lineCount: 3
- id: "XV9V"
name: "Spec Example 6.5. Empty Lines [1.3]"
localPath: yaml-test-suite/XV9V/in.yaml
category: json-mismatch
expected: parse-ok
oracle: json
current: value-mismatch
byteCount: 79
lineCount: 8
- id: "XW4D"
name: "Various Trailing Comments"
localPath: yaml-test-suite/XW4D/in.yaml

1
Tests/Fixtures/yaml-test-suite/3RLN/00/in.json
View File

@ -1 +0,0 @@
"1 leading \ttab"

2
Tests/Fixtures/yaml-test-suite/3RLN/00/in.yaml
View File

@ -1,2 +0,0 @@
"1 leading
\ttab"

1
Tests/Fixtures/yaml-test-suite/3RLN/01/in.json
View File

@ -1 +0,0 @@
"2 leading \ttab"

2
Tests/Fixtures/yaml-test-suite/3RLN/01/in.yaml
View File

@ -1,2 +0,0 @@
"2 leading
\ tab"

1
Tests/Fixtures/yaml-test-suite/3RLN/02/in.json
View File

@ -1 +0,0 @@
"3 leading tab"

2
Tests/Fixtures/yaml-test-suite/3RLN/02/in.yaml
View File

@ -1,2 +0,0 @@
"3 leading
tab"

1
Tests/Fixtures/yaml-test-suite/3RLN/03/in.json
View File

@ -1 +0,0 @@
"4 leading \t tab"

2
Tests/Fixtures/yaml-test-suite/3RLN/03/in.yaml
View File

@ -1,2 +0,0 @@
"4 leading
\t tab"

1
Tests/Fixtures/yaml-test-suite/3RLN/04/in.json
View File

@ -1 +0,0 @@
"5 leading \t tab"

2
Tests/Fixtures/yaml-test-suite/3RLN/04/in.yaml
View File

@ -1,2 +0,0 @@
"5 leading
\ tab"

1
Tests/Fixtures/yaml-test-suite/3RLN/05/in.json
View File

@ -1 +0,0 @@
"6 leading tab"

2
Tests/Fixtures/yaml-test-suite/3RLN/05/in.yaml
View File

@ -1,2 +0,0 @@
"6 leading
tab"

5
Tests/Fixtures/yaml-test-suite/4ZYM/in.json
View File

@ -1,5 +0,0 @@
{
"plain": "text lines",
"quoted": "text lines",
"block": "text\n \tlines\n"
}

7
Tests/Fixtures/yaml-test-suite/4ZYM/in.yaml
View File

@ -1,7 +0,0 @@
plain: text
lines
quoted: "text
lines"
block: |
text
lines

5
Tests/Fixtures/yaml-test-suite/565N/in.json
View File

@ -1,5 +0,0 @@
{
"canonical": "R0lGODlhDAAMAIQAAP//9/X17unp5WZmZgAAAOfn515eXvPz7Y6OjuDg4J+fn5OTk6enp56enmlpaWNjY6Ojo4SEhP/++f/++f/++f/++f/++f/++f/++f/++f/++f/++f/++f/++f/++f/++SH+Dk1hZGUgd2l0aCBHSU1QACwAAAAADAAMAAAFLCAgjoEwnuNAFOhpEMTRiggcz4BNJHrv/zCFcLiwMWYNG84BwwEeECcgggoBADs=",
"generic": "R0lGODlhDAAMAIQAAP//9/X17unp5WZmZgAAAOfn515eXvPz7Y6OjuDg4J+fn5\nOTk6enp56enmlpaWNjY6Ojo4SEhP/++f/++f/++f/++f/++f/++f/++f/++f/+\n+f/++f/++f/++f/++f/++SH+Dk1hZGUgd2l0aCBHSU1QACwAAAAADAAMAAAFLC\nAgjoEwnuNAFOhpEMTRiggcz4BNJHrv/zCFcLiwMWYNG84BwwEeECcgggoBADs=\n",
"description": "The binary value above is a tiny arrow encoded as a gif image."
}

12
Tests/Fixtures/yaml-test-suite/565N/in.yaml
View File

@ -1,12 +0,0 @@
canonical: !!binary "\
R0lGODlhDAAMAIQAAP//9/X17unp5WZmZgAAAOfn515eXvPz7Y6OjuDg4J+fn5\
OTk6enp56enmlpaWNjY6Ojo4SEhP/++f/++f/++f/++f/++f/++f/++f/++f/+\
+f/++f/++f/++f/++f/++SH+Dk1hZGUgd2l0aCBHSU1QACwAAAAADAAMAAAFLC\
AgjoEwnuNAFOhpEMTRiggcz4BNJHrv/zCFcLiwMWYNG84BwwEeECcgggoBADs="
generic: !!binary |
R0lGODlhDAAMAIQAAP//9/X17unp5WZmZgAAAOfn515eXvPz7Y6OjuDg4J+fn5
OTk6enp56enmlpaWNjY6Ojo4SEhP/++f/++f/++f/++f/++f/++f/++f/++f/+
+f/++f/++f/++f/++f/++SH+Dk1hZGUgd2l0aCBHSU1QACwAAAAADAAMAAAFLC
AgjoEwnuNAFOhpEMTRiggcz4BNJHrv/zCFcLiwMWYNG84BwwEeECcgggoBADs=
description:
The binary value above is a tiny arrow encoded as a gif image.

4
Tests/Fixtures/yaml-test-suite/5GBF/in.json
View File

@ -1,4 +0,0 @@
{
"Folding": "Empty line\nas a line feed",
"Chomping": "Clipped empty lines\n"
}

8
Tests/Fixtures/yaml-test-suite/5GBF/in.yaml
View File

@ -1,8 +0,0 @@
Folding:
"Empty line
as a line feed"
Chomping: |
Clipped empty lines

1
Tests/Fixtures/yaml-test-suite/6WPF/in.json
View File

@ -1 +0,0 @@
" foo\nbar\nbaz "

8
Tests/Fixtures/yaml-test-suite/6WPF/in.yaml
View File

@ -1,8 +0,0 @@
---
"
foo
bar
baz
"

1
Tests/Fixtures/yaml-test-suite/7A4E/in.json
View File

@ -1 +0,0 @@
" 1st non-empty\n2nd non-empty 3rd non-empty "

4
Tests/Fixtures/yaml-test-suite/7A4E/in.yaml
View File

@ -1,4 +0,0 @@
" 1st non-empty
2nd non-empty
3rd non-empty "

11
Tests/Fixtures/yaml-test-suite/8UDB/in.json
View File

@ -1,11 +0,0 @@
[
"double quoted",
"single quoted",
"plain text",
[
"nested"
],
{
"single": "pair"
}
]

8
Tests/Fixtures/yaml-test-suite/8UDB/in.yaml
View File

@ -1,8 +0,0 @@
[
"double
quoted", 'single
quoted',
plain
text, [ nested ],
single: pair,
]

8
Tests/Fixtures/yaml-test-suite/9SA2/in.json
View File

@ -1,8 +0,0 @@
[
{
"single line": "value"
},
{
"multi line": "value"
}
]

4
Tests/Fixtures/yaml-test-suite/9SA2/in.yaml
View File

@ -1,4 +0,0 @@
---
- { "single line": value}
- { "multi
line": value}

1
Tests/Fixtures/yaml-test-suite/9TFX/in.json
View File

@ -1 +0,0 @@
" 1st non-empty\n2nd non-empty 3rd non-empty "

5
Tests/Fixtures/yaml-test-suite/9TFX/in.yaml
View File

@ -1,5 +0,0 @@
---
" 1st non-empty
2nd non-empty
3rd non-empty "

1
Tests/Fixtures/yaml-test-suite/DE56/00/in.json
View File

@ -1 +0,0 @@
"1 trailing\t tab"

2
Tests/Fixtures/yaml-test-suite/DE56/00/in.yaml
View File

@ -1,2 +0,0 @@
"1 trailing\t
tab"

1
Tests/Fixtures/yaml-test-suite/DE56/01/in.json
View File

@ -1 +0,0 @@
"2 trailing\t tab"

2
Tests/Fixtures/yaml-test-suite/DE56/01/in.yaml
View File

@ -1,2 +0,0 @@
"2 trailing\t
tab"

1
Tests/Fixtures/yaml-test-suite/DE56/02/in.json
View File

@ -1 +0,0 @@
"3 trailing\t tab"

2
Tests/Fixtures/yaml-test-suite/DE56/02/in.yaml
View File

@ -1,2 +0,0 @@
"3 trailing\
tab"

1
Tests/Fixtures/yaml-test-suite/DE56/03/in.json
View File

@ -1 +0,0 @@
"4 trailing\t tab"

2
Tests/Fixtures/yaml-test-suite/DE56/03/in.yaml
View File

@ -1,2 +0,0 @@
"4 trailing\
tab"

1
Tests/Fixtures/yaml-test-suite/DE56/04/in.json
View File

@ -1 +0,0 @@
"5 trailing tab"

2
Tests/Fixtures/yaml-test-suite/DE56/04/in.yaml
View File

@ -1,2 +0,0 @@
"5 trailing
tab"

1
Tests/Fixtures/yaml-test-suite/DE56/05/in.json
View File

@ -1 +0,0 @@
"6 trailing tab"

2
Tests/Fixtures/yaml-test-suite/DE56/05/in.yaml
View File

@ -1,2 +0,0 @@
"6 trailing
tab"

3
Tests/Fixtures/yaml-test-suite/DK95/02/in.json
View File

@ -1,3 +0,0 @@
{
"foo" : "bar baz"
}

2
Tests/Fixtures/yaml-test-suite/DK95/02/in.yaml
View File

@ -1,2 +0,0 @@
foo: "bar
baz"

3
Tests/Fixtures/yaml-test-suite/DK95/08/in.json
View File

@ -1,3 +0,0 @@
{
"foo" : "bar baz \t \t "
}

2
Tests/Fixtures/yaml-test-suite/DK95/08/in.yaml
View File

@ -1,2 +0,0 @@
foo: "bar
baz "

8
Tests/Fixtures/yaml-test-suite/G4RS/in.json
View File

@ -1,8 +0,0 @@
{
"unicode": "Sosa did fine.☺",
"control": "\b1998\t1999\t2000\n",
"hex esc": "\r\n is \r\n",
"single": "\"Howdy!\" he cried.",
"quoted": " # Not a 'comment'.",
"tie-fighter": "|\\-*-/|"
}

7
Tests/Fixtures/yaml-test-suite/G4RS/in.yaml
View File

@ -1,7 +0,0 @@
unicode: "Sosa did fine.\u263A"
control: "\b1998\t1999\t2000\n"
hex esc: "\x0d\x0a is \r\n"
single: '"Howdy!" he cried.'
quoted: ' # Not a ''comment''.'
tie-fighter: '|\-*-/|'

10
Tests/Fixtures/yaml-test-suite/NAT4/in.json
View File

@ -1,10 +0,0 @@
{
"a": " ",
"b": " ",
"c": " ",
"d": " ",
"e": "\n",
"f": "\n",
"g": "\n\n",
"h": "\n\n"
}

23
Tests/Fixtures/yaml-test-suite/NAT4/in.yaml
View File

@ -1,23 +0,0 @@
---
a: '
'
b: '
'
c: "
"
d: "
"
e: '
'
f: "
"
g: '
'
h: "
"

1
Tests/Fixtures/yaml-test-suite/NP9H/in.json
View File

@ -1 +0,0 @@
"folded to a space,\nto a line feed, or \t \tnon-content"

5
Tests/Fixtures/yaml-test-suite/NP9H/in.yaml
View File

@ -1,5 +0,0 @@
"folded
to a space,
to a line feed, or \
\ non-content"

1
Tests/Fixtures/yaml-test-suite/PRH3/in.json
View File

@ -1 +0,0 @@
" 1st non-empty\n2nd non-empty 3rd non-empty "

4
Tests/Fixtures/yaml-test-suite/PRH3/in.yaml
View File

@ -1,4 +0,0 @@
' 1st non-empty
2nd non-empty
3rd non-empty '

1
Tests/Fixtures/yaml-test-suite/Q8AD/in.json
View File

@ -1 +0,0 @@
"folded to a space,\nto a line feed, or \t \tnon-content"

6
Tests/Fixtures/yaml-test-suite/Q8AD/in.yaml
View File

@ -1,6 +0,0 @@
---
"folded
to a space,
to a line feed, or \
\ non-content"

1
Tests/Fixtures/yaml-test-suite/T4YY/in.json
View File

@ -1 +0,0 @@
" 1st non-empty\n2nd non-empty 3rd non-empty "

5
Tests/Fixtures/yaml-test-suite/T4YY/in.yaml
View File

@ -1,5 +0,0 @@
---
' 1st non-empty
2nd non-empty
3rd non-empty '

1
Tests/Fixtures/yaml-test-suite/TL85/in.json
View File

@ -1 +0,0 @@
" foo\nbar\nbaz "

7
Tests/Fixtures/yaml-test-suite/TL85/in.yaml
View File

@ -1,7 +0,0 @@
"
foo
bar
baz
"

4
Tests/Fixtures/yaml-test-suite/XV9V/in.json
View File

@ -1,4 +0,0 @@
{
"Folding": "Empty line\nas a line feed",
"Chomping": "Clipped empty lines\n"
}

8
Tests/Fixtures/yaml-test-suite/XV9V/in.yaml
View File

@ -1,8 +0,0 @@
Folding:
"Empty line
as a line feed"
Chomping: |
Clipped empty lines

4
Tests/YAMLTestSuiteGapsTests.swift
View File

@ -18,9 +18,9 @@ struct YAMLTestSuiteGapsTests {
let manifest = try YAMLTestSuiteManifest.load()
#expect(manifest.version == 1)
#expect(manifest.cases.count == 182)
#expect(manifest.cases.count == 152)
#expect(manifest.cases.count(where: { $0.category == "false-reject" }) == 84)
#expect(manifest.cases.count(where: { $0.category == "json-mismatch" }) == 46)
#expect(manifest.cases.count(where: { $0.category == "json-mismatch" }) == 16)
#expect(manifest.cases.count(where: { $0.category == "accepted-invalid" }) == 52)
#expect(Set(manifest.cases.map(\.id)).count == manifest.cases.count)

31
docs/YAML_TEST_SUITE_CONFORMANCE.md
View File

@ -3,7 +3,7 @@
This document tracks PureYAML against the official
[YAML test suite](https://github.com/yaml/yaml-test-suite) and lays out the
ordered plan for closing the gaps. It is a living document: bucket 1 (block
scalars) has landed; buckets 2-6 remain.
scalars) and bucket 2 (quoted-scalar folding) have landed; buckets 3-6 remain.
## How conformance is measured
@ -19,7 +19,8 @@ its own oracle:
### Result at the pinned commit
402 cases (including numbered sub-cases). First measurement was 196 pass / 206
fail; after bucket 1 (block scalars) it is now **220 pass, 182 fail.** The 182
fail; after bucket 1 (block scalars, +24) and bucket 2 (quoted-scalar folding,
+30) it is now **250 pass, 152 fail.** The 152
remaining failing cases are vendored under `Tests/Fixtures/yaml-test-suite/` and
recorded in `Tests/Fixtures/yaml-test-suite-gaps.yaml`. Run the opt-in sweep with:
@ -38,7 +39,7 @@ updated, the deliberate forcing function for keeping this document honest.
| Category | Count | Meaning |
|---|---|---|
| `false-reject` | 84 | Well-formed YAML that PureYAML wrongly rejects at parse time. |
| `json-mismatch` | 46 | Parses, but produces a value that disagrees with the expected JSON. |
| `json-mismatch` | 16 | Parses, but produces a value that disagrees with the expected JSON. |
| `accepted-invalid` | 52 | Malformed YAML that PureYAML wrongly accepts. |
## Root-cause buckets and fix order
@ -65,14 +66,26 @@ Flipped 24 cases to passing: `4QFQ`, `5LLU`, `6VJK`, `7T8X`, `96NN/00`, `96NN/01
(`T5N4` looks like a block-scalar case but is blocked by document-start handling,
bucket 5.)
### 2. Quoted-scalar line breaks and empty-line folding (subset of `json-mismatch`)
### 2. Quoted-scalar line breaks and empty-line folding (DONE)
Double- and single-quoted multi-line scalars fold line breaks and collapse blank
lines incorrectly. Examples: `NP9H`, `Q8AD`, `PRH3`, `T4YY`, `TL85`, `NAT4`.
Single (`'`) and double (`"`) quoted multi-line scalars now apply YAML 1.2 flow
folding (§6.5, §7.3.1, §7.3.2): trailing whitespace before a break is stripped,
leading whitespace on a continuation line is stripped, a single break between
non-empty lines folds to one space, and N consecutive breaks yield N-1 literal
line feeds (the first and last lines keep their own leading and trailing spaces as
content). A double quoted backslash at end of line suppresses the break entirely.
The double quoted escape set is completed with `\xXX` and `\UXXXXXXXX`, plus the
previously missing `\0 \a \v \e`, literal `\<space>`, and named breaks
`\N \_ \L \P` (the existing `\uXXXX` surrogate-pair handling is kept). The folding
state machine mirrors `scanner.c` `yaml_parser_scan_flow_scalar`; implemented in
`Sources/Parsing/Scalar.swift` (`foldFlowScalar` and the escape decoder).
- **Reference:** `scanner.c` `yaml_parser_scan_flow_scalar` (line-join, leading/
trailing whitespace, escape handling).
- **Why second:** same scanner area as bucket 1, shares the line-folding helper.
Flipped 30 cases to passing: `3RLN/00`, `3RLN/01`, `3RLN/02`, `3RLN/03`,
`3RLN/04`, `3RLN/05`, `4ZYM`, `565N`, `5GBF`, `6WPF`, `7A4E`, `8UDB`, `9SA2`,
`9TFX`, `DE56/00`, `DE56/01`, `DE56/02`, `DE56/03`, `DE56/04`, `DE56/05`,
`DK95/02`, `DK95/08`, `G4RS`, `NAT4`, `NP9H`, `PRH3`, `Q8AD`, `T4YY`, `TL85`,
`XV9V`. (`DK95/01` looks similar but is blocked by tab-as-indentation handling,
bucket 6.)
### 3. Empty nodes and empty keys (≈15 cases, `false-reject`)