fix(parsing): skip tabs as mid-line separation white space

skipSeparation consumed only spaces between tokens, so a tab after an
indicator leaked into the following node: an explicit key written `?\tkey`
became the key `\tkey` rather than `key`. A tab is separation white space in
YAML, forbidden only as indentation, which the line-start scan already handles
before this point.

Skip tabs there too. This closes the post-indicator tab leak left open when
the indicator boundary itself was changed, and handles a run of tabs between a
`:` and its value. The test-suite rates are unchanged, since no comparable
case isolates this, but the parser now matches the spec on tab separation.

435 tests pass.
This commit is contained in:
Mihaela M 2026-06-28 20:35:04 +02:00
parent 738408d458
commit 67d72c8b15
3 changed files with 25 additions and 1 deletions

View File

@ -84,6 +84,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Mid-line separation white space between tokens now includes tabs, not only spaces, completing the
tab-as-separation handling. An indicator followed by a tab no longer leaks the tab into the next
node: `?\tkey` is the key `key`, not `\tkey`, and a run of tabs between a `:` and its value
(`a:\t\tb`) is separation. This closes the post-indicator tab leak noted when the indicator boundary
itself was fixed. Tabs as indentation are still handled separately by the line-start scan, and the
yaml-test-suite rates are unchanged (no comparable case isolates this), but the parser now matches
the spec on tab separation.
- A tab now follows the `:` mapping-value indicator the way a space does, in both block and flow
context, since a tab is separation white space in YAML. `foo:\tbar` is the mapping `{foo: bar}`
rather than the plain scalar `foo:\tbar`. A `:` with no following white space at all (`foo:bar`)

View File

@ -341,7 +341,9 @@ extension PureYAML.Parsing.Scanner {
}
func skipSeparation(_ state: inout State) {
while state.reader.peek() == " " {
// Mid-line separation may be spaces or tabs: a tab is separation white space in YAML and is
// forbidden only as indentation, which the line-start indentation scan handles before this.
while state.reader.peek() == " " || state.reader.peek() == "\t" {
state.reader.advance()
}
}

View File

@ -16,6 +16,20 @@ struct IndicatorTabBoundaryTests {
#expect(try PureYAML.parse("-\titem\n") == .sequence([.string("item")]))
}
@Test("An explicit key indicator followed by a tab starts a mapping key with the tab stripped")
func test_explicitKeyTab() throws {
#expect(try PureYAML.parse("?\tkey\n: value\n") == .mapping(.init([
.init(key: "key", value: .string("value")),
])))
}
@Test("A run of tabs between tokens is separation, not content")
func test_multipleTabSeparation() throws {
#expect(try PureYAML.parse("a:\t\tb\n") == .mapping(.init([
.init(key: "a", value: .string("b")),
])))
}
@Test("A space after an indicator is unchanged")
func test_spaceStillWorks() throws {
#expect(try PureYAML.parse("--- scalar\n") == .string("scalar"))