fix(parsing): skip tabs as mid-line separation white space
skipSeparation consumed only spaces between tokens, so a tab after an indicator leaked into the following node: an explicit key written `?\tkey` became the key `\tkey` rather than `key`. A tab is separation white space in YAML, forbidden only as indentation, which the line-start scan already handles before this point. Skip tabs there too. This closes the post-indicator tab leak left open when the indicator boundary itself was changed, and handles a run of tabs between a `:` and its value. The test-suite rates are unchanged, since no comparable case isolates this, but the parser now matches the spec on tab separation. 435 tests pass.
This commit is contained in:
parent
738408d458
commit
67d72c8b15
@ -84,6 +84,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- Mid-line separation white space between tokens now includes tabs, not only spaces, completing the
|
||||
tab-as-separation handling. An indicator followed by a tab no longer leaks the tab into the next
|
||||
node: `?\tkey` is the key `key`, not `\tkey`, and a run of tabs between a `:` and its value
|
||||
(`a:\t\tb`) is separation. This closes the post-indicator tab leak noted when the indicator boundary
|
||||
itself was fixed. Tabs as indentation are still handled separately by the line-start scan, and the
|
||||
yaml-test-suite rates are unchanged (no comparable case isolates this), but the parser now matches
|
||||
the spec on tab separation.
|
||||
|
||||
- A tab now follows the `:` mapping-value indicator the way a space does, in both block and flow
|
||||
context, since a tab is separation white space in YAML. `foo:\tbar` is the mapping `{foo: bar}`
|
||||
rather than the plain scalar `foo:\tbar`. A `:` with no following white space at all (`foo:bar`)
|
||||
|
||||
@ -341,7 +341,9 @@ extension PureYAML.Parsing.Scanner {
|
||||
}
|
||||
|
||||
func skipSeparation(_ state: inout State) {
|
||||
while state.reader.peek() == " " {
|
||||
// Mid-line separation may be spaces or tabs: a tab is separation white space in YAML and is
|
||||
// forbidden only as indentation, which the line-start indentation scan handles before this.
|
||||
while state.reader.peek() == " " || state.reader.peek() == "\t" {
|
||||
state.reader.advance()
|
||||
}
|
||||
}
|
||||
|
||||
@ -16,6 +16,20 @@ struct IndicatorTabBoundaryTests {
|
||||
#expect(try PureYAML.parse("-\titem\n") == .sequence([.string("item")]))
|
||||
}
|
||||
|
||||
@Test("An explicit key indicator followed by a tab starts a mapping key with the tab stripped")
|
||||
func test_explicitKeyTab() throws {
|
||||
#expect(try PureYAML.parse("?\tkey\n: value\n") == .mapping(.init([
|
||||
.init(key: "key", value: .string("value")),
|
||||
])))
|
||||
}
|
||||
|
||||
@Test("A run of tabs between tokens is separation, not content")
|
||||
func test_multipleTabSeparation() throws {
|
||||
#expect(try PureYAML.parse("a:\t\tb\n") == .mapping(.init([
|
||||
.init(key: "a", value: .string("b")),
|
||||
])))
|
||||
}
|
||||
|
||||
@Test("A space after an indicator is unchanged")
|
||||
func test_spaceStillWorks() throws {
|
||||
#expect(try PureYAML.parse("--- scalar\n") == .string("scalar"))
|
||||
|
||||
Loading…
Reference in New Issue
Block a user