fix(parsing): reject a tab used to indent flow content

A tab is never indentation, so flow-collection content reached through a
tab below the continuation indent ('- [' / '\tfoo') is malformed even
though the tab inflates the column. A tab at or past the indent is
separation, and a tab-only blank line stays blank, so both remain valid.

Track a tab in a flow continuation's leading whitespace below the
parent + 1 indent and reject content (but not a closing bracket) reached
through it. Surpasses libyaml on Y79Y/003; whole-suite conformance
400/402, canonical events 308/308.

CHANGELOG.md

@@ -22,6 +22,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Reject a tab used to indent flow-collection content. A tab is never indentation,
+  so flow content reached through a tab below the continuation indent (`- [`\n
+  `\tfoo`) is malformed, even though the tab inflates the column. A tab at or past
+  the indent stays separation, and a tab-only blank line stays blank. Surpasses
+  libyaml on Y79Y/003.
 - Reject a block scalar whose leading empty line is more indented than its first
   content line. libyaml auto-detects the deeper indentation and silently drops the
   content; the auto-detection now errors when the content line is less indented

Sources/Parsing/LibYAML.Scanner.Guards.swift

@@ -14,7 +14,9 @@ extension PureYAML.Parsing.LibYAML.Scanner {
         if next == "]" || next == "}" || next == "," {
             return
         }
-        if reader.column <= indentEngine.indent {
+        // A tab in the leading whitespace is never valid indentation, even where
+        // spaces of the same width would be (the tab inflates the column).
+        if reader.column <= indentEngine.indent || flowLeadingTab {
             throw ScannerError(
                 context: "while parsing a flow node",
                 problem: "found insufficiently indented line in a flow collection",

Sources/Parsing/LibYAML.Scanner.SkipToToken.swift

@@ -9,14 +9,15 @@ extension PureYAML.Parsing.LibYAML.Scanner {
     /// indicator or implicit key (where the tab would be indentation) can reject.
     mutating func scanToNextToken() throws {
         crossedLineBreak = false
+        flowLeadingTab = false
         while true {
             tabbedSeparation = false
             if reader.column == 0, reader.peek() == "\u{FEFF}" {
                 reader.skip()
             }
             while reader.peek() == " " || Classify.isTab(reader.peek()) {
-                if Classify.isTab(reader.peek()), flowLevel == 0, simpleKeyAllowed {
-                    tabbedSeparation = true
+                if Classify.isTab(reader.peek()) {
+                    noteLeadingTab()
                 }
                 reader.skip()
             }
@@ -44,6 +45,19 @@ extension PureYAML.Parsing.LibYAML.Scanner {
         }
     }
 
+    /// Record what a tab in the leading whitespace implies. In block context with a
+    /// simple key allowed it is potential indentation before an indicator. In flow
+    /// context, below the continuation indent on a non-blank line it is indentation
+    /// abuse (a tab is never indentation); at or above the indent it is separation.
+    private mutating func noteLeadingTab() {
+        if flowLevel == 0, simpleKeyAllowed {
+            tabbedSeparation = true
+        }
+        if flowLevel > 0, crossedLineBreak, reader.column < indentEngine.indent + 1, !blankLineAhead() {
+            flowLeadingTab = true
+        }
+    }
+
     /// Consume a comment to the end of the line. A comment must be preceded by
     /// whitespace or start the line; a `#` adjacent to the previous token (e.g.
     /// `"x"#`, `]#`, `,#`) is not a comment. The plain-scalar scanner consumes its

Sources/Parsing/LibYAML.Scanner.swift

@@ -22,6 +22,10 @@ extension PureYAML.Parsing.LibYAML {
         /// flow context a continuation line must be indented past the parent block,
         /// so a token on a new line at or below that indent is an error.
         var crossedLineBreak = false
+        /// Whether a tab appeared in the leading whitespace of a flow continuation
+        /// line. A tab is never indentation, so flow content reached through one is
+        /// an error even though the tab inflates the column.
+        var flowLeadingTab = false
         /// Whether the most recent `:` value indicator sat on the same line as its
         /// key (an inline implicit-key value). Such a value may not open a block
         /// sequence on that line (`key: - a`), whereas an explicit key whose `:` is

Tests/Fixtures/yaml-test-suite-manifest.yaml

@@ -10,8 +10,8 @@ source:
 # oracle:   json if in.json present, else none
 # status:   passing (PureYAML conformant today) | gap (not yet conformant)
 total: 402
-passing: 399
-gap: 3
+passing: 400
+gap: 2
 cases:
   - id: "229Q"
     name: "Spec Example 2.4. Sequence of Mappings"
@@ -3074,7 +3074,7 @@ cases:
     localPath: yaml-test-suite/Y79Y/003/in.yaml
     expected: reject
     oracle: none
-    status: gap
+    status: passing
     byteCount: 18
     lineCount: 4
   - id: "Y79Y/004"