feat(parsing): handle directives and document-end boundaries

Three root causes: (1) a '%' begins a directive only in directive position (before document content, outside flow); everywhere else it is plain content, fixing misfires inside plain scalars, flow mappings, and block scalars. (2) %YAML accepts any 1.x (a higher minor is treated as 1.2 without error), rejects major != 1 and duplicate %YAML, and requires a following '---'; reserved and %TAG directives are accepted and ignored. A directive not followed by a document start is rejected with the new directiveWithoutDocument error. (3) An explicit '...' end marker closes the current document so following content begins a fresh document in parseStream; single-document parse still rejects multi-document input at compose time.

Tests/UnsupportedYAMLGapsFixtures.swift, StreamParsingTests, ScannerTests, and ParsingErrorTests were updated in lockstep to assert the now-correct behavior (percent-after-content is plain text; '...' then content is two documents; %YAML 1.3 is valid while 2.0 and duplicate %YAML are rejected); no test was weakened and multi-document rejection for single-document parse is still covered.

Flips 15 official YAML test suite cases (295/402); SF5V, 9MMA, B63P, H7TQ now correctly rejected. 6LVF and W4TN are reclassified to json-mismatch (blocked by other scalar gaps). Updates the manifest, fixtures, counts, and roadmap.

CHANGELOG.md

@@ -9,6 +9,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Added YAML directive handling and completed document-boundary support, fixing
+  three root causes. First, a `%` now begins a directive only in directive
+  position (before any document content and before the `---` that starts a
+  document, including immediately after a `...` end marker); everywhere else, and
+  in any flow context, `%` is ordinary plain text rather than a misfiring
+  directive. Second, `%YAML` is parsed leniently per the spec: a `1.x` version is
+  accepted (a higher minor is treated as 1.2 with a warning rather than an
+  error), while an unsupported major version, a duplicate `%YAML` in one
+  document, or extra non-comment fields are rejected; reserved and unknown
+  directives (and `%TAG` handle definitions) are accepted and ignored, and a
+  directive must be followed by a `---` document start. Third, a `...` end marker
+  now closes the current document so that following content begins a fresh
+  document in the multi-document stream parser (`parseStream`), while the
+  single-document `parse` still rejects multi-document input. This makes 11
+  official YAML test suite cases parse with matching values (2LFX, 5TYM, 6WLZ,
+  6ZKB, 7Z25, 9DXL, 9WXW, BEC7, MUS6/03, MUS6/05, MUS6/06), correctly rejects
+  four previously accepted-invalid cases (9MMA, B63P, H7TQ, SF5V), and unblocks
+  6LVF and W4TN from parse errors to value comparisons that now depend on
+  separate scalar gaps (`---`-line content and literal block chomping).
 - Rewrote literal (`|`) and folded (`>`) block scalar content handling to follow
   YAML 1.2 §8.1 semantics, fixing four root causes: interior blank lines are no
   longer dropped from literal scalars; folded scalars now apply true §8.1.3 line

Sources/Parsing/ParseError.swift

@@ -11,6 +11,7 @@ public extension PureYAML.Parsing {
         case expectedNode(line: Int, column: Int)
         case expectedScalarKey(line: Int, column: Int)
         case incompatibleYAMLDirective(line: Int)
+        case directiveWithoutDocument(line: Int)
         case invalidTaggedScalar(tag: String, value: String, line: Int, column: Int)
         case invalidMergeValue(line: Int, column: Int)
         case unterminatedTag(line: Int)
@@ -44,6 +45,8 @@ public extension PureYAML.Parsing {
                 "expected a scalar mapping key at line \(line), column \(column)"
             case let .incompatibleYAMLDirective(line):
                 "incompatible YAML directive at line \(line)"
+            case let .directiveWithoutDocument(line):
+                "directive is not followed by a document start at line \(line)"
             case let .invalidTaggedScalar(tag, value, line, column):
                 "invalid scalar '\(value)' for tag '\(tag)' at line \(line), column \(column)"
             case let .invalidMergeValue(line, column):
@@ -89,6 +92,8 @@ public extension PureYAML.Parsing {
                 "expectedScalarKey"
             case .incompatibleYAMLDirective:
                 "incompatibleYAMLDirective"
+            case .directiveWithoutDocument:
+                "directiveWithoutDocument"
             case .invalidTaggedScalar:
                 "invalidTaggedScalar"
             case .invalidMergeValue:
@@ -123,6 +128,7 @@ public extension PureYAML.Parsing {
                  let .expectedAnchorName(line),
                  let .expectedAliasName(line),
                  let .incompatibleYAMLDirective(line),
+                 let .directiveWithoutDocument(line),
                  let .unterminatedTag(line),
                  let .unterminatedQuotedString(line),
                  let .unsupportedDirective(_, line),
@@ -150,6 +156,7 @@ public extension PureYAML.Parsing {
                  .expectedAnchorName,
                  .expectedAliasName,
                  .incompatibleYAMLDirective,
+                 .directiveWithoutDocument,
                  .unterminatedTag,
                  .unterminatedQuotedString,
                  .unsupportedDirective,

Sources/Parsing/Scanner.Directives.swift

@@ -1,9 +1,18 @@
 extension PureYAML.Parsing.Scanner {
     func scanDirectiveOrDocumentMarker(_ state: inout State) throws -> Bool {
-        guard state.indentation.last == 0 else {
+        guard state.indentation.last == 0, state.flowDepth == 0 else {
             return false
         }
         if state.reader.peek() == "%" {
+            // `%` introduces a directive only in directive position: before any
+            // document content and before the `---` that starts a document. After
+            // content, after `---`, or anywhere inside a node, `%` is plain text,
+            // so defer to ordinary token scanning by returning false. This mirrors
+            // the C scanner, which only enters `yaml_parser_scan_directive` from
+            // the start-of-line directive context, never inside a node or scalar.
+            guard !state.hasDocumentContent, !state.hasDocumentStartMarker else {
+                return false
+            }
             try scanDirective(&state)
             return true
         }
@@ -16,16 +25,16 @@ extension PureYAML.Parsing.Scanner {
             return true
         }
         if startsWith("...", state.reader), isIndicatorBoundary(state.reader.peek(offset: 3)) {
-            guard !state.isDocumentClosed else {
-                throw PureYAML.Parsing.ParseError.unsupportedMultiDocumentStream(line: state.reader.mark.line)
+            // A `...` end marker terminates the directive section it follows just
+            // as a `---` would, except that no document was started, so a pending
+            // directive here was never attached to a document and is rejected.
+            if state.hasPendingDirective {
+                throw PureYAML.Parsing.ParseError.directiveWithoutDocument(line: state.reader.mark.line)
             }
             scanDocumentMarker(.documentEnd, state: &state)
             resetForDocumentEnd(&state)
             return true
         }
-        guard !state.isDocumentClosed else {
-            throw PureYAML.Parsing.ParseError.unsupportedMultiDocumentStream(line: state.reader.mark.line)
-        }
         return false
     }
 
@@ -45,12 +54,16 @@ extension PureYAML.Parsing.Scanner {
         state.hasDocumentContent = false
         state.hasDocumentStartMarker = true
         state.isDocumentClosed = false
+        state.sawYAMLDirective = false
+        state.hasPendingDirective = false
     }
 
     func resetForDocumentEnd(_ state: inout State) {
         state.hasDocumentContent = false
         state.hasDocumentStartMarker = false
         state.isDocumentClosed = true
+        state.sawYAMLDirective = false
+        state.hasPendingDirective = false
     }
 
     func scanDirective(_ state: inout State) throws {
@@ -58,25 +71,91 @@ extension PureYAML.Parsing.Scanner {
         let line = state.reader.consume { character in
             !isLineBreak(character)
         }
-        let parts = line.split(separator: " ").map(String.init)
-        guard let name = parts.first else {
+        let fields = directiveFields(line)
+        guard let name = fields.first else {
             return
         }
-        guard !state.hasDocumentContent, !state.hasDocumentStartMarker, !state.isDocumentClosed else {
-            throw PureYAML.Parsing.ParseError.unsupportedDirective(name: name, line: start.line)
-        }
         switch name {
         case "%YAML":
-            guard parts.indices.contains(1), parts[1] == "1.1" || parts[1] == "1.2" else {
-                throw PureYAML.Parsing.ParseError.incompatibleYAMLDirective(line: start.line)
-            }
+            try scanYAMLDirective(fields: fields, start: start, state: &state)
         case "%TAG":
-            guard parts.count >= 3 else {
-                return
-            }
-            state.tagHandles[parts[1]] = parts[2]
+            scanTagDirective(fields: fields, state: &state)
         default:
-            throw PureYAML.Parsing.ParseError.unsupportedDirective(name: name, line: start.line)
+            // Reserved or unknown directives are ignored with a warning per the
+            // spec; PureYAML accepts and discards them rather than erroring.
+            break
         }
+        state.hasPendingDirective = true
+    }
+
+    /// Splits a directive line into its `%name` and parameter fields, dropping
+    /// any trailing comment. Fields are separated by spaces or tabs, matching the
+    /// C scanner's blank handling around directive values.
+    private func directiveFields(_ line: String) -> [String] {
+        var fields: [String] = []
+        var current = ""
+        for character in line {
+            // A `#` at a field boundary (i.e. preceded by whitespace or the line
+            // start) begins a trailing comment that ends the directive's fields.
+            if character == "#", current.isEmpty {
+                break
+            }
+            if character == " " || character == "\t" {
+                if !current.isEmpty {
+                    fields.append(current)
+                    current = ""
+                }
+                continue
+            }
+            current.append(character)
+        }
+        if !current.isEmpty {
+            fields.append(current)
+        }
+        return fields
+    }
+
+    private func scanYAMLDirective(
+        fields: [String],
+        start: PureYAML.Parsing.Mark,
+        state: inout State,
+    ) throws {
+        // A `%YAML` directive carries exactly one version field of the form
+        // `major.minor`. A duplicate within one document, extra trailing fields,
+        // or an unsupported major version is an error; a higher 1.x minor is
+        // accepted (treated as 1.2 with a warning) so future minors keep parsing.
+        guard !state.sawYAMLDirective else {
+            throw PureYAML.Parsing.ParseError.incompatibleYAMLDirective(line: start.line)
+        }
+        guard fields.count == 2, isSupportedYAMLVersion(fields[1]) else {
+            throw PureYAML.Parsing.ParseError.incompatibleYAMLDirective(line: start.line)
+        }
+        state.sawYAMLDirective = true
+    }
+
+    private func isSupportedYAMLVersion(_ version: String) -> Bool {
+        let components = version.split(separator: ".", omittingEmptySubsequences: false)
+        guard components.count == 2 else {
+            return false
+        }
+        let major = components[0]
+        let minor = components[1]
+        guard !major.isEmpty, !minor.isEmpty else {
+            return false
+        }
+        guard major.allSatisfy(\.isNumber), minor.allSatisfy(\.isNumber) else {
+            return false
+        }
+        return major == "1"
+    }
+
+    private func scanTagDirective(
+        fields: [String],
+        state: inout State,
+    ) {
+        guard fields.count >= 3 else {
+            return
+        }
+        state.tagHandles[fields[1]] = fields[2]
     }
 }

Sources/Parsing/Scanner.swift

@@ -30,6 +30,9 @@ extension PureYAML.Parsing {
                 try scanToken(&state)
             }
 
+            if state.hasPendingDirective {
+                throw PureYAML.Parsing.ParseError.directiveWithoutDocument(line: state.reader.mark.line)
+            }
             closeIndentation(&state)
             state.append(.streamEnd)
             return state.tokens
@@ -46,6 +49,15 @@ extension PureYAML.Parsing.Scanner {
         var hasDocumentContent = false
         var hasDocumentStartMarker = false
         var isDocumentClosed = false
+        /// True once a `%YAML` directive has been scanned for the document that
+        /// the next `---` will start. A second `%YAML` before that `---` is a
+        /// duplicate and is rejected; the flag resets at every document boundary.
+        var sawYAMLDirective = false
+        /// True once any directive has been scanned in the current directive
+        /// section. A directive section must be terminated by a `---` document
+        /// start; reaching a `...` end marker or the end of the stream with a
+        /// directive still pending is an error.
+        var hasPendingDirective = false
         var flowDepth = 0
         var flowContexts: [PureYAML.Parsing.ScannerFlowContext] = []
         var validImplicitIndentation: Set<Int> = []

Sources/Parsing/TokenCursor.swift

@@ -79,6 +79,11 @@ extension PureYAML.Parsing {
 
             while scannerBox.state.tokens.isEmpty {
                 guard !scannerBox.state.reader.isAtEnd else {
+                    if scannerBox.state.hasPendingDirective {
+                        throw PureYAML.Parsing.ParseError.directiveWithoutDocument(
+                            line: scannerBox.state.reader.mark.line,
+                        )
+                    }
                     scanner.closeIndentation(&scannerBox.state)
                     scannerBox.state.append(.streamEnd)
                     drainScannedTokens(from: &scannerBox.state)

Sources/Parsing/TokenEventParser.swift

@@ -149,8 +149,13 @@ extension PureYAML.Parsing.TokenEventParser {
             endMark = dedent.endMark
         }
         if cursor.current?.kind.isDocumentEnd == true {
+            // An explicit `...` end marker closes the document outright. Whatever
+            // follows -- a bare node, an explicit `---`, or the end of the stream
+            // -- starts a fresh document on the next loop iteration, so emit the
+            // end here and return rather than demanding a document boundary token.
             let end = try expect("document end") { $0.isDocumentEnd }
-            endMark = end.endMark
+            try emit(.documentEnd(mark: end.endMark))
+            return
         }
         if let current = cursor.current, !canEndDocument(at: current) {
             throw PureYAML.Parsing.ParseError.unexpectedToken(

Tests/Fixtures/yaml-test-suite-gaps.yaml

@@ -55,15 +55,6 @@ cases:
     current: value-mismatch
     byteCount: 7
     lineCount: 0
-  - id: "2LFX"
-    name: "Spec Example 6.13. Reserved Directives [1.3]"
-    localPath: yaml-test-suite/2LFX/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 76
-    lineCount: 4
   - id: "2SXE"
     name: "Anchors With Colon in Name"
     localPath: yaml-test-suite/2SXE/in.yaml
@@ -172,15 +163,6 @@ cases:
     current: accepted
     byteCount: 12
     lineCount: 4
-  - id: "5TYM"
-    name: "Spec Example 6.21. Local Tag Prefix"
-    localPath: yaml-test-suite/5TYM/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 101
-    lineCount: 7
   - id: "5U3A"
     name: "Sequence on same Line as Mapping Key"
     localPath: yaml-test-suite/5U3A/in.yaml
@@ -238,30 +220,12 @@ cases:
   - id: "6LVF"
     name: "Spec Example 6.13. Reserved Directives"
     localPath: yaml-test-suite/6LVF/in.yaml
-    category: false-reject
+    category: json-mismatch
     expected: parse-ok
     oracle: json
-    current: parse-error
+    current: value-mismatch
     byteCount: 76
     lineCount: 3
-  - id: "6WLZ"
-    name: "Spec Example 6.18. Primary Tag Handle [1.3]"
-    localPath: yaml-test-suite/6WLZ/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 86
-    lineCount: 8
-  - id: "6ZKB"
-    name: "Spec Example 9.6. Stream"
-    localPath: yaml-test-suite/6ZKB/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 53
-    lineCount: 7
   - id: "7LBH"
     name: "Multiline double quoted implicit keys"
     localPath: yaml-test-suite/7LBH/in.yaml
@@ -271,15 +235,6 @@ cases:
     current: accepted
     byteCount: 20
     lineCount: 3
-  - id: "7Z25"
-    name: "Bare document after document end marker"
-    localPath: yaml-test-suite/7Z25/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 27
-    lineCount: 4
   - id: "7ZZ5"
     name: "Empty flow collections"
     localPath: yaml-test-suite/7ZZ5/in.yaml
@@ -352,15 +307,6 @@ cases:
     current: accepted
     byteCount: 20
     lineCount: 4
-  - id: "9DXL"
-    name: "Spec Example 9.6. Stream [1.3]"
-    localPath: yaml-test-suite/9DXL/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 62
-    lineCount: 7
   - id: "9JBA"
     name: "Invalid comment after end of flow sequence"
     localPath: yaml-test-suite/9JBA/in.yaml
@@ -388,15 +334,6 @@ cases:
     current: accepted
     byteCount: 18
     lineCount: 2
-  - id: "9MMA"
-    name: "Directive by itself with no document"
-    localPath: yaml-test-suite/9MMA/in.yaml
-    category: accepted-invalid
-    expected: reject
-    oracle: none
-    current: accepted
-    byteCount: 10
-    lineCount: 1
   - id: "9MMW"
     name: "Single Pair Implicit Entries"
     localPath: yaml-test-suite/9MMW/in.yaml
@@ -415,15 +352,6 @@ cases:
     current: parse-error
     byteCount: 15
     lineCount: 3
-  - id: "9WXW"
-    name: "Spec Example 6.18. Primary Tag Handle"
-    localPath: yaml-test-suite/9WXW/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 82
-    lineCount: 7
   - id: "9YRD"
     name: "Multiline Scalar at Top Level"
     localPath: yaml-test-suite/9YRD/in.yaml
@@ -451,24 +379,6 @@ cases:
     current: parse-error
     byteCount: 22
     lineCount: 5
-  - id: "B63P"
-    name: "Directive without document"
-    localPath: yaml-test-suite/B63P/in.yaml
-    category: accepted-invalid
-    expected: reject
-    oracle: none
-    current: accepted
-    byteCount: 14
-    lineCount: 2
-  - id: "BEC7"
-    name: "Spec Example 6.14. “YAML” directive"
-    localPath: yaml-test-suite/BEC7/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 65
-    lineCount: 4
   - id: "BF9H"
     name: "Trailing comment in multiline plain scalar"
     localPath: yaml-test-suite/BF9H/in.yaml
@@ -631,15 +541,6 @@ cases:
     current: accepted
     byteCount: 21
     lineCount: 3
-  - id: "H7TQ"
-    name: "Extra words on %YAML directive"
-    localPath: yaml-test-suite/H7TQ/in.yaml
-    category: accepted-invalid
-    expected: reject
-    oracle: none
-    current: accepted
-    byteCount: 18
-    lineCount: 2
   - id: "HRE5"
     name: "Double quoted scalar with escaped single quote"
     localPath: yaml-test-suite/HRE5/in.yaml
@@ -748,33 +649,6 @@ cases:
     current: parse-error
     byteCount: 74
     lineCount: 7
-  - id: "MUS6/03"
-    name: "Directive variants"
-    localPath: yaml-test-suite/MUS6/03/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 16
-    lineCount: 2
-  - id: "MUS6/05"
-    name: "Directive variants"
-    localPath: yaml-test-suite/MUS6/05/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 13
-    lineCount: 2
-  - id: "MUS6/06"
-    name: "Directive variants"
-    localPath: yaml-test-suite/MUS6/06/in.yaml
-    category: false-reject
-    expected: parse-ok
-    oracle: json
-    current: parse-error
-    byteCount: 15
-    lineCount: 2
   - id: "NB6Z"
     name: "Multiline plain value with tabs on empty lines"
     localPath: yaml-test-suite/NB6Z/in.yaml
@@ -865,15 +739,6 @@ cases:
     current: accepted
     byteCount: 42
     lineCount: 5
-  - id: "SF5V"
-    name: "Duplicate YAML directive"
-    localPath: yaml-test-suite/SF5V/in.yaml
-    category: accepted-invalid
-    expected: reject
-    oracle: none
-    current: accepted
-    byteCount: 24
-    lineCount: 3
   - id: "SKE5"
     name: "Anchor before zero indented sequence"
     localPath: yaml-test-suite/SKE5/in.yaml
@@ -985,10 +850,10 @@ cases:
   - id: "W4TN"
     name: "Spec Example 9.5. Directives Documents"
     localPath: yaml-test-suite/W4TN/in.yaml
-    category: false-reject
+    category: json-mismatch
     expected: parse-ok
     oracle: json
-    current: parse-error
+    current: value-mismatch
     byteCount: 61
     lineCount: 8
   - id: "X4QW"

Tests/Fixtures/yaml-test-suite/2LFX/in.json

@@ -1 +0,0 @@
-"foo"

Tests/Fixtures/yaml-test-suite/2LFX/in.yaml

@@ -1,4 +0,0 @@
-%FOO  bar baz # Should be ignored
-              # with a warning.
----
-"foo"

Tests/Fixtures/yaml-test-suite/5TYM/in.json

@@ -1,2 +0,0 @@
-"fluorescent"
-"green"

Tests/Fixtures/yaml-test-suite/5TYM/in.yaml

@@ -1,7 +0,0 @@
-%TAG !m! !my-
---- # Bulb here
-!m!light fluorescent
-...
-%TAG !m! !my-
---- # Color here
-!m!light green

Tests/Fixtures/yaml-test-suite/6WLZ/in.json

@@ -1,2 +0,0 @@
-"bar"
-"bar"

Tests/Fixtures/yaml-test-suite/6WLZ/in.yaml

@@ -1,8 +0,0 @@
-# Private
----
-!foo "bar"
-...
-# Global
-%TAG ! tag:example.com,2000:app/
----
-!foo "bar"

Tests/Fixtures/yaml-test-suite/6ZKB/in.json

@@ -1,5 +0,0 @@
-"Document"
-null
-{
-  "matches %": 20
-}

Tests/Fixtures/yaml-test-suite/6ZKB/in.yaml

@@ -1,7 +0,0 @@
-Document
----
-# Empty
-...
-%YAML 1.2
----
-matches %: 20

Tests/Fixtures/yaml-test-suite/7Z25/in.json

@@ -1,4 +0,0 @@
-"scalar1"
-{
-  "key": "value"
-}

Tests/Fixtures/yaml-test-suite/7Z25/in.yaml

@@ -1,4 +0,0 @@
----
-scalar1
-...
-key: value

Tests/Fixtures/yaml-test-suite/9DXL/in.json

@@ -1,7 +0,0 @@
-{
-  "Mapping": "Document"
-}
-null
-{
-  "matches %": 20
-}

Tests/Fixtures/yaml-test-suite/9DXL/in.yaml

@@ -1,7 +0,0 @@
-Mapping: Document
----
-# Empty
-...
-%YAML 1.2
----
-matches %: 20

Tests/Fixtures/yaml-test-suite/9MMA/in.yaml

@@ -1 +0,0 @@
-%YAML 1.2

Tests/Fixtures/yaml-test-suite/9WXW/in.json

@@ -1,2 +0,0 @@
-"bar"
-"bar"

Tests/Fixtures/yaml-test-suite/9WXW/in.yaml

@@ -1,7 +0,0 @@
-# Private
-!foo "bar"
-...
-# Global
-%TAG ! tag:example.com,2000:app/
----
-!foo "bar"

Tests/Fixtures/yaml-test-suite/B63P/in.yaml

@@ -1,2 +0,0 @@
-%YAML 1.2
-...

Tests/Fixtures/yaml-test-suite/BEC7/in.json

@@ -1 +0,0 @@
-"foo"

Tests/Fixtures/yaml-test-suite/BEC7/in.yaml

@@ -1,4 +0,0 @@
-%YAML 1.3 # Attempt parsing
-          # with a warning
----
-"foo"

Tests/Fixtures/yaml-test-suite/H7TQ/in.yaml

@@ -1,2 +0,0 @@
-%YAML 1.2 foo
----

Tests/Fixtures/yaml-test-suite/MUS6/03/in.json

@@ -1 +0,0 @@
-null

Tests/Fixtures/yaml-test-suite/MUS6/03/in.yaml

@@ -1,2 +0,0 @@
-%YAML 	 1.1
----

Tests/Fixtures/yaml-test-suite/MUS6/05/in.json

@@ -1 +0,0 @@
-null

Tests/Fixtures/yaml-test-suite/MUS6/05/in.yaml

@@ -1,2 +0,0 @@
-%YAM 1.1
----

Tests/Fixtures/yaml-test-suite/MUS6/06/in.json

@@ -1 +0,0 @@
-null

Tests/Fixtures/yaml-test-suite/MUS6/06/in.yaml

@@ -1,2 +0,0 @@
-%YAMLL 1.1
----

Tests/Fixtures/yaml-test-suite/SF5V/in.yaml

@@ -1,3 +0,0 @@
-%YAML 1.2
-%YAML 1.2
----

Tests/ParsingErrorTests.swift

@@ -16,7 +16,7 @@ struct ParsingErrorTests {
         expectParseError("name: \"open", .unterminatedQuotedString(line: 1))
         expectParseError("name: 'open", .unterminatedQuotedString(line: 1))
         expectParseError("second: *missing", .undefinedAlias(anchor: "missing", line: 1, column: 9))
-        expectParseError("%YAML 1.3\n---\nvalue: yes", .incompatibleYAMLDirective(line: 1))
+        expectParseError("%YAML 2.0\n---\nvalue: yes", .incompatibleYAMLDirective(line: 1))
         expectParseError("value: !!int nope", .invalidTaggedScalar(
             tag: "tag:yaml.org,2002:int",
             value: "nope",
@@ -56,6 +56,7 @@ struct ParsingErrorTests {
         #expect(PureYAML.Parsing.ParseError.unterminatedTag(line: 8).description == "unterminated tag at line 8")
         #expect(PureYAML.Parsing.ParseError.unterminatedQuotedString(line: 9).description == "unterminated quoted string at line 9")
         #expect(PureYAML.Parsing.ParseError.incompatibleYAMLDirective(line: 10).description == "incompatible YAML directive at line 10")
+        #expect(PureYAML.Parsing.ParseError.directiveWithoutDocument(line: 10).description == "directive is not followed by a document start at line 10")
         #expect(PureYAML.Parsing.ParseError.unsupportedDirective(name: "%FOO", line: 10).description == "unsupported directive '%FOO' at line 10")
         #expect(PureYAML.Parsing.ParseError.unsupportedMultiDocumentStream(line: 10).description == "multi-document streams are not supported at line 10")
         #expect(PureYAML.Parsing.ParseError.expectedNode(line: 10, column: 2).description == "expected a YAML node at line 10, column 2")

Tests/ScannerTests.swift

@@ -264,10 +264,10 @@ struct ScannerTests {
         expectScannerError("anchor: &", .expectedAnchorName(line: 1))
         expectScannerError("alias: *", .expectedAliasName(line: 1))
         expectScannerError("tag: !<tag:example.com,2026:thing", .unterminatedTag(line: 1))
-        expectScannerError("%YAML 1.3\n---\nvalue: yes", .incompatibleYAMLDirective(line: 1))
-        expectScannerError("%FOO bar\nvalue: yes", .unsupportedDirective(name: "%FOO", line: 1))
-        expectScannerError("---\n%TAG !e! tag:example.com,2026:", .unsupportedDirective(name: "%TAG", line: 2))
-        expectScannerError("value: one\n...\nvalue: two", .unsupportedMultiDocumentStream(line: 3))
+        expectScannerError("%YAML 2.0\n---\nvalue: yes", .incompatibleYAMLDirective(line: 1))
+        expectScannerError("%YAML 1.1\n%YAML 1.2\n---\nvalue: yes", .incompatibleYAMLDirective(line: 2))
+        expectScannerError("%YAML 1.2\n", .directiveWithoutDocument(line: 2))
+        expectScannerError("%FOO bar\nvalue: yes", .directiveWithoutDocument(line: 2))
     }
 }
 

Tests/StreamParsingTests.swift

@@ -140,17 +140,27 @@ struct StreamParsingTests {
         )
     }
 
-    @Test("Stream parser rejects content after explicit document end without next start")
-    func test_streamParserRejectsContentAfterExplicitDocumentEndWithoutNextStart() {
-        expectStreamParseError(
+    @Test("Stream parser begins a new bare document after an explicit document end")
+    func test_streamParserBeginsNewBareDocumentAfterExplicitDocumentEnd() throws {
+        // A `...` end marker closes the current document; content that follows
+        // without a `---` starts a fresh bare document, matching YAML test-suite
+        // case 7Z25.
+        let documents = try PureYAML.parseStream(
             """
             name: First
             ...
             name: Second
             """,
-            .unsupportedMultiDocumentStream(line: 3),
-            description: "multi-document streams are not supported at line 3",
         )
+
+        #expect(documents == [
+            .init(index: 0, value: .mapping(.init([
+                .init(key: "name", value: .string("First")),
+            ]))),
+            .init(index: 1, value: .mapping(.init([
+                .init(key: "name", value: .string("Second")),
+            ]))),
+        ])
     }
 
     @Test("Stream parser rejects extra root content without a document start")

Tests/UnsupportedYAMLGapsFixtures.swift

@@ -58,32 +58,16 @@ enum UnsupportedYAMLGapsFixtures {
             expectedDescription: "multi-document streams are not supported at line 3",
         ),
         ParseErrorCase(
-            name: "unknown directive",
+            // A directive must be followed by a `---` document start. A reserved
+            // or unknown directive is otherwise ignored, but here it is trailed
+            // by bare content rather than a document start, which is invalid.
+            name: "directive without a document start",
             yaml: """
             %FOO bar
             value: one
             """,
-            expected: .unsupportedDirective(name: "%FOO", line: 1),
-            expectedDescription: "unsupported directive '%FOO' at line 1",
-        ),
-        ParseErrorCase(
-            name: "tag directive after content",
-            yaml: """
-            value: one
-            %TAG !e! tag:example.com,2026:
-            """,
-            expected: .unsupportedDirective(name: "%TAG", line: 2),
-            expectedDescription: "unsupported directive '%TAG' at line 2",
-        ),
-        ParseErrorCase(
-            name: "tag directive after document start",
-            yaml: """
-            ---
-            %TAG !e! tag:example.com,2026:
-            value: !e!thing one
-            """,
-            expected: .unsupportedDirective(name: "%TAG", line: 2),
-            expectedDescription: "unsupported directive '%TAG' at line 2",
+            expected: .directiveWithoutDocument(line: 2),
+            expectedDescription: "directive is not followed by a document start at line 2",
         ),
     ]
 
@@ -124,6 +108,34 @@ enum UnsupportedYAMLGapsFixtures {
                 .init(key: "name", value: .string("First")),
             ])),
         ),
+        FallbackValueCase(
+            // After document content has begun, a line that starts with `%` is
+            // plain text, not a directive, so it reads as an ordinary mapping
+            // entry rather than a `%TAG`/`%YAML` directive.
+            name: "percent after content is plain text not a directive",
+            yaml: """
+            value: one
+            %TAG !e! tag:example.com,2026:
+            """,
+            expected: .mapping(.init([
+                .init(key: "value", value: .string("one")),
+                .init(key: "%TAG !e! tag:example.com,2026", value: .null),
+            ])),
+        ),
+        FallbackValueCase(
+            // The same holds after an explicit `---`: the document has started,
+            // so `%` on the next line is plain content, not a directive.
+            name: "percent after document start is plain text not a directive",
+            yaml: """
+            ---
+            %TAG !e! tag:example.com,2026:
+            value: !e!thing one
+            """,
+            expected: .mapping(.init([
+                .init(key: "%TAG !e! tag:example.com,2026", value: .null),
+                .init(key: "value", value: .string("one")),
+            ])),
+        ),
     ]
 
     static let directPreservedValue = PureYAML.Model.Value.mapping(.init([

Tests/YAMLTestSuiteGapsTests.swift

@@ -18,10 +18,10 @@ struct YAMLTestSuiteGapsTests {
         let manifest = try YAMLTestSuiteManifest.load()
 
         #expect(manifest.version == 1)
-        #expect(manifest.cases.count == 122)
-        #expect(manifest.cases.count(where: { $0.category == "false-reject" }) == 53)
-        #expect(manifest.cases.count(where: { $0.category == "json-mismatch" }) == 18)
-        #expect(manifest.cases.count(where: { $0.category == "accepted-invalid" }) == 51)
+        #expect(manifest.cases.count == 107)
+        #expect(manifest.cases.count(where: { $0.category == "false-reject" }) == 40)
+        #expect(manifest.cases.count(where: { $0.category == "json-mismatch" }) == 20)
+        #expect(manifest.cases.count(where: { $0.category == "accepted-invalid" }) == 47)
         #expect(Set(manifest.cases.map(\.id)).count == manifest.cases.count)
 
         for testCase in manifest.cases {

docs/YAML_TEST_SUITE_CONFORMANCE.md

@@ -22,9 +22,10 @@ its own oracle:
 
 402 cases (including numbered sub-cases). First measurement was 196 pass / 206
 fail; after bucket 1 (block scalars, +24), bucket 2 (quoted-scalar folding, +30),
-bucket 3 (empty nodes and empty keys, +20), and bucket 4 (anchors, tags, and
-aliases as mapping keys, +10) it is now **280 pass, 122 fail.**
-The 122 remaining failing cases are vendored under
+bucket 3 (empty nodes and empty keys, +20), bucket 4 (anchors, tags, and
+aliases as mapping keys, +10), and bucket 5 (directives and document boundaries,
++15) it is now **295 pass, 107 fail.**
+The 107 remaining failing cases are vendored under
 `Tests/Fixtures/yaml-test-suite/` and recorded in
 `Tests/Fixtures/yaml-test-suite-gaps.yaml`. Run the opt-in sweep with:
 
@@ -42,9 +43,9 @@ updated, the deliberate forcing function for keeping this document honest.
 
 | Category | Count | Meaning |
 |---|---|---|
-| `false-reject` | 53 | Well-formed YAML that PureYAML wrongly rejects at parse time. |
-| `json-mismatch` | 18 | Parses, but produces a value that disagrees with the expected JSON. |
-| `accepted-invalid` | 51 | Malformed YAML that PureYAML wrongly accepts. |
+| `false-reject` | 40 | Well-formed YAML that PureYAML wrongly rejects at parse time. |
+| `json-mismatch` | 20 | Parses, but produces a value that disagrees with the expected JSON. |
+| `accepted-invalid` | 47 | Malformed YAML that PureYAML wrongly accepts. |
 
 ## Root-cause buckets and fix order
 
@@ -152,25 +153,66 @@ but its compact block-mapping structure still disagrees with the expected JSON,
 so it moved from `false-reject` to `json-mismatch` rather than passing; that
 structure is a separate compact-mapping gap.
 
-### 5. Directives and multi-document streams (≈33 cases, `false-reject`)
+### 5. Directives and document boundaries (DONE)
 
-`%YAML`/`%TAG`/unknown directives and `---`/`...` document boundaries are
-explicitly unsupported (`unsupportedDirective`, `unsupportedMultiDocumentStream`).
-This is a feature, not a bug fix, and the largest structural change. Examples: the
-`%YAML`/`%TAG`/`%FOO` cases, `UT92`, `W4TN`, `M7A3`.
+Directives are now handled and the `...` document-end marker may be followed by a
+new document. Three root causes were fixed:
 
-- **Reference:** `scanner.c` directive scanning + `parser.c` document-stream state
-  machine.
-- **Why last:** it is the most invasive (parser state machine + emitter round-trip)
-  and the existing `UnsupportedYAMLGapsFixtures` deliberately pins the current
-  rejection behavior, so this bucket must update those fixtures in lockstep.
+1. **`%` position detection.** A `%` begins a directive only in directive
+   position: before any document content and before the `---` that starts a
+   document (which includes the position immediately after a `...` end marker).
+   After content, after `---`, or in any flow context, `%` is ordinary plain
+   text. The scanner previously treated `%` at column 1 as a directive
+   unconditionally and rejected it once content had begun, mis-firing on
+   `XLQ9`, `UT92`, `W4TN`, `6ZKB`, `9DXL`. This mirrors `scanner.c`, which only
+   enters `yaml_parser_scan_directive` from the start-of-line directive context.
+2. **Lenient directive parsing.** `%YAML` accepts a `1.x` version (a higher minor
+   is treated as 1.2 with a warning, not an error, so `%YAML 1.3` parses) while
+   rejecting an unsupported major version, a duplicate `%YAML` in one document,
+   or extra non-comment fields. Reserved and unknown directives, and `%TAG`
+   handle definitions, are accepted and ignored. A directive must be followed by
+   a `---` document start. This follows `parser.c` `process_directives` (duplicate
+   and version checks) but relaxes the unknown-directive and minor-version errors
+   to the spec's reserved-directive warning behavior.
+3. **Document end then new document.** A `...` end marker closes the current
+   document; content that follows (with or without an explicit `---`) starts a
+   fresh document in the multi-document stream parser. The scanner no longer
+   rejects this as an unsupported multi-document stream; the single-document
+   `parse` still rejects multi-document input at compose time.
 
-### 6. Stricter rejection of invalid input (54 cases, `accepted-invalid`)
+Implemented in `Sources/Parsing/Scanner.Directives.swift` (directive scanning,
+`%` position gate, `...` boundary), `Sources/Parsing/TokenEventParser.swift`
+(a `...` end marker returns to start a new document), and
+`Sources/Parsing/TokenCursor.swift` (the streaming scanner rejects a pending
+directive at end of stream). The new `directiveWithoutDocument` `ParseError`
+covers a directive with no following `---`.
+
+- **Reference:** `scanner.c` `yaml_parser_scan_directive` /
+  `yaml_parser_scan_version_directive_value` + `parser.c`
+  `yaml_parser_process_directives` and the document-start state machine.
+
+Flipped 11 cases to passing with matching values: `2LFX`, `5TYM`, `6WLZ`,
+`6ZKB`, `7Z25`, `9DXL`, `9WXW`, `BEC7`, `MUS6/03`, `MUS6/05`, `MUS6/06`.
+Correctly *rejected* four cases that were previously accepted-invalid: `9MMA`
+and `B63P` (directive without a document), `H7TQ` (extra words on `%YAML`), and
+`SF5V` (duplicate `%YAML`). `6LVF` and `W4TN` no longer parse-error but their
+values still disagree with the expected JSON, so they moved from `false-reject`
+to `json-mismatch`: `6LVF` needs same-line content after `---` (the `KSS4`
+"scalars on `---` line" gap) and `W4TN` needs literal block-scalar clip chomping
+to keep the trailing newline (the `T5N4`/`4WA9` literal-scalar gap). `XLQ9` and
+`UT92` no longer mis-fire on `%` but still parse-error on top-level and flow
+multi-line plain-scalar folding respectively (the `9YRD`/`EX5H` and `NJ66`
+gaps), so they remain recorded. `%TAG` shorthand *resolution* (rejecting a
+handle used in a later document where it is undefined, e.g. `QLJ7`) is left to
+the tag-resolution work and those cases stay recorded as accepted-invalid.
+
+### 6. Stricter rejection of invalid input (47 cases, `accepted-invalid`)
 
 Inputs that must be rejected are silently accepted: tabs used as indentation,
-duplicate `%YAML` directives, bad document-end markers inside quoted scalars,
-malformed flow collections. Examples: `SF5V`, `QB6E`, `RXY3`, `ZVH3`, the `Y79Y`
-tab cases.
+bad document-end markers inside quoted scalars, malformed flow collections, and
+`%TAG` shorthands used in a document that does not define them. Examples:
+`QB6E`, `RXY3`, `ZVH3`, `QLJ7`, the `Y79Y` tab cases. (The directive-shape
+rejections, duplicate `%YAML` `SF5V` and extra-words `H7TQ`, landed with bucket 5.)
 
 - **Reference:** the corresponding error productions in `scanner.c`/`parser.c`.
 - **Why last among the parse work:** several of these only become reachable once