feat(parser): expand YAML merge keys

This commit is contained in:
Mihaela Mihaljevic 2026-06-05 15:12:55 +02:00
parent 8be6e68a23
commit e1f8acb3a0
13 changed files with 389 additions and 74 deletions

View File

@ -15,11 +15,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add `PureYAML.Stream.Document`, document-indexed stream validation issues, and
stream validation APIs that preserve document indexes without changing
document-local validation paths.
- Add merge-key expansion for plain `<<` and explicit `!!merge` keys, including
scalar merge mappings, sequence-of-mapping merge values, local override
behavior, and exact errors for invalid merge values.
### Changed
- Tokenize YAML document start and end markers so scanner, event-parser, and
stream tests can pin exact document-boundary behavior.
- Keep quoted `"<<":` and explicitly string-tagged `!!str <<` keys as ordinary
string keys while merge syntax is expanded during parsing.
## [0.1.0] - 2026-06-05

View File

@ -45,18 +45,19 @@ This repository has completed readiness for the first tagged release, version
0.1.0. Current main includes block mappings, block sequences, ordered mappings,
common scalars, quoted strings, comments, flow collections, literal and folded
block scalars, anchors, aliases, YAML directives, document markers, explicit
built-in scalar tags, multi-document stream parsing, and a matching dumper with
block and flow output policies. It also includes path-aware validation for
structural YAML checks such as duplicate mapping keys, plus stream validation
that preserves document indexes.
built-in scalar tags, merge-key expansion, multi-document stream parsing, and a
matching dumper with block and flow output policies. It also includes path-aware
validation for structural YAML checks such as duplicate mapping keys, plus
stream validation that preserves document indexes.
It is not yet a full YAML 1.2 implementation. The internal event parser now
recognizes anchors, aliases, tags, flow collections, and block scalar styles from
scanner tokens, and the public value parser composes those events into
`PureYAML.Model.Value`. Full tag-specific collection semantics, merge keys,
directives beyond the selected compatibility subset, and custom decoding are
planned work. Unsupported gaps are pinned by executable tests so they produce
exact errors or exact fallback value trees instead of silent parser drift.
`PureYAML.Model.Value`. Full tag-specific collection semantics, complex mapping
keys, directives beyond the selected compatibility subset, and custom decoding
are planned work. Unsupported gaps are pinned by executable tests so they
produce exact errors or exact fallback value trees instead of silent parser
drift.
## Attribution

View File

@ -0,0 +1,107 @@
extension PureYAML.Parsing.EventComposer {
struct ComposedMappingKey {
var value: String
var tag: String?
var style: PureYAML.Parsing.ScalarStyle
var isMergeKey: Bool {
tag == "tag:yaml.org,2002:merge"
|| (tag == nil && style == .plain && value == "<<")
}
}
mutating func composeMergeSources() throws -> [[PureYAML.Model.Pair]] {
guard let event = current else {
throw unexpectedEvent(expected: "merge value")
}
switch event {
case .sequenceStart:
return try composeMergeSequenceSources()
default:
let value = try composeNode()
return try mergeSources(from: value, mark: event.mark)
}
}
mutating func composeMergeSequenceSources() throws -> [[PureYAML.Model.Pair]] {
_ = try expect("merge sequence start") { event in
if case .sequenceStart = event {
return true
}
return false
}
var sources: [[PureYAML.Model.Pair]] = []
while current?.isSequenceEnd != true {
guard let event = current else {
throw unexpectedEvent(expected: "merge sequence item")
}
let value = try composeNode()
try sources.append(mergeSequenceItemSource(from: value, mark: event.mark))
}
_ = try expect("merge sequence end") { $0.isSequenceEnd }
return sources
}
func mergeSources(
from value: PureYAML.Model.Value,
mark: PureYAML.Parsing.Mark,
) throws -> [[PureYAML.Model.Pair]] {
switch value {
case let .mapping(mapping):
return [mapping.pairs]
case let .sequence(values):
return try values.map { value in
guard case let .mapping(mapping) = value else {
throw invalidMergeValue(mark: mark)
}
return mapping.pairs
}
case .bool, .double, .int, .null, .string:
throw invalidMergeValue(mark: mark)
}
}
func mergeSequenceItemSource(
from value: PureYAML.Model.Value,
mark: PureYAML.Parsing.Mark,
) throws -> [PureYAML.Model.Pair] {
guard case let .mapping(mapping) = value else {
throw invalidMergeValue(mark: mark)
}
return mapping.pairs
}
func mergedPairs(
from mergeSources: [[PureYAML.Model.Pair]],
localPairs: [PureYAML.Model.Pair],
) -> [PureYAML.Model.Pair] {
guard !mergeSources.isEmpty else {
return localPairs
}
let localKeys = Set(localPairs.map(\.key))
var inheritedKeys = Set<String>()
var inheritedPairs: [PureYAML.Model.Pair] = []
for source in mergeSources {
var sourceKeys = Set<String>()
for pair in source {
guard !localKeys.contains(pair.key),
!inheritedKeys.contains(pair.key)
else {
continue
}
inheritedPairs.append(pair)
sourceKeys.insert(pair.key)
}
inheritedKeys.formUnion(sourceKeys)
}
return inheritedPairs + localPairs
}
func invalidMergeValue(mark: PureYAML.Parsing.Mark) -> PureYAML.Parsing.ParseError {
PureYAML.Parsing.ParseError.invalidMergeValue(
line: mark.line,
column: mark.column,
)
}
}

View File

@ -137,27 +137,37 @@ extension PureYAML.Parsing.EventComposer {
}
return false
}
var pairs: [PureYAML.Model.Pair] = []
var mergeSources: [[PureYAML.Model.Pair]] = []
var localPairs: [PureYAML.Model.Pair] = []
while current?.isMappingEnd != true {
let key = try composeMappingKey()
let value = try composeNode()
pairs.append(PureYAML.Model.Pair(key: key, value: value))
if key.isMergeKey {
try mergeSources.append(contentsOf: composeMergeSources())
} else {
let value = try composeNode()
localPairs.append(PureYAML.Model.Pair(key: key.value, value: value))
}
}
_ = try expect("mapping end") { $0.isMappingEnd }
let pairs = mergedPairs(from: mergeSources, localPairs: localPairs)
return .mapping(PureYAML.Model.Mapping(pairs))
}
mutating func composeMappingKey() throws -> String {
mutating func composeMappingKey() throws -> ComposedMappingKey {
guard let event = current else {
throw unexpectedEvent(expected: "mapping key")
}
guard case let .scalar(value, anchor, _, _, _) = event else {
guard case let .scalar(value, anchor, tag, style, _) = event else {
let mark = event.mark
throw PureYAML.Parsing.ParseError.expectedScalarKey(line: mark.line, column: mark.column)
}
index += 1
store(.string(value), anchor: anchor)
return value
return ComposedMappingKey(
value: value,
tag: normalizedScalarTag(tag),
style: style,
)
}
func composeScalar(
@ -238,6 +248,8 @@ extension PureYAML.Parsing.EventComposer {
return "tag:yaml.org,2002:bool"
case "!!null":
return "tag:yaml.org,2002:null"
case "!!merge":
return "tag:yaml.org,2002:merge"
default:
return tag
}

View File

@ -12,6 +12,7 @@ public extension PureYAML.Parsing {
case expectedScalarKey(line: Int, column: Int)
case incompatibleYAMLDirective(line: Int)
case invalidTaggedScalar(tag: String, value: String, line: Int, column: Int)
case invalidMergeValue(line: Int, column: Int)
case unterminatedTag(line: Int)
case unterminatedQuotedString(line: Int)
case unsupportedDirective(name: String, line: Int)
@ -44,6 +45,8 @@ public extension PureYAML.Parsing {
"incompatible YAML directive at line \(line)"
case let .invalidTaggedScalar(tag, value, line, column):
"invalid scalar '\(value)' for tag '\(tag)' at line \(line), column \(column)"
case let .invalidMergeValue(line, column):
"expected merge value to be a mapping or sequence of mappings at line \(line), column \(column)"
case let .unterminatedTag(line):
"unterminated tag at line \(line)"
case let .unterminatedQuotedString(line):

View File

@ -101,22 +101,21 @@ struct DownstreamDocumentTests {
)
}
@Test("Pins unsupported downstream merge keys as unflattened fallback values")
func test_downstreamMergeKeyFallbackFixture() throws {
@Test("Expands downstream merge keys into effective service mappings")
func test_downstreamMergeKeyFixture() throws {
let root = try requireRootMapping(downstreamMergeKeyYAML)
let defaults = expectMapping(root["defaults"], "expected defaults mapping")
let services = expectMapping(root["services"], "expected services mapping")
let api = expectMapping(services?["api"], "expected api service mapping")
let merge = expectMapping(api?["<<"], "expected unflattened merge mapping")
#expect(root.pairs.map(\.key) == ["defaults", "services"])
#expect(api?.pairs.map(\.key) == ["<<", "name"])
#expect(api?.pairs.map(\.key) == ["retries", "timeoutSeconds", "name"])
#expect(defaults?["retries"] == .int(3))
#expect(defaults?["timeoutSeconds"] == .int(30))
#expect(merge == defaults)
#expect(api?["<<"] == nil)
#expect(api?["name"] == .string("API"))
#expect(api?["retries"] == nil)
#expect(api?["timeoutSeconds"] == nil)
#expect(api?["retries"] == .int(3))
#expect(api?["timeoutSeconds"] == .int(30))
#expect(PureYAML.Validation.Validator().collect(.mapping(root)) == .init())
}

View File

@ -0,0 +1,198 @@
@testable import PureYAML
import Testing
@Suite("Merge Key Compatibility")
struct MergeKeyCompatibilityTests {
@Test("Expands a scalar merge key from an anchored mapping")
func test_scalarMergeKeyExpandsAnchoredMapping() throws {
let root = try requireMapping(PureYAML.parse(
"""
defaults: &defaults {enabled: true, retries: 3}
service:
<<: *defaults
name: API
""",
))
let service = requireMapping(root?["service"], "expected service mapping")
#expect(service?.pairs == [
.init(key: "enabled", value: .bool(true)),
.init(key: "retries", value: .int(3)),
.init(key: "name", value: .string("API")),
])
#expect(service?["enabled"] == .bool(true))
#expect(service?["retries"] == .int(3))
#expect(service?["name"] == .string("API"))
#expect(service?["<<"] == nil)
#expect(PureYAML.Validation.Validator().collect(.mapping(root ?? .init())) == .init())
}
@Test("Expands merge sequence values with first source precedence and local overrides")
func test_mergeSequencePrecedenceAndLocalOverrides() throws {
let root = try requireSequence(PureYAML.parse(
"""
- &BIG {r: 10}
- &LEFT {x: 0, y: 2}
- &SMALL {r: 1}
- <<: [*BIG, *LEFT, *SMALL]
x: 1
label: center/big
""",
))
let merged = requireMapping(root?.last, "expected merged mapping")
#expect(merged?.pairs == [
.init(key: "r", value: .int(10)),
.init(key: "y", value: .int(2)),
.init(key: "x", value: .int(1)),
.init(key: "label", value: .string("center/big")),
])
#expect(merged?["r"] == .int(10))
#expect(merged?["x"] == .int(1))
#expect(merged?["y"] == .int(2))
#expect(merged?["<<"] == nil)
}
@Test("Expands a merge value alias that resolves to a sequence of mappings")
func test_mergeValueAliasResolvesToSequenceOfMappings() throws {
let root = try requireMapping(PureYAML.parse(
"""
left: &left {x: 1}
right: &right {y: 2}
sources: &sources [*left, *right]
service:
<<: *sources
name: API
""",
))
let service = requireMapping(root?["service"], "expected service mapping")
#expect(service?.pairs == [
.init(key: "x", value: .int(1)),
.init(key: "y", value: .int(2)),
.init(key: "name", value: .string("API")),
])
}
@Test("Preserves duplicate local keys after merge expansion for validation")
func test_mergeExpansionPreservesDuplicateLocalKeysForValidation() throws {
let value = try PureYAML.parse(
"""
defaults: &defaults {retries: 3, timeoutSeconds: 30}
service:
<<: *defaults
retries: 5
name: API
name: Backend
""",
)
let result = PureYAML.Validation.Validator().collect(value)
let expectedIssues = [
PureYAML.Validation.Issue(
severity: .error,
reason: "Duplicate mapping key 'name'",
path: .init([.key("service"), .key("name")]),
),
]
#expect(result.issues == expectedIssues)
#expect(result.errors == expectedIssues)
expectValidationError(value) { collection in
#expect(collection.issues == expectedIssues)
#expect(collection.description == "error: Duplicate mapping key 'name' at $.service.name")
}
let root = requireMapping(value, "expected root mapping")
let service = requireMapping(root?["service"], "expected service mapping")
#expect(service?.pairs.map(\.key) == ["timeoutSeconds", "retries", "name", "name"])
#expect(service?["retries"] == .int(5))
}
@Test("Keeps quoted and explicit string merge-like keys ordinary")
func test_quotedAndStringTaggedMergeKeysRemainOrdinary() throws {
let root = try requireMapping(PureYAML.parse(
"""
quoted:
"<<": literal
tagged:
? !!str <<
: literal
""",
))
let quoted = requireMapping(root?["quoted"], "expected quoted mapping")
let tagged = requireMapping(root?["tagged"], "expected tagged mapping")
#expect(quoted?.pairs == [
.init(key: "<<", value: .string("literal")),
])
#expect(tagged?.pairs == [
.init(key: "<<", value: .string("literal")),
])
#expect(quoted?["<<"] == .string("literal"))
#expect(tagged?["<<"] == .string("literal"))
}
@Test("Expands explicit merge-tagged keys")
func test_explicitMergeTaggedKeyExpands() throws {
let root = try requireMapping(PureYAML.parse(
"""
defaults: &defaults {enabled: true}
service:
!!merge <<: *defaults
name: API
""",
))
let service = requireMapping(root?["service"], "expected service mapping")
#expect(service?.pairs == [
.init(key: "enabled", value: .bool(true)),
.init(key: "name", value: .string("API")),
])
#expect(service?["<<"] == nil)
}
@Test("Rejects scalar merge values exactly")
func test_scalarMergeValueFailsExactly() {
expectParseError(
"""
service:
<<: nope
""",
.invalidMergeValue(line: 2, column: 7),
)
}
@Test("Rejects non-mapping merge sequence entries exactly")
func test_nonMappingMergeSequenceEntryFailsExactly() {
expectParseError(
"""
defaults: &defaults {enabled: true}
service:
<<: [*defaults, nope]
""",
.invalidMergeValue(line: 3, column: 19),
)
}
@Test("Rejects nested merge sequence entries exactly")
func test_nestedMergeSequenceEntryFailsExactly() {
expectParseError(
"""
service:
<<: [[{enabled: true}]]
""",
.invalidMergeValue(line: 2, column: 8),
)
}
}
private func requireMapping(
_ value: PureYAML.Model.Value?,
_ message: String,
) -> PureYAML.Model.Mapping? {
guard case let .mapping(mapping)? = value else {
recordIssue(message)
return nil
}
return mapping
}

View File

@ -86,8 +86,8 @@ struct ParsingCompatibilityTests {
}
}
@Test("Pins merge keys as unsupported unflattened mappings")
func test_mergeKeysRemainUnflattenedUntilExplicitlySupported() throws {
@Test("Expands merge keys while keeping source mappings visible")
func test_mergeKeysExpandIntoEffectiveMappings() throws {
let root = try requireMapping(PureYAML.parse(
"""
defaults: &defaults {enabled: true, retries: 3}
@ -97,19 +97,17 @@ struct ParsingCompatibilityTests {
""",
))
guard
case let .mapping(service)? = root?["service"],
case let .mapping(mergedDefaults)? = service["<<"]
else {
recordIssue("expected unflattened merge-key mapping")
guard case let .mapping(service)? = root?["service"] else {
recordIssue("expected merged service mapping")
return
}
#expect(mergedDefaults["enabled"] == .bool(true))
#expect(mergedDefaults["retries"] == .int(3))
#expect(service.pairs.map(\.key) == ["enabled", "retries", "name"])
#expect(service["enabled"] == .bool(true))
#expect(service["retries"] == .int(3))
#expect(service["name"] == .string("API"))
#expect(service["enabled"] == nil)
#expect(service["retries"] == nil)
#expect(service["<<"] == nil)
#expect(root?["defaults"] != nil)
#expect(root?["enabled"] == nil)
#expect(root?["missing"] == nil)
}

View File

@ -132,28 +132,6 @@ enum UnsupportedYAMLGapsFixtures {
])),
])),
),
FallbackValueCase(
name: "merge key remains ordinary mapping entry",
yaml: """
defaults: &defaults {enabled: true, retries: 3}
service:
<<: *defaults
name: API
""",
expected: .mapping(.init([
.init(key: "defaults", value: .mapping(.init([
.init(key: "enabled", value: .bool(true)),
.init(key: "retries", value: .int(3)),
]))),
.init(key: "service", value: .mapping(.init([
.init(key: "<<", value: .mapping(.init([
.init(key: "enabled", value: .bool(true)),
.init(key: "retries", value: .int(3)),
]))),
.init(key: "name", value: .string("API")),
]))),
])),
),
FallbackValueCase(
name: "comments after explicit document end stay non-content",
yaml: """

View File

@ -196,24 +196,29 @@ let validationCorpusIssueFixtures: [ValidationCorpusIssueFixture] = [
],
),
.init(
name: "merge-like duplicate keys remain inspectable until merge support exists",
name: "merged mappings preserve duplicate local keys for validation",
source: .yaml("""
defaults: &defaults {retries: 1}
service: {<<: *defaults, <<: {retries: 2}}
service: {<<: *defaults, name: API, name: Backend}
"""),
validator: .init(),
expectedIssues: [
.init(
severity: .error,
reason: "Duplicate mapping key '<<'",
path: .init([.key("service"), .key("<<")]),
reason: "Duplicate mapping key 'name'",
path: .init([.key("service"), .key("name")]),
),
],
forbiddenIssues: [
.init(
severity: .error,
reason: "Duplicate mapping key 'retries'",
path: .init([.key("service"), .key("<<"), .key("retries")]),
path: .init([.key("service"), .key("retries")]),
),
.init(
severity: .error,
reason: "Duplicate mapping key '<<'",
path: .init([.key("service"), .key("<<")]),
),
],
),

View File

@ -93,6 +93,7 @@ The initial parser supports:
- flow sequences and mappings
- literal and folded block scalars
- anchors and aliases
- merge-key expansion for mappings and sequences of mappings
- YAML directives, document markers, and multi-document streams
- explicit built-in scalar tags for strings, integers, floats, booleans, and
nulls
@ -157,10 +158,11 @@ must stay in `PureYAML.Model.Value`; known YAML should use typed `Codable`.
This is a deterministic validation decision. A Swift dictionary-shaped value
cannot preserve all YAML mapping entries when keys repeat, cannot represent
complex mapping keys with the current public model, and makes merge/tag
normalization ambiguous. The research constructor also includes
Foundation-specific scalar conversions, which would break the dependency-free
and WASM-compatible library boundary.
complex mapping keys with the current public model, and makes tag normalization
ambiguous. Merge keys are handled explicitly by the parser instead of being
hidden inside an arbitrary-value projection. The research constructor also
includes Foundation-specific scalar conversions, which would break the
dependency-free and WASM-compatible library boundary.
Application code may project `Model.Value` into JSON-like dictionaries after
validation, but that projection must own the data-loss policy explicitly.
@ -169,11 +171,12 @@ validation, but that projection must own the data-loss policy explicitly.
Compatibility should be added in small, test-backed slices:
1. Tag-aware collection handling and merge keys.
1. Tag-aware collection handling.
2. Multi-document stream dumping.
3. Additional built-in validation rules beyond duplicate-key behavior.
4. Broader Codable compatibility beyond scalar, keyed, and unkeyed containers.
5. Yams corpus comparison tests in a separate compatibility suite.
3. First-class complex mapping keys.
4. Additional built-in validation rules beyond duplicate-key behavior.
5. Broader Codable compatibility beyond scalar, keyed, and unkeyed containers.
6. Yams corpus comparison tests in a separate compatibility suite.
The private `PureYAMLResearch` repository may be used to study Yams behavior, but
the public implementation must be written in Swift and must not copy C parser

View File

@ -40,8 +40,8 @@ but its observable shape conflicts with PureYAML's first principles:
- mapping conversion becomes dictionary-shaped, so duplicate keys and insertion
order are not a first-class contract;
- merge keys may be flattened during construction instead of remaining visible
to validation;
- merge-key behavior can be hidden inside construction rather than exposed as an
explicit parser contract;
- scalar construction includes Foundation-specific values such as binary data,
dates, and null objects;
- custom constructors can hide whether a document was parsed, normalized,
@ -67,6 +67,7 @@ add a fixture beside the closest existing suite.
| Flow collections | Flow sequences and flow mappings compose into model values. | `ParsingTests`, `CollectionCompatibilityFixtures` |
| Literal and folded blocks | Block scalar text is preserved according to current parser rules. | `ParsingTests`, `DumperTests` |
| Anchors and aliases | Scalar and mapping anchors can be reused through aliases. Undefined aliases throw exact parse errors. | `ParsingTests`, `CollectionCompatibilityFixtures` |
| Merge keys | Plain `<<` and explicit `!!merge` keys expand mapping values and sequence-of-mapping values. Local keys override inherited keys; duplicate local keys remain visible to validation. | `MergeKeyCompatibilityTests`, `ParsingCompatibilityTests`, `DownstreamDocumentTests` |
| YAML directives and document markers | `%YAML 1.2`, selected `%TAG` expansion, `---`, and `...` are supported. | `ParsingCompatibilityTests`, `StreamParsingTests` |
| Multi-document streams | `PureYAML.parseStream(_:)` parses implicit and explicit documents, empty explicit documents as `null`, `...` document ends, and trailing comments before the next `---`. | `StreamParsingTests` |
| Stream validation | Stream validation preserves document indexes while keeping validation paths document-local. | `StreamParsingTests` |
@ -90,7 +91,7 @@ explicit fallback value tree. This avoids silent compatibility drift.
| `%TAG` after content or after document start | Throws `unsupportedDirective`. | `UnsupportedYAMLGapsFixtures.parseErrors` |
| Undefined aliases | Throws `undefinedAlias`. | `CollectionCompatibilityFixtures.parseErrors` |
| Invalid explicit scalar tag values | Throws `invalidTaggedScalar`. | `ScalarCompatibilityFixtures.invalidExplicitScalarTags` |
| Merge keys | Keeps `<<` as an ordinary mapping entry; it does not flatten merged keys. | `UnsupportedYAMLGapsFixtures.fallbackValues`, `DownstreamDocumentTests` |
| Invalid merge values | Throws `invalidMergeValue` when `<<` is not followed by a mapping or sequence of mappings. | `MergeKeyCompatibilityTests` |
| Unsupported built-in tags such as `!!timestamp`, `!!binary`, `!!set`, and `!!omap` | Keeps parseable value trees without special tag semantics. | `UnsupportedYAMLGapsFixtures.fallbackValues` |
| Sexagesimal-style scalars and date-looking plain scalars | Keep string values for currently unsupported spellings such as `1:20`, `2002-04-28`, and `09`. | `ScalarCompatibilityFixtures.unsupportedScalars` |
| Custom constructors, custom resolvers, and environment expansion | Not implemented. Build project-specific conversion on top of `Model.Value` or typed `Codable`. | Planned future work |
@ -113,7 +114,6 @@ explicit fallback value tree. This avoids silent compatibility drift.
Do not migrate a caller yet if it requires:
- multi-document stream dumping;
- YAML merge-key flattening;
- complex mapping keys as first-class keys;
- binary or timestamp conversion into Foundation types;
- arbitrary `Any` load and dump APIs;

View File

@ -91,10 +91,16 @@ title: First
explicit document, including an explicit empty document, and throws
`unsupportedMultiDocumentStream` when a second document is present.
Merge keys are expanded during parsing. Plain `<<` and explicit `!!merge` keys
inherit mapping entries from a mapping or sequence of mappings; local keys
override inherited keys, and duplicate local keys remain visible to validation.
Quoted `"<<":` and explicitly string-tagged `!!str <<` keys remain ordinary
string keys.
If a caller needs a JSON-like dictionary after validation, build that projection
in application code after deciding how duplicate keys, unsupported tags, merge
keys, and complex keys should behave. Keeping that step outside PureYAML makes
the lossy boundary explicit.
in application code after deciding how duplicate keys, unsupported tags, and
complex keys should behave. Keeping that step outside PureYAML makes the lossy
boundary explicit.
## Typed Conversion