Mihaela Mihaljevic
43b20ab7bc
Stage 3 of the faithful libyaml scanner port: the token producer. Translates scan_to_next_token, fetch_more_tokens/fetch_next_token dispatch, and the fetchers for stream start/end, document indicators, block entry, key, value (with the simple-key splice and roll-indent), plus scan_plain_scalar with its line-break folding. Reuses the ported indent engine, simple-key mechanism, character classes, and reader. Constructs not yet ported (flow collections, quoted scalars, anchors, tags, directives, block scalars) throw an explicit 'not yet ported' error rather than mis-scan; the port is not yet wired to the public API. Milestone: the ported scanner tokenizes '- a:' followed by a same-column 'b:' as siblings (one BLOCK-MAPPING-START), matching libyaml, fixing the original block-mapping bug by construction; a more-indented value still nests. Unit-tested for the sibling case, the nested case, a simple pair, and multi-line plain-scalar folding. 305 tests, no regression.
19 lines
614 B
Swift
19 lines
614 B
Swift
extension PureYAML.Parsing.LibYAML {
|
|
/// A scanner token kind, mirroring `yaml_token_type_t`. The flow-collection,
|
|
/// anchor, tag, alias, and directive kinds are added as their fetchers are
|
|
/// ported; the block + plain-scalar core is present.
|
|
enum TokenType: Equatable {
|
|
case streamStart
|
|
case streamEnd
|
|
case documentStart
|
|
case documentEnd
|
|
case blockSequenceStart
|
|
case blockMappingStart
|
|
case blockEnd
|
|
case blockEntry
|
|
case key
|
|
case value
|
|
case scalar(value: String, style: PureYAML.Parsing.ScalarStyle)
|
|
}
|
|
}
|