Merge conformance/yaml-test-suite-gaps: the libyaml port becomes the engine on main
The ported engine wins every conflict: main's post-fork patches to the hand-rolled Scanner and TokenEventParser land on deleted files and are dropped; their behaviors are carried by the port and guarded by the whole-suite conformance gate (402/402 passing). Merge adaptations, all gates green (style, namespacing, forbidden patterns, changelog, roadmap, swiftformat, swiftlint strict, build, full test suite, whole-suite conformance sweep): - PureYAMLJSON representability rules move onto the Check algebra, replacing the retired legacy Rule type. - The permissive tag-handle pre-pass is removed; the port rejects an undefined handle at parse time (suite case QLJ7) and the validation report carries it as a parse diagnostic. - EventStreamComposer's nested types split one-per-file per the namespacing gate. - Tests pinned to old-engine behavior were re-verified against the reference and updated: libyaml rejects an explicit-key continuation ending at a value colon, and suite case QB6E requires rejecting a quoted continuation at column 0. - CHANGELOG unreleased entries from both lines are preserved in two groups with a provenance note. Closes #1, closes #3
This commit is contained in:
commit
a8950e118c
1
.claude/commands/exp
Symbolic link
1
.claude/commands/exp
Symbolic link
@ -0,0 +1 @@
|
||||
../../../../private/mihaela-agents/skills/exp
|
||||
1
.claude/commands/openapi
Symbolic link
1
.claude/commands/openapi
Symbolic link
@ -0,0 +1 @@
|
||||
../../../../private/mihaela-agents/skills/openapi
|
||||
1
.claude/commands/swift
Symbolic link
1
.claude/commands/swift
Symbolic link
@ -0,0 +1 @@
|
||||
../../../../private/mihaela-agents/skills/swift
|
||||
1
.claude/commands/ui
Symbolic link
1
.claude/commands/ui
Symbolic link
@ -0,0 +1 @@
|
||||
../../../../private/mihaela-agents/skills/ui
|
||||
@ -67,6 +67,7 @@ SKIP_PATTERNS=(
|
||||
# Vendored upstream rules canon (mirror of rules-swift). Its enforcement docs
|
||||
# legitimately quote the forbidden phrases. See docs/rules-swift/PROVENANCE.md.
|
||||
'^docs/rules-swift/'
|
||||
'^Tests/Fixtures/yaml-test-suite/'
|
||||
# Rule files that DEFINE the prohibitions; they MUST be able to cite
|
||||
# the forbidden strings to document them.
|
||||
'/github-discipline\.md$'
|
||||
|
||||
4
.github/ISSUE_TEMPLATE/config.yml
vendored
4
.github/ISSUE_TEMPLATE/config.yml
vendored
@ -1,8 +1,8 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Questions and ideas
|
||||
url: https://github.com/mihaelamj/PureYAML/discussions
|
||||
url: https://codeberg.org/Mihaela/PureYAML/discussions
|
||||
about: Ask questions, float ideas, or discuss design in Discussions.
|
||||
- name: Security issue
|
||||
url: https://github.com/mihaelamj/PureYAML/blob/main/SECURITY.md
|
||||
url: https://codeberg.org/Mihaela/PureYAML/blob/main/SECURITY.md
|
||||
about: Do not file security issues publicly. Follow the security policy instead.
|
||||
|
||||
@ -22,3 +22,14 @@ file is kept to the standard MIT text so it is correctly detected as MIT.
|
||||
The private `PureYAMLResearch` repository contains a full Yams source snapshot
|
||||
for study and compatibility research. That repository is not the implementation
|
||||
source for PureYAML.
|
||||
|
||||
## Vendored test data
|
||||
|
||||
`Tests/Fixtures/yaml-test-suite/` contains input cases vendored from the official
|
||||
[YAML test suite](https://github.com/yaml/yaml-test-suite) (`data` branch, commit
|
||||
`6ad3d2c62885d82fc349026c136ef560838fdf3d`), which is distributed under the MIT
|
||||
license. All 402 cases (numbered sub-cases counted) are vendored, each as its input
|
||||
(`in.yaml`), expected JSON projection (`in.json`), and/or expected-error marker
|
||||
(`error`). `Tests/Fixtures/yaml-test-suite-manifest.yaml` records the provenance and
|
||||
each case's conformance status. These files are test data, not implementation
|
||||
source.
|
||||
|
||||
447
CHANGELOG.md
447
CHANGELOG.md
@ -7,6 +7,453 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
|
||||
- Merge the libyaml-port line into main. The ported engine is the sole parser;
|
||||
the hand-rolled `Scanner`/`TokenEventParser` files and their post-fork
|
||||
patches are removed. Adaptations made in the merge: the JSON-representability
|
||||
rules move from the retired legacy `Rule` type onto the Check algebra
|
||||
(`BuiltinCheck.mappingKeysAreStrings`, `BuiltinCheck.numbersAreFinite` in
|
||||
`PureYAMLJSON`); the permissive tag-handle pre-pass
|
||||
(`Scanner.tagHandleIssues`) is removed because the ported engine rejects an
|
||||
undefined handle at parse time (official suite case QLJ7), so the validation
|
||||
report carries it as a parse diagnostic; and tests pinned to old-engine
|
||||
behavior or error text were updated to the reference-verified behavior
|
||||
(libyaml rejects an explicit-key continuation ending at a value colon, and
|
||||
suite case QB6E requires rejecting a quoted continuation at column 0).
|
||||
|
||||
The unreleased entries below come from two parallel lines of work, merged
|
||||
in one step: the libyaml-port line (listed first), whose ported engine is
|
||||
now the sole parsing engine, and the earlier hand-rolled-engine hardening
|
||||
line (listed second). Engine-internal file references in the second group
|
||||
are historical; the behaviors they describe are carried by the port and
|
||||
guarded by the whole-suite conformance gate.
|
||||
|
||||
### Changed
|
||||
|
||||
- Refresh `docs/YAML_TEST_SUITE_CONFORMANCE.md` to match the manifest: the
|
||||
headline is 402 passing, 0 gap; the "remaining gaps" narrative is retired; a
|
||||
closing section records the libyaml-port cutover and the final fix series,
|
||||
and the manifest is named as the source of truth over the prose.
|
||||
|
||||
### Added
|
||||
|
||||
- The located parse now reports an exact end position for scalars, not just a start: the
|
||||
scalar event carries the scanner token's end mark, so a `LocatedValue.scalar`'s range covers
|
||||
the whole token (end exclusive, one past the last character) instead of being start-anchored.
|
||||
Collections already spanned open to close. The event dump compares value and style, not marks,
|
||||
so conformance is unaffected; the whole-suite gates stay green.
|
||||
- Add an opt-in located parse, `PureYAML.parseLocated(_:)` and
|
||||
`PureYAML.parseStreamLocated(_:)`, returning a `Model.LocatedValue` tree parallel
|
||||
in shape to `Value` where every node carries its source span (1-based line and
|
||||
column) and located children. Collections span from their opening to closing
|
||||
event, scalars carry a start-anchored span, and aliases and merge-inherited pairs
|
||||
point at their reference site. The plain `Value`, its equality, and the
|
||||
conformance oracle are untouched: locations are built only when the streaming
|
||||
composer is asked for them, so the default value path is byte-identical. A
|
||||
whole-suite test proves the located projection equals a plain parse for every
|
||||
passing case, including through merge keys and aliases.
|
||||
- Begin a faithful Swift port of libyaml's scanner under
|
||||
`PureYAML.Parsing.LibYAML`, to replace the hand-rolled approximation whose
|
||||
indentation and simple-key bugs the test suite surfaced. The first piece is the
|
||||
indentation engine, a line-for-line translation of `yaml_parser_roll_indent` and
|
||||
`yaml_parser_unroll_indent`: a block collection opens only when content is
|
||||
strictly more indented than the current level, which fixes the `- a:` followed
|
||||
by a same-column `b:` mis-nesting by construction. The port is verified against
|
||||
libyaml (built from the `CYaml` source) and lands as a parallel implementation,
|
||||
swapped in only when it matches the reference with no regression. See
|
||||
`docs/LIBYAML_PORT.md`. No public behavior changed yet.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Reject an under-indented or tab-indented continuation in a multi-line quoted
|
||||
scalar. A quoted scalar's continuation must be indented past the parent block, so
|
||||
`"a`\n`b"` (continuation in column 0) and `"bar`\n`\tbaz"` (tab indentation) are
|
||||
malformed. The error is deferred until the closing quote is found, so an
|
||||
unterminated quote still reports the missing quote (its root cause) rather than
|
||||
the indentation symptom. Surpasses libyaml on QB6E and DK95/01, bringing
|
||||
whole-suite conformance to 402/402.
|
||||
- Reject a tab used to indent flow-collection content. A tab is never indentation,
|
||||
so flow content reached through a tab below the continuation indent (`- [`\n
|
||||
`\tfoo`) is malformed, even though the tab inflates the column. A tab at or past
|
||||
the indent stays separation, and a tab-only blank line stays blank. Surpasses
|
||||
libyaml on Y79Y/003.
|
||||
- Reject a block scalar whose leading empty line is more indented than its first
|
||||
content line. libyaml auto-detects the deeper indentation and silently drops the
|
||||
content; the auto-detection now errors when the content line is less indented
|
||||
than an earlier empty line. Surpasses libyaml on S98Z.
|
||||
- Reject a multi-line implicit key in a flow sequence. A flow-sequence single-pair
|
||||
implicit key is single-line (`c-s-implicit-json-key`), so its `:` must sit on the
|
||||
key's line: both `[ "key"\n :value ]` and `[ key\n : value ]` are malformed. A
|
||||
flow MAPPING adjacent value may still span lines (`{ "foo"\n :bar }` stays
|
||||
valid), so the rejection is scoped to flow levels that are sequences, tracked
|
||||
alongside the flow-level stack. The key/`:` line comparison catches the plain-key
|
||||
form too, which folds its line break. Surpasses libyaml on ZXT5 and DK4H.
|
||||
- Reject a block sequence that opens on the same line as an earlier node. A `-`
|
||||
entry mid-line after an inline mapping value (`key: - a`) or a node property
|
||||
(`&anchor - x`) is invalid, while a `-` after another `-` stays a valid compact
|
||||
sequence (`- - a`) and an explicit key's value on a later line (`? k`\n`: - a`)
|
||||
stays valid. The inline case is told apart by a pending simple key (an explicit
|
||||
key's has gone stale across the line break). Surpasses libyaml on 5U3A and SY6V.
|
||||
- Reject a flow-collection continuation line that is not indented past the parent
|
||||
block. YAML requires multi-line flow content to be more indented than the
|
||||
enclosing block context, so `flow: [a,` / `b]` (with `b` in column 0) and the
|
||||
equivalent multi-line flow mapping are malformed; libyaml does not enforce it. A
|
||||
closing `]`/`}` or a leading `,` may still sit at any indentation. Surpasses
|
||||
libyaml on 9C9N and VJP3/00.
|
||||
- Reject content after a `...` document-end marker on the same line. Only
|
||||
whitespace and a comment may follow `...`; unlike `--- x`, a `... x` cannot
|
||||
carry inline content. Surpasses libyaml on 3HFZ.
|
||||
- Resolve a scalar with the non-specific `!` tag as a string. The `!` tag forces
|
||||
resolution by kind alone, with no implicit typing, so `! 12` is the string "12"
|
||||
rather than the integer 12. The typed value now honours it (the event stream
|
||||
already did). Surpasses libyaml on S4JQ.
|
||||
- Reject a bare `-` before a flow indicator inside a flow collection. Per
|
||||
`ns-plain-first` a `-` begins a plain scalar in flow only when a flow-safe
|
||||
character follows; `[-]` and `[-, -]` are malformed, not a sequence of `-`
|
||||
scalars. libyaml reads the `-` as a scalar. Surpasses libyaml on G5U8 and YJV2.
|
||||
- Reject a comment that is not preceded by whitespace. A `#` only begins a comment
|
||||
after a space, a tab, or at the start of a line; a `#` adjacent to the previous
|
||||
token (`key: "value"# c`, `[ a ]#c`, `[ a,#c`, a `>#c` block scalar header) is
|
||||
malformed. The check uses the character immediately before the `#`, since a plain
|
||||
scalar consumes its own trailing blanks. libyaml accepts the adjacent form.
|
||||
Surpasses libyaml on 9JBA, CVW2, SU5Z, and X4QW. Whole-suite conformance is now
|
||||
388/402.
|
||||
- Reject a directive that follows a document without an explicit `...` end marker.
|
||||
Directives apply to the document that follows, so the previous document must be
|
||||
closed with `...` before them; after an implicit end a `%YAML`/`%TAG`/reserved
|
||||
directive is now an error. Also reject a `%YAML` version followed immediately by
|
||||
`#` with no separating space (`%YAML 1.1#...`), since a comment requires leading
|
||||
separation. Restores reject-side conformance lost when the libyaml port became
|
||||
the sole engine (9HCY, EB22, MUS6/00, MUS6/01, RHX7).
|
||||
- Refresh `yaml-test-suite-manifest.yaml` (now version 3): the ported engine's
|
||||
whole-suite conformance is 384/402, up from the recorded 331, after the
|
||||
canonical-event and reject-side work. 53 cases the manifest still marked as gaps
|
||||
now pass.
|
||||
- Keep the final line break of a block scalar that ends at end of input without a
|
||||
trailing newline. A whitespace-only last line still counts as one line break
|
||||
under keep (`+`) chomping, so `- |+` followed by a spaces-only unterminated line
|
||||
yields `"\n"` rather than `""`. With this the ported engine emits the exact
|
||||
canonical event stream for every valid case in the vendored YAML test suite
|
||||
(308/308), surpassing libyaml (252) and the previous hand-rolled parser (267).
|
||||
Surpasses libyaml on JEF9/02.
|
||||
- End a root-level block scalar at a `---` or `...` document marker in column 0.
|
||||
Now that a root block scalar can take indentation 0, its content loop reaches
|
||||
column-0 lines, so a marker line must end the scalar instead of being folded
|
||||
into it. Without this a `--- |` literal swallowed the following `...`, the next
|
||||
directives, and later documents. Surpasses libyaml on W4TN.
|
||||
- Fold over a tab-only blank line inside a multi-line plain scalar. A plain
|
||||
scalar may span blank lines, and a blank line that contains a tab is still
|
||||
blank, so the tab is not indentation. libyaml rejects any tab below the scalar
|
||||
indentation; the scanner now rejects it only when it indents real content, not
|
||||
when the rest of the line is blank. Surpasses libyaml on DK95/04.
|
||||
- Accept a tab as in-line separation while still rejecting it as indentation. A
|
||||
tab after a `-`, `?`, or `:` indicator, or before a flow collection, is valid
|
||||
separation, but libyaml refuses to skip it whenever a simple key is allowed and
|
||||
reports "found character that cannot start any token". The scanner now skips
|
||||
tabs in the inter-token whitespace and records whether one preceded the token;
|
||||
a block indicator or an implicit mapping key reached through a tab is rejected
|
||||
(the tab would be indentation), while a scalar value or flow collection is
|
||||
accepted. So `- foo:\tbar`, `-\t-1`, `: -\tb`, `\t[ ]`, `\t{}`, and a
|
||||
space-then-tab continuation parse, while tab-indented keys and `-\t-` style
|
||||
nested entries are still rejected. Surpasses libyaml on 6BCT, 6CA3, A2M4,
|
||||
DK95/00, Q5MG, and Y79Y/010.
|
||||
- Parse a flow sequence entry that begins with `:` as a single-pair mapping with
|
||||
an empty key (`[ : v ]`). libyaml hands the leading VALUE token to its node
|
||||
parser and fails. The flow-sequence-entry parser now opens the implicit mapping
|
||||
on a leading `:` and leaves the indicator for the value state, so the value is
|
||||
parsed rather than dropped. Surpasses libyaml on CFD4.
|
||||
- Apply the YAML 1.2 flow `:` rule instead of libyaml's "always a value
|
||||
indicator in flow". In a flow collection a `:` only ends a node when it is
|
||||
followed by whitespace, the end of input, or a flow indicator, or when it is
|
||||
adjacent to a JSON key (a quoted scalar or a flow collection used as the key).
|
||||
Otherwise the `:` is plain-scalar content. So `[:x]` and `{x: :x}` keep the
|
||||
scalar `:x`, `http://foo.com` and `::vector` stay single scalars, `"key":value`
|
||||
and `{JSON: like}:adjacent` bind the value across the adjacent `:`, and
|
||||
`omitted value:,` is a key with an empty value rather than a scan error. The
|
||||
pending simple key now records whether it is a JSON node, and the plain scalar
|
||||
scanner ends (rather than rejecting) at a `:` before a flow indicator. Surpasses
|
||||
libyaml on 4ABK, 58MP, 5T43, DBG4, FRK4, and HM87/00, with 9MMW (mixed
|
||||
adjacent/separate keys) still correct.
|
||||
- Treat `?` in a flow collection as plain-scalar content unless it is followed by
|
||||
whitespace, the end of input, or a flow indicator. libyaml makes every `?` in
|
||||
flow context a mapping-key indicator, dropping the `?` from `?foo`, but
|
||||
`ns-plain-first` lets a `?` followed by a flow-safe character begin a plain
|
||||
scalar. So `{ ?foo: bar }` keys on `?foo` and `[?x]` is the scalar `?x` rather
|
||||
than an empty single-pair mapping. Surpasses libyaml on 652Z and HM87/01.
|
||||
- Accept a tab as block-scalar content once the indentation is satisfied. The
|
||||
block-scalar break scanner rejected any tab seen while auto-detecting the
|
||||
content indentation, but a tab at or beyond the content column is content, not
|
||||
indentation. The tab is now an error only below the content column (the parent
|
||||
indent + 1 floor while auto-detecting), so `foo: |-` / `bar` lines whose
|
||||
content begins with a tab parse, while a tab used as indentation (a tab-only
|
||||
line shallower than the content) is still rejected. Surpasses libyaml on
|
||||
96NN/00, 96NN/01, R4YG, and Y79Y/001.
|
||||
- Auto-detect indentation 0 for a root-level block scalar. The block-scalar
|
||||
scanner used 0 as the "indentation not yet detected" sentinel and floored the
|
||||
detected indent at 1 (as libyaml does in `scan_block_scalar_breaks`), so a
|
||||
`--- >` or `--- |` whose content begins in column 0 detected no content and the
|
||||
following line was misread as a new document ("did not find expected document
|
||||
start"). The sentinel is now -1, leaving 0 a representable detected indent, and
|
||||
the floor is the parent indent + 1 (0 at the document root). Surpasses libyaml
|
||||
on DK3J, FP8R, and M7A3.
|
||||
- Parse block mappings with empty (null) keys. A `:` value indicator with no
|
||||
preceding key on its line denotes a mapping entry whose key is empty, which
|
||||
YAML 1.2 permits but libyaml rejects with "did not find expected key". The
|
||||
block-mapping-key parser now emits the empty scalar key and lets the value
|
||||
state consume the indicator, fixing the canonical event streams for `: a`
|
||||
style mappings, explicit empty keys (`? &a a` / `: *a`), empty keys nested in
|
||||
sequences (`- :`, `- ? : x`), and mixed empty-key/value entries. Surpasses
|
||||
libyaml on these seven test-suite cases (2JQS, 6M2F, M2N8/00, NHX8, S3PD,
|
||||
SM9W/01, UKK6/00).
|
||||
- Parse flow mappings with empty (null) keys. The same `:`-without-a-key entry
|
||||
is valid inside `{ ... }`, where libyaml hands the bare VALUE token to its node
|
||||
parser and fails with "did not find expected node content". The flow-mapping-key
|
||||
parser now emits the empty scalar key for a leading `:`, so `{ : }` and
|
||||
`{ key: value, : empty key }` parse, completing the multi-document NKF9 case.
|
||||
- Emit no document for a stream whose only content is comments or whitespace. A
|
||||
comment-only or empty stream has zero documents in YAML 1.2 (matching libyaml
|
||||
and the test suite), but the event stream was emitting a spurious empty null
|
||||
document. Found by differential testing against libyaml over real-world YAML
|
||||
(a comment-only `values.yaml`) in addition to the canonical event gate. The
|
||||
single-document value API still maps an empty stream to `null`, so
|
||||
`PureYAML.parse("")` is unchanged. Fixes 8G76, 98YD, and AVM7, raising exact
|
||||
canonical-event conformance to 267 of 308.
|
||||
|
||||
- Strip the trailing folded line break from a flow-context plain scalar. A plain
|
||||
scalar inside `[...]`/`{...}` whose next line begins with a flow indicator
|
||||
(`,` `]` `}`) or the `:` value indicator was retaining the line break folded to
|
||||
a trailing space (for example `[\nfoo: bar\n]` produced the value `"bar "`),
|
||||
which YAML 1.2 section 7.3.3 forbids. Derived from the plain-scalar production
|
||||
and verified against the canonical event gate: fixes 10 official YAML test suite
|
||||
cases (4FJ6, 4MUZ/02, CT4Q, LP6E, NKF9, QF4Y, VJP3/01, Y79Y/002, ZF4X, ZK9H)
|
||||
with no regression, raising exact canonical-event conformance to 264 of 308.
|
||||
|
||||
### Added
|
||||
|
||||
- Add an authoritative canonical-event conformance gate. The parser's event
|
||||
stream is serialized into the official YAML test suite event (yeast) format and
|
||||
compared exactly to each case's vendored `test.event`, which checks anchors,
|
||||
tags, scalar styles, flow vs block, and explicit document markers for every
|
||||
valid case (including the roughly half with no `in.json` projection). This
|
||||
replaces the JSON-projection oracle, which was lossy: measured against ground
|
||||
truth, 254 of 308 valid suite cases emit exactly correct events (40 emit a
|
||||
wrong event stream, 14 false-reject), where the JSON proxy had reported a higher
|
||||
figure. The gate records the gap set and fails on any regression or unrecorded
|
||||
fix. All 402 cases' `test.event` files are vendored alongside their inputs.
|
||||
|
||||
### Changed
|
||||
|
||||
- Record whether a document boundary was explicit in the parser `Event` model
|
||||
(`documentStart`/`documentEnd` now carry an `explicit` flag), so a `---`/`...`
|
||||
marker is distinguished from an implicit boundary. This is required for exact
|
||||
canonical-event comparison and resolves 89 cases that were previously correct
|
||||
but unverifiable.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Make the YAML test suite conformance gate comprehensive and repair the
|
||||
regressions it revealed. The gap-only manifest (which deleted a case once fixed,
|
||||
so nothing guarded it afterward and later work could silently regress it) is
|
||||
replaced by a full manifest of all 402 official cases
|
||||
(`Tests/Fixtures/yaml-test-suite-manifest.yaml`), each with its `id`, `name`,
|
||||
`localPath`, conformant `expected` behavior, `oracle`, `byteCount`, `lineCount`,
|
||||
and a `status` (`passing` | `gap`). Every case's files are vendored under
|
||||
`Tests/Fixtures/yaml-test-suite/`. The gated sweep
|
||||
(`PUREYAML_RUN_YAML_TEST_SUITE=1`) now asserts every `passing` case still passes
|
||||
and every `gap` still reproduces, so a regression in any passing case fails the
|
||||
sweep. Result at the pinned commit: **329 of 402 passing, 73 gap.**
|
||||
- Stop over-rejecting tabs. A tab is invalid only as block-structure indentation
|
||||
(`scanner.c` `scan_to_next_token` / `scan_block_scalar_breaks` /
|
||||
`scan_plain_scalar`). Block indentation is now measured in spaces only; a tab is
|
||||
left for the token scanner, which allows it inside block-scalar content once a
|
||||
leading space fixes the content column (96NN/00, 96NN/01, R4YG, Y79Y/001), as
|
||||
separation before a scalar value (DK95/00) or a flow node (6CA3, Q5MG, J3BT), and
|
||||
on a whitespace-only blank line (DK95/03, DK95/07), while still rejecting genuine
|
||||
tab indentation (4EJS, Y79Y/000, Y79Y/004-008, DK95/06). The batch scanner and the
|
||||
incremental token cursor were unified onto a single `scanStep`, since their
|
||||
divergence previously let the streaming parser miss a tab error the batch scanner
|
||||
caught.
|
||||
- Restore anchors/aliases as mapping keys for `26DV`: an anchor on a `key:` value
|
||||
line followed by an indented `*alias :` key now anchors the nested mapping rather
|
||||
than the alias key (which cannot carry properties); `&b *alias` on one line is
|
||||
still rejected (SU74).
|
||||
- Restore flow bare-tag/anchor keys for `WZ62` (`{ foo : !!str, !!str : bar, }` to
|
||||
`{"foo":"", "":"bar"}`): a flow key introduced by a spliced `mappingKey` uses the
|
||||
same key parser as the implicit form, so a bare tag before the `:` composes an
|
||||
empty tagged scalar key instead of a mapping-as-key.
|
||||
- Complete flow-collection (`[...]`, `{...}`) parsing for keys, empty nodes, and
|
||||
multi-line scalars, fixing 10 official YAML test suite cases and removing them
|
||||
from the conformance gap manifest. The scanner now implements libyaml's
|
||||
simple-key mechanism inside flow context: a potential key position is recorded
|
||||
before a plain or quoted scalar, an anchor or tag, or a nested `[`/`{` opener,
|
||||
and a later `:` value indicator splices a `mappingKey` token in front of that
|
||||
node so a collection or property-carrying node can be a flow-mapping key. In
|
||||
flow context a `:` is now always a value indicator and a `?` always an explicit
|
||||
key indicator regardless of the following character (matching libyaml's
|
||||
`flow_level || IS_BLANKZ_AT` checks), so `"k":v`, `{k}:v`, and a trailing `?`
|
||||
parse correctly; the implicit-key forward lookahead is restricted to block
|
||||
context. A multi-line plain key folds in full before its `:` is read. The flow
|
||||
parser was rebuilt around libyaml's flow productions (`flow_sequence_entry`,
|
||||
`flow_mapping_key`/`value`), composing empty scalar keys and values where
|
||||
absent. The streaming token cursor holds an open flow collection's tokens until
|
||||
it closes so the retroactive key splice stays valid. Fixed false-rejects 4FJ6,
|
||||
8KB6, 9BXH, 9MMW, CN3R, DFF7, FRK4, NJ66, UT92 and corrected values for CT4Q.
|
||||
The flow-node-as-block-mapping-key cases LX3P and 6BFJ remain tracked (they need
|
||||
the block-context simple-key splice, a separate change).
|
||||
|
||||
- Parse multi-line plain scalars and block scalars at the document root, fixing 35
|
||||
official YAML test suite cases and removing them from the conformance gap
|
||||
manifest. The scanner now tracks the enclosing block-collection indent
|
||||
(libyaml's `parser->indent`) separately from per-line indentation, defaulting to
|
||||
-1 at the bare document root, so a plain scalar's continuation lines fold using
|
||||
the spec rule "more indented than the enclosing block": a root scalar folds its
|
||||
column-0 continuations (single break to space, blank line to newline, per-line
|
||||
whitespace stripped) while a scalar inside a collection still ends at a sibling
|
||||
line. A `---` document-start marker may now be followed by the document's root
|
||||
node on the same line (`--- foo`, `--- [1, 2]`, `--- |`, `--- >`), and a block
|
||||
scalar whose body sits at the document root's column 0 captures that body and
|
||||
ends at the next `---`/`...` marker. Fixed false-rejects 35KP, 4CQQ, 4Q9F, 6JQW,
|
||||
82AN, 93WF, 96L6, 9MQT/00, 9YRD, B3HG, DK3J, EX5H, EXG3, FP8R, RZT7, T26H, UGM3,
|
||||
XLQ9; corrected values for 27NA, 2G84/02, 2G84/03, 4WA9, 6FWR, 6LVF, A2M4, K54U,
|
||||
KSS4, L383, T5N4, W4TN; and started correctly rejecting the invalid 9KBC, ZVH3,
|
||||
9MQT/01. The still-imperfect M7A3 (multi-document `...` splitting) and NB6Z (tabs
|
||||
on empty plain-scalar lines) now parse but with mismatched values and stay
|
||||
tracked. Added root multi-line plain scalar regression tests.
|
||||
|
||||
- Escape carriage returns and line feeds in double-quoted output per unicode
|
||||
scalar rather than per Character. A CR+LF pair forms a single Swift extended
|
||||
grapheme cluster, so the previous Character-level escape switch matched neither
|
||||
`\r` nor `\n` and emitted the pair unescaped; the folding parser then read it
|
||||
back differently, breaking parse/dump/parse stability for values containing
|
||||
CRLF (observed on the Zoom OpenAPI corpus seed). Added a dumper regression test.
|
||||
|
||||
- Tightened error detection so 15 more official YAML test suite cases are handled
|
||||
correctly, repairing four error-detection regressions and removing them from the
|
||||
conformance gap manifest. Tabs are now rejected when they would supply
|
||||
block-structure indentation: a tab at the start of a block line, a tab in a
|
||||
block scalar's indentation, and a tab in separation position that precedes a
|
||||
nested block node (a `-` entry, a `?` key, or a compact mapping key) all fail,
|
||||
while a tab used purely as in-line separation before a scalar value stays valid
|
||||
(4EJS, Y79Y/000, Y79Y/004, Y79Y/005, Y79Y/006, Y79Y/008, Y79Y/009 now reject;
|
||||
6BCT and Y79Y/010 now parse with matching values; DC7X and J3BT no longer
|
||||
falsely reject). A `---`/`...` document marker at column 0 is now rejected when
|
||||
it appears inside an open flow collection (N782) or inside an open quoted scalar
|
||||
(RXY3, 5TRB) rather than silently ending the document. A second `%YAML`/`%TAG`
|
||||
directive section that opens a new document without an intervening `...` suffix
|
||||
is rejected (MUS6/01). An anchor or tag attached to an alias node, and an
|
||||
anchor-plus-alias on one node, are rejected (SU74, SR86). Remaining
|
||||
accepted-invalid cases that need `---`-line content scanning or full simple-key
|
||||
tracking (for example 2G84/00, 2G84/01, 9MQT/01, ZCZ6) are left tracked in the
|
||||
gap manifest.
|
||||
- Added YAML directive handling and completed document-boundary support, fixing
|
||||
three root causes. First, a `%` now begins a directive only in directive
|
||||
position (before any document content and before the `---` that starts a
|
||||
document, including immediately after a `...` end marker); everywhere else, and
|
||||
in any flow context, `%` is ordinary plain text rather than a misfiring
|
||||
directive. Second, `%YAML` is parsed leniently per the spec: a `1.x` version is
|
||||
accepted (a higher minor is treated as 1.2 with a warning rather than an
|
||||
error), while an unsupported major version, a duplicate `%YAML` in one
|
||||
document, or extra non-comment fields are rejected; reserved and unknown
|
||||
directives (and `%TAG` handle definitions) are accepted and ignored, and a
|
||||
directive must be followed by a `---` document start. Third, a `...` end marker
|
||||
now closes the current document so that following content begins a fresh
|
||||
document in the multi-document stream parser (`parseStream`), while the
|
||||
single-document `parse` still rejects multi-document input. This makes 11
|
||||
official YAML test suite cases parse with matching values (2LFX, 5TYM, 6WLZ,
|
||||
6ZKB, 7Z25, 9DXL, 9WXW, BEC7, MUS6/03, MUS6/05, MUS6/06), correctly rejects
|
||||
four previously accepted-invalid cases (9MMA, B63P, H7TQ, SF5V), and unblocks
|
||||
6LVF and W4TN from parse errors to value comparisons that now depend on
|
||||
separate scalar gaps (`---`-line content and literal block chomping).
|
||||
- Rewrote literal (`|`) and folded (`>`) block scalar content handling to follow
|
||||
YAML 1.2 §8.1 semantics, fixing four root causes: interior blank lines are no
|
||||
longer dropped from literal scalars; folded scalars now apply true §8.1.3 line
|
||||
folding (a single break between equally indented non-empty lines folds to a
|
||||
space, blank lines stay literal newlines, and more-indented lines keep both
|
||||
their leading whitespace and their surrounding breaks unfolded); tabs inside
|
||||
block scalar content are preserved instead of being rewritten as spaces (only
|
||||
the block indentation columns are stripped); and the `+` keep chomping
|
||||
indicator is now supported alongside clip and strip. The block scalar is now
|
||||
scanned as a single unit (header line plus all content lines) with indentation
|
||||
auto-detection, mirroring the reference scanner, rather than reassembled from a
|
||||
lossy per-line token array. This makes 24 additional official YAML test suite
|
||||
cases pass: 4QFQ, 5LLU, 6VJK, 7T8X, 96NN/00, 96NN/01, DWX9, F6MC, F8F9, H2RW,
|
||||
JEF9/00, JEF9/01, K527, K858, L24T/00, L24T/01, M29M, M9B4, MJS9, P2AD, R4YG,
|
||||
TS54, W9L4, and Y79Y/001.
|
||||
- Implemented YAML 1.2 flow (quoted) scalar line folding for both single (`'`)
|
||||
and double (`"`) quoted scalars, mirroring the reference scanner's
|
||||
`yaml_parser_scan_flow_scalar`. Multi-line quoted scalars now fold correctly:
|
||||
trailing whitespace before a break is stripped, leading whitespace on a
|
||||
continuation line is stripped, a single break between non-empty lines folds to
|
||||
one space, and N consecutive breaks yield N-1 literal line feeds (the first and
|
||||
last lines keep their own leading and trailing spaces as content). A double
|
||||
quoted backslash at end of line now suppresses the break entirely (no space, no
|
||||
newline). Completed the double quoted escape set with `\xXX` (2 hex digits) and
|
||||
`\UXXXXXXXX` (8 hex digits), and added the previously missing `\0 \a \v \e`,
|
||||
literal `\<space>`, and the named breaks `\N \_ \L \P`, alongside the existing
|
||||
`\uXXXX` (including surrogate pairs). This makes 30 additional official YAML
|
||||
test suite cases pass: 3RLN/00, 3RLN/01, 3RLN/02, 3RLN/03, 3RLN/04, 3RLN/05,
|
||||
4ZYM, 565N, 5GBF, 6WPF, 7A4E, 8UDB, 9SA2, 9TFX, DE56/00, DE56/01, DE56/02,
|
||||
DE56/03, DE56/04, DE56/05, DK95/02, DK95/08, G4RS, NAT4, NP9H, PRH3, Q8AD,
|
||||
T4YY, TL85, and XV9V.
|
||||
- Composed an empty (null) plain scalar wherever a node is grammatically required
|
||||
but absent, instead of raising "expected a YAML node". This single production
|
||||
mirrors `parser.c`'s empty-scalar fallback and covers every position a node can
|
||||
go missing: after a tag or anchor property with no following node, an absent
|
||||
mapping value, an absent mapping key (both the plain `: value` form and the
|
||||
explicit `?` form), and the empty key and value forms inside flow collections.
|
||||
An empty node carrying an anchor or tag is explicit; a bare absent node with no
|
||||
property in a grammatically-required position is still a parse error, matching
|
||||
the reference parser. The empty scalar resolves through the normal scalar
|
||||
resolver against an empty value, so `!!str` yields `""`, `!!null` and a bare
|
||||
empty yield null, and an empty key resolves like any other empty scalar.
|
||||
Implemented in `Sources/Parsing/TokenEventParser.EmptyNode.swift` with the
|
||||
flow and block call sites in the sibling `TokenEventParser.*` files. This makes
|
||||
20 additional official YAML test suite cases pass: 2JQS, 2XXW, 5WE3, 6M2F,
|
||||
7W2P, BU8L, CFD4, JTV5, KK5P, LE5A, M2N8/00, NHX8, NKF9, PW8X, RZP5, S3PD,
|
||||
SM9W/01, UKK6/02, WZ62, and XW4D. CT4Q now parses but still mismatches its
|
||||
expected JSON (a separate flow multi-line plain-scalar folding gap) and is
|
||||
re-recorded as `json-mismatch`.
|
||||
- Accepted node properties (anchor, tag) and aliases in mapping-key position, so
|
||||
a mapping key is parsed as the full node it is: it may carry `&anchor` and/or
|
||||
`!tag` properties, and it may itself be an `*alias`. Fixed four root causes.
|
||||
(1) The scanner now terminates an anchor or alias name at the first character
|
||||
outside libyaml's `IS_ALPHA` set (ASCII letters, digits, `-`, `_`), mirroring
|
||||
`yaml_parser_scan_anchor`, so `*alias:` reads the alias `alias` followed by a
|
||||
`:` mapping indicator instead of swallowing the colon into the name. (2) The
|
||||
block-mapping parser now begins a pair at leading properties or an alias, not
|
||||
only at the key indicator the scanner emits after them, and properties on the
|
||||
same line as the first key attach to that key node rather than to the mapping
|
||||
(`&a a: b` anchors the key `a`, while `&a` alone on its own line anchors the
|
||||
map). (3) An anchor or tag at value position with no node following on the same
|
||||
line now composes an empty (null) value that still carries the property, so
|
||||
`a: &anchor` registers `anchor` as null and a later `*anchor` resolves. (4) An
|
||||
anchored scalar now registers its resolved value (so an empty anchored scalar
|
||||
registers null, not `""`). The implicit-key scanner also now recognizes a key
|
||||
with trailing spaces before its `:` (`"top1" :`). Implemented in
|
||||
`Sources/Parsing/Scanner.swift`, `Scanner.Helpers.swift`,
|
||||
`TokenEventParser.swift`, `TokenEventParser.EmptyNode.swift`,
|
||||
`TokenEventParser.Helpers.swift`, `TokenEventParser.Block.swift`,
|
||||
`TokenEventParser.Flow.swift`, `TokenCursor.swift`, and the anchor registration
|
||||
in `EventStreamComposer.swift` and `EventComposer.swift`. This makes 10
|
||||
additional official YAML test suite cases pass: 26DV, 6KGN, 74H7, 87E4, 9KAX,
|
||||
E76Z, G9HC, HMQ5, LQZ7, and ZWK4 (G9HC is an invalid-anchor case that is now
|
||||
correctly rejected). 2SXE now parses but still mismatches its expected JSON (a
|
||||
separate compact block-mapping structural gap) and is re-recorded as
|
||||
`json-mismatch`.
|
||||
|
||||
### Added
|
||||
|
||||
- Track conformance against the official YAML test suite
|
||||
(github.com/yaml/yaml-test-suite, `data` branch). The 182 cases PureYAML does
|
||||
not yet handle correctly are vendored under `Tests/Fixtures/yaml-test-suite/`
|
||||
with a machine-readable manifest (`Tests/Fixtures/yaml-test-suite-gaps.yaml`)
|
||||
recording each gap's category and current behavior. A new opt-in sweep
|
||||
(`bash scripts/check-yaml-test-suite.sh`, gated behind
|
||||
`PUREYAML_RUN_YAML_TEST_SUITE=1`) reproduces every recorded gap so future fixes
|
||||
must update the manifest. `docs/YAML_TEST_SUITE_CONFORMANCE.md` documents the
|
||||
measured result (220/402 passing) and the ordered plan for closing the gaps.
|
||||
|
||||
### Added
|
||||
|
||||
- Opt-in located parse: `parseStreamLocated` returns a source-span projection of every document,
|
||||
|
||||
@ -9,20 +9,36 @@ let package = Package(
|
||||
name: "PureYAML",
|
||||
targets: ["PureYAML"],
|
||||
),
|
||||
.library(
|
||||
name: "PureYAMLJSON",
|
||||
targets: ["PureYAMLJSON"],
|
||||
),
|
||||
],
|
||||
dependencies: [],
|
||||
targets: [
|
||||
.target(
|
||||
name: "PureYAML",
|
||||
path: "Sources",
|
||||
exclude: ["PureYAMLJSON"],
|
||||
),
|
||||
.target(
|
||||
name: "PureYAMLJSON",
|
||||
dependencies: ["PureYAML"],
|
||||
path: "Sources/PureYAMLJSON",
|
||||
),
|
||||
.testTarget(
|
||||
name: "PureYAMLTests",
|
||||
dependencies: ["PureYAML"],
|
||||
path: "Tests",
|
||||
exclude: ["PureYAMLJSONTests"],
|
||||
resources: [
|
||||
.copy("Fixtures"),
|
||||
],
|
||||
),
|
||||
.testTarget(
|
||||
name: "PureYAMLJSONTests",
|
||||
dependencies: ["PureYAMLJSON"],
|
||||
path: "Tests/PureYAMLJSONTests",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
12
README.md
12
README.md
@ -1,11 +1,11 @@
|
||||
# PureYAML
|
||||
|
||||
[](https://github.com/mihaelamj/PureYAML/actions/workflows/gates.yml)
|
||||
[](https://github.com/mihaelamj/PureYAML/actions/workflows/macos.yml)
|
||||
[](https://github.com/mihaelamj/PureYAML/actions/workflows/linux.yml)
|
||||
[](https://github.com/mihaelamj/PureYAML/actions/workflows/windows.yml)
|
||||
[](https://github.com/mihaelamj/PureYAML/actions/workflows/wasm.yml)
|
||||
[](https://github.com/mihaelamj/PureYAML/actions/workflows/conformance.yml)
|
||||
[](https://codeberg.org/Mihaela/PureYAML/actions?workflow=gates.yml)
|
||||
[](https://codeberg.org/Mihaela/PureYAML/actions?workflow=macos.yml)
|
||||
[](https://codeberg.org/Mihaela/PureYAML/actions?workflow=linux.yml)
|
||||
[](https://codeberg.org/Mihaela/PureYAML/actions?workflow=windows.yml)
|
||||
[](https://codeberg.org/Mihaela/PureYAML/actions?workflow=wasm.yml)
|
||||
[](https://codeberg.org/Mihaela/PureYAML/actions?workflow=conformance.yml)
|
||||
[](LICENSE)
|
||||
|
||||
PureYAML is a dependency-free YAML package written entirely in Swift.
|
||||
|
||||
@ -356,9 +356,13 @@ extension PureYAML.Emitting.Dumper {
|
||||
}
|
||||
|
||||
func quote(_ value: String) -> String {
|
||||
// Iterate over unicode scalars, not Characters: a CR+LF pair forms a
|
||||
// single extended grapheme cluster in Swift, so a Character-level switch
|
||||
// would match neither "\r" nor "\n" and emit the line break unescaped,
|
||||
// which does not round-trip through the folding parser.
|
||||
var output = "\""
|
||||
for character in value {
|
||||
switch character {
|
||||
for scalar in value.unicodeScalars {
|
||||
switch scalar {
|
||||
case "\"":
|
||||
output += "\\\""
|
||||
case "\\":
|
||||
@ -370,7 +374,7 @@ extension PureYAML.Emitting.Dumper {
|
||||
case "\t":
|
||||
output += "\\t"
|
||||
default:
|
||||
output.append(character)
|
||||
output.unicodeScalars.append(scalar)
|
||||
}
|
||||
}
|
||||
output += "\""
|
||||
|
||||
@ -2,8 +2,8 @@ extension PureYAML.Parsing {
|
||||
enum Event: Equatable, CustomStringConvertible {
|
||||
case streamStart(mark: Mark)
|
||||
case streamEnd(mark: Mark)
|
||||
case documentStart(mark: Mark)
|
||||
case documentEnd(mark: Mark)
|
||||
case documentStart(explicit: Bool, mark: Mark)
|
||||
case documentEnd(explicit: Bool, mark: Mark)
|
||||
case sequenceStart(anchor: String?, tag: String?, style: CollectionStyle, mark: Mark)
|
||||
case sequenceEnd(mark: Mark)
|
||||
case mappingStart(anchor: String?, tag: String?, style: CollectionStyle, mark: Mark)
|
||||
@ -17,10 +17,10 @@ extension PureYAML.Parsing {
|
||||
"streamStart @\(mark)"
|
||||
case let .streamEnd(mark):
|
||||
"streamEnd @\(mark)"
|
||||
case let .documentStart(mark):
|
||||
"documentStart @\(mark)"
|
||||
case let .documentEnd(mark):
|
||||
"documentEnd @\(mark)"
|
||||
case let .documentStart(explicit, mark):
|
||||
"documentStart explicit=\(explicit) @\(mark)"
|
||||
case let .documentEnd(explicit, mark):
|
||||
"documentEnd explicit=\(explicit) @\(mark)"
|
||||
case let .sequenceStart(anchor, tag, style, mark):
|
||||
"sequenceStart anchor=\(describe(anchor)) tag=\(describe(tag)) style=\(style) @\(mark)"
|
||||
case let .sequenceEnd(mark):
|
||||
@ -39,6 +39,36 @@ extension PureYAML.Parsing {
|
||||
}
|
||||
|
||||
extension PureYAML.Parsing.Event {
|
||||
/// Re-emit the event with libyaml's 0-based line/column mark converted to the
|
||||
/// package's 1-based convention (the byte index is already 0-based in both).
|
||||
func bumpingMark() -> Self {
|
||||
let source = mark
|
||||
let bumped = PureYAML.Parsing.Mark(line: source.line + 1, column: source.column + 1, index: source.index)
|
||||
switch self {
|
||||
case .streamStart:
|
||||
return .streamStart(mark: bumped)
|
||||
case .streamEnd:
|
||||
return .streamEnd(mark: bumped)
|
||||
case let .documentStart(explicit, _):
|
||||
return .documentStart(explicit: explicit, mark: bumped)
|
||||
case let .documentEnd(explicit, _):
|
||||
return .documentEnd(explicit: explicit, mark: bumped)
|
||||
case let .sequenceStart(anchor, tag, style, _):
|
||||
return .sequenceStart(anchor: anchor, tag: tag, style: style, mark: bumped)
|
||||
case .sequenceEnd:
|
||||
return .sequenceEnd(mark: bumped)
|
||||
case let .mappingStart(anchor, tag, style, _):
|
||||
return .mappingStart(anchor: anchor, tag: tag, style: style, mark: bumped)
|
||||
case .mappingEnd:
|
||||
return .mappingEnd(mark: bumped)
|
||||
case let .scalar(value, anchor, tag, style, _, endMark):
|
||||
let bumpedEnd = PureYAML.Parsing.Mark(line: endMark.line + 1, column: endMark.column + 1, index: endMark.index)
|
||||
return .scalar(value: value, anchor: anchor, tag: tag, style: style, mark: bumped, endMark: bumpedEnd)
|
||||
case let .alias(anchor, _):
|
||||
return .alias(anchor: anchor, mark: bumped)
|
||||
}
|
||||
}
|
||||
|
||||
var isMappingEnd: Bool {
|
||||
if case .mappingEnd = self {
|
||||
return true
|
||||
@ -63,8 +93,6 @@ extension PureYAML.Parsing.Event {
|
||||
var mark: PureYAML.Parsing.Mark {
|
||||
switch self {
|
||||
case let .alias(_, mark),
|
||||
let .documentEnd(mark),
|
||||
let .documentStart(mark),
|
||||
let .mappingEnd(mark),
|
||||
let .mappingStart(_, _, _, mark),
|
||||
let .scalar(_, _, _, _, mark, _),
|
||||
@ -73,6 +101,9 @@ extension PureYAML.Parsing.Event {
|
||||
let .streamEnd(mark),
|
||||
let .streamStart(mark):
|
||||
mark
|
||||
case let .documentEnd(_, mark),
|
||||
let .documentStart(_, mark):
|
||||
mark
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -41,7 +41,7 @@ extension PureYAML.Parsing {
|
||||
}
|
||||
return false
|
||||
}
|
||||
if case let .documentStart(mark)? = current {
|
||||
if case let .documentStart(_, mark)? = current {
|
||||
throw PureYAML.Parsing.ParseError.unsupportedMultiDocumentStream(line: mark.line)
|
||||
}
|
||||
_ = try expectStreamEnd()
|
||||
@ -173,7 +173,10 @@ extension PureYAML.Parsing.EventComposer {
|
||||
)
|
||||
}
|
||||
index += 1
|
||||
store(.string(value), anchor: anchor)
|
||||
if let anchor {
|
||||
let model = try composeScalar(value, tag: tag, style: style, mark: .start)
|
||||
store(model, anchor: anchor)
|
||||
}
|
||||
return ComposedMappingKey(
|
||||
keyNode: .string(value),
|
||||
scalarValue: value,
|
||||
|
||||
@ -30,9 +30,9 @@ extension PureYAML.Parsing {
|
||||
didSeeStreamStart = true
|
||||
case .streamEnd:
|
||||
didSeeStreamEnd = true
|
||||
case let .documentStart(mark):
|
||||
case let .documentStart(_, mark):
|
||||
try startDocument(mark: mark)
|
||||
case let .documentEnd(mark):
|
||||
case let .documentEnd(_, mark):
|
||||
try endDocument(mark: mark)
|
||||
case let .scalar(value, anchor, tag, style, mark, endMark):
|
||||
try consumeScalar(value, anchor: anchor, tag: tag, style: style, marks: (mark, endMark))
|
||||
@ -104,7 +104,6 @@ private extension PureYAML.Parsing.EventStreamComposer {
|
||||
}
|
||||
anchors = [:]
|
||||
currentDocumentValue = nil
|
||||
currentDocumentLocated = nil
|
||||
}
|
||||
|
||||
mutating func endDocument(mark: PureYAML.Parsing.Mark) throws {
|
||||
@ -128,8 +127,6 @@ private extension PureYAML.Parsing.EventStreamComposer {
|
||||
) throws {
|
||||
let model = try scalarParser.composeScalarValue(value, tag: tag, style: style, mark: marks.start)
|
||||
if let anchor {
|
||||
// Register the resolved node, not the raw text: an alias resolves to the anchored node's
|
||||
// value, so an empty scalar anchors null and a typed scalar (`&a 42`) anchors that value.
|
||||
anchors[anchor] = model
|
||||
}
|
||||
try complete(.init(
|
||||
@ -198,7 +195,7 @@ private extension PureYAML.Parsing.EventStreamComposer {
|
||||
guard container.pendingKey == nil else {
|
||||
throw unexpectedEvent(expected: "mapping value", mark: mark)
|
||||
}
|
||||
let pairs = PureYAML.Parsing.MergeResolution.mergedPairs(
|
||||
let pairs = mergedPairs(
|
||||
from: container.mergeSources,
|
||||
localPairs: container.localPairs,
|
||||
)
|
||||
@ -297,7 +294,7 @@ private extension PureYAML.Parsing.EventStreamComposer {
|
||||
guard !node.sequenceItems.isEmpty else {
|
||||
return try values.map { value in
|
||||
guard case let .mapping(mapping) = value else {
|
||||
throw PureYAML.Parsing.MergeResolution.invalidMergeValue(mark: node.mark)
|
||||
throw invalidMergeValue(mark: node.mark)
|
||||
}
|
||||
return mapping.pairs
|
||||
}
|
||||
@ -305,15 +302,51 @@ private extension PureYAML.Parsing.EventStreamComposer {
|
||||
return try values.enumerated().map { index, value in
|
||||
let mark = node.sequenceItems[index].mark
|
||||
guard case let .mapping(mapping) = value else {
|
||||
throw PureYAML.Parsing.MergeResolution.invalidMergeValue(mark: mark)
|
||||
throw invalidMergeValue(mark: mark)
|
||||
}
|
||||
return mapping.pairs
|
||||
}
|
||||
case .bool, .double, .int, .null, .string:
|
||||
throw PureYAML.Parsing.MergeResolution.invalidMergeValue(mark: node.mark)
|
||||
throw invalidMergeValue(mark: node.mark)
|
||||
}
|
||||
}
|
||||
|
||||
func mergedPairs(
|
||||
from mergeSources: [[PureYAML.Model.Pair]],
|
||||
localPairs: [PureYAML.Model.Pair],
|
||||
) -> [PureYAML.Model.Pair] {
|
||||
guard !mergeSources.isEmpty else {
|
||||
return localPairs
|
||||
}
|
||||
|
||||
let localKeys = Set(localPairs.map(\.keyNode))
|
||||
var inheritedKeys = Set<PureYAML.Model.Key>()
|
||||
var inheritedPairs: [PureYAML.Model.Pair] = []
|
||||
for source in mergeSources {
|
||||
var sourceKeys = Set<PureYAML.Model.Key>()
|
||||
for pair in source {
|
||||
guard !localKeys.contains(pair.keyNode),
|
||||
!inheritedKeys.contains(pair.keyNode)
|
||||
else {
|
||||
continue
|
||||
}
|
||||
inheritedPairs.append(pair)
|
||||
sourceKeys.insert(pair.keyNode)
|
||||
}
|
||||
inheritedKeys.formUnion(sourceKeys)
|
||||
}
|
||||
return inheritedPairs + localPairs
|
||||
}
|
||||
|
||||
func invalidMergeValue(
|
||||
mark: PureYAML.Parsing.Mark,
|
||||
) -> PureYAML.Parsing.ParseError {
|
||||
PureYAML.Parsing.ParseError.invalidMergeValue(
|
||||
line: mark.line,
|
||||
column: mark.column,
|
||||
)
|
||||
}
|
||||
|
||||
func unexpectedEvent(
|
||||
expected: String,
|
||||
mark: PureYAML.Parsing.Mark,
|
||||
|
||||
103
Sources/Parsing/LibYAML.CharacterClass.swift
Normal file
103
Sources/Parsing/LibYAML.CharacterClass.swift
Normal file
@ -0,0 +1,103 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// Port of libyaml's character-class macros (`IS_*` in `yaml_private.h`).
|
||||
/// libyaml inspects UTF-8 bytes; this operates on `Unicode.Scalar`, where the
|
||||
/// multi-byte line breaks NEL (`U+0085`), LS (`U+2028`), and PS (`U+2029`) are
|
||||
/// single scalars. End of input is `nil`, which is libyaml's `\0` sentinel, so
|
||||
/// `isZ(nil)` is true and every other predicate is false for `nil`.
|
||||
enum CharacterClass {
|
||||
/// `IS_Z`: the end of the input (libyaml's `\0`).
|
||||
static func isZ(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
scalar == nil
|
||||
}
|
||||
|
||||
/// `IS_SPACE`.
|
||||
static func isSpace(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
scalar == " "
|
||||
}
|
||||
|
||||
/// `IS_TAB`.
|
||||
static func isTab(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
scalar == "\t"
|
||||
}
|
||||
|
||||
/// `IS_BLANK`: space or tab.
|
||||
static func isBlank(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
isSpace(scalar) || isTab(scalar)
|
||||
}
|
||||
|
||||
/// `IS_BREAK`: CR, LF, NEL, LS, or PS.
|
||||
static func isBreak(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
switch scalar {
|
||||
case "\r", "\n", "\u{85}", "\u{2028}", "\u{2029}": true
|
||||
default: false
|
||||
}
|
||||
}
|
||||
|
||||
/// `IS_BREAKZ`: a line break or the end of input.
|
||||
static func isBreakZ(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
isBreak(scalar) || isZ(scalar)
|
||||
}
|
||||
|
||||
/// `IS_SPACEZ`: a space, a line break, or the end of input.
|
||||
static func isSpaceZ(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
isSpace(scalar) || isBreakZ(scalar)
|
||||
}
|
||||
|
||||
/// `IS_BLANKZ`: blank (space or tab), a line break, or the end of input.
|
||||
static func isBlankZ(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
isBlank(scalar) || isBreakZ(scalar)
|
||||
}
|
||||
|
||||
/// A flow collection indicator (`,`, `[`, `]`, `{`, `}`); the YAML
|
||||
/// `c-flow-indicator` set excluded from `ns-plain-safe` in flow context.
|
||||
static func isFlowIndicator(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
switch scalar {
|
||||
case ",", "[", "]", "{", "}": true
|
||||
default: false
|
||||
}
|
||||
}
|
||||
|
||||
/// `IS_DIGIT`.
|
||||
static func isDigit(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
guard let scalar else { return false }
|
||||
return scalar >= "0" && scalar <= "9"
|
||||
}
|
||||
|
||||
/// `IS_ALPHA`: a digit, ASCII letter, `_`, or `-` (the directive, anchor,
|
||||
/// and tag-handle name characters).
|
||||
static func isAlpha(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
guard let scalar else { return false }
|
||||
return (scalar >= "0" && scalar <= "9")
|
||||
|| (scalar >= "A" && scalar <= "Z")
|
||||
|| (scalar >= "a" && scalar <= "z")
|
||||
|| scalar == "_"
|
||||
|| scalar == "-"
|
||||
}
|
||||
|
||||
/// `IS_HEX`: a hexadecimal digit.
|
||||
static func isHex(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
guard let scalar else { return false }
|
||||
return (scalar >= "0" && scalar <= "9")
|
||||
|| (scalar >= "A" && scalar <= "F")
|
||||
|| (scalar >= "a" && scalar <= "f")
|
||||
}
|
||||
|
||||
/// `IS_ASCII`.
|
||||
static func isASCII(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
guard let scalar else { return false }
|
||||
return scalar.value <= 0x7F
|
||||
}
|
||||
|
||||
/// `IS_PRINTABLE`: the YAML `c-printable` set.
|
||||
static func isPrintable(_ scalar: Unicode.Scalar?) -> Bool {
|
||||
guard let scalar else { return false }
|
||||
let value = scalar.value
|
||||
return value == 0x09 || value == 0x0A || value == 0x0D
|
||||
|| (value >= 0x20 && value <= 0x7E)
|
||||
|| value == 0x85
|
||||
|| (value >= 0xA0 && value <= 0xD7FF)
|
||||
|| (value >= 0xE000 && value <= 0xFFFD)
|
||||
|| (value >= 0x10000 && value <= 0x10FFFF)
|
||||
}
|
||||
}
|
||||
}
|
||||
85
Sources/Parsing/LibYAML.IndentEngine.swift
Normal file
85
Sources/Parsing/LibYAML.IndentEngine.swift
Normal file
@ -0,0 +1,85 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// Port of the indentation stack from `yaml_parser_t` plus
|
||||
/// `yaml_parser_roll_indent` and `yaml_parser_unroll_indent`. libyaml tracks a
|
||||
/// single `indent` column (the current block collection's indentation, -1 at
|
||||
/// the document root) and a stack of the enclosing levels. A block collection
|
||||
/// is opened only when content is strictly more indented than the current
|
||||
/// level, which is exactly why `- a:` followed by a same-column `b:` keeps `b`
|
||||
/// a sibling and leaves `a` an empty value.
|
||||
struct IndentEngine {
|
||||
/// `parser->indent`. -1 at the document root, matching libyaml.
|
||||
private(set) var indent = -1
|
||||
/// `parser->indents`, the stack of enclosing indentation levels.
|
||||
private(set) var indents: [Int] = []
|
||||
/// `parser->flow_level`; indentation is inert inside flow collections.
|
||||
var flowLevel = 0
|
||||
/// The token queue (`parser->tokens`) the engine appends to or, for
|
||||
/// `roll_indent`, inserts a start token into at a recorded position.
|
||||
private(set) var tokens: [Token] = []
|
||||
/// `parser->tokens_parsed`, the count already dequeued, used to turn an
|
||||
/// absolute token number into an index into the live queue.
|
||||
var tokensParsed = 0
|
||||
|
||||
/// Port of `yaml_parser_roll_indent`. In flow context it does nothing.
|
||||
/// When the column is greater than the current indent, it pushes the
|
||||
/// current indent, adopts the new column, and emits the block start token.
|
||||
/// `number == -1` appends; otherwise the token is inserted at the recorded
|
||||
/// queue position so a `BLOCK-MAPPING-START` precedes a simple key that was
|
||||
/// already queued.
|
||||
mutating func rollIndent(
|
||||
column: Int,
|
||||
number: Int,
|
||||
type: TokenType,
|
||||
mark: PureYAML.Parsing.Mark,
|
||||
) {
|
||||
if flowLevel != 0 {
|
||||
return
|
||||
}
|
||||
if indent < column {
|
||||
indents.append(indent)
|
||||
indent = column
|
||||
let token = Token(type: type, startMark: mark, endMark: mark)
|
||||
if number == -1 {
|
||||
tokens.append(token)
|
||||
} else {
|
||||
tokens.insert(token, at: number - tokensParsed)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_unroll_indent`. In flow context it does nothing.
|
||||
/// While the current indent is deeper than the column, it emits a
|
||||
/// `BLOCK-END` and pops the enclosing level, closing every block collection
|
||||
/// that the dedent ends.
|
||||
mutating func unrollIndent(column: Int, mark: PureYAML.Parsing.Mark) {
|
||||
if flowLevel != 0 {
|
||||
return
|
||||
}
|
||||
while indent > column {
|
||||
tokens.append(Token(type: .blockEnd, startMark: mark, endMark: mark))
|
||||
indent = indents.removeLast()
|
||||
}
|
||||
}
|
||||
|
||||
/// `ENQUEUE`: append a token to the queue.
|
||||
mutating func enqueue(_ token: Token) {
|
||||
tokens.append(token)
|
||||
}
|
||||
|
||||
/// `QUEUE_INSERT`: insert a token at an absolute token number, used to
|
||||
/// splice a `KEY` token in front of a node already queued.
|
||||
mutating func insert(_ token: Token, atTokenNumber number: Int) {
|
||||
tokens.insert(token, at: number - tokensParsed)
|
||||
}
|
||||
|
||||
/// Dequeue the front token, advancing `tokensParsed` like libyaml's
|
||||
/// `yaml_parser_scan` does when it hands a token to the parser.
|
||||
mutating func dequeue() -> Token? {
|
||||
guard !tokens.isEmpty else {
|
||||
return nil
|
||||
}
|
||||
tokensParsed += 1
|
||||
return tokens.removeFirst()
|
||||
}
|
||||
}
|
||||
}
|
||||
12
Sources/Parsing/LibYAML.NodeProperties.swift
Normal file
12
Sources/Parsing/LibYAML.NodeProperties.swift
Normal file
@ -0,0 +1,12 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// The anchor and tag scanned ahead of a node's content, with the marks needed
|
||||
/// for events and tag-resolution errors. Mirrors the locals in
|
||||
/// `yaml_parser_parse_node`.
|
||||
struct NodeProperties {
|
||||
var anchor: String?
|
||||
var tagHandle: String?
|
||||
var tagSuffix: String?
|
||||
var startMark: PureYAML.Parsing.Mark
|
||||
var tagMark: PureYAML.Parsing.Mark
|
||||
}
|
||||
}
|
||||
109
Sources/Parsing/LibYAML.Parser.Block.swift
Normal file
109
Sources/Parsing/LibYAML.Parser.Block.swift
Normal file
@ -0,0 +1,109 @@
|
||||
extension PureYAML.Parsing.LibYAML.Parser {
|
||||
/// Port of `yaml_parser_parse_block_sequence_entry`.
|
||||
mutating func parseBlockSequenceEntry(first: Bool) throws -> Event {
|
||||
if first {
|
||||
marks.append(peekToken().startMark)
|
||||
skipToken()
|
||||
}
|
||||
let token = peekToken()
|
||||
if case .blockEntry = token.type {
|
||||
let mark = token.endMark
|
||||
skipToken()
|
||||
let next = peekToken()
|
||||
if !isBlockEntry(next), !isBlockEnd(next) {
|
||||
states.append(.blockSequenceEntry)
|
||||
return try parseNode(block: true, indentlessSequence: false)
|
||||
}
|
||||
state = .blockSequenceEntry
|
||||
return processEmptyScalar(at: mark)
|
||||
}
|
||||
if case .blockEnd = token.type {
|
||||
state = states.removeLast()
|
||||
_ = marks.popLast()
|
||||
skipToken()
|
||||
return .sequenceEnd(mark: token.startMark)
|
||||
}
|
||||
throw ParserError(
|
||||
context: "while parsing a block collection",
|
||||
problem: "did not find expected '-' indicator",
|
||||
mark: marks.popLast() ?? token.startMark,
|
||||
)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_indentless_sequence_entry`. The closing token
|
||||
/// belongs to the enclosing collection, so the SEQUENCE-END is emitted without
|
||||
/// consuming it.
|
||||
mutating func parseIndentlessSequenceEntry() throws -> Event {
|
||||
let token = peekToken()
|
||||
guard case .blockEntry = token.type else {
|
||||
state = states.removeLast()
|
||||
return .sequenceEnd(mark: token.startMark)
|
||||
}
|
||||
let mark = token.endMark
|
||||
skipToken()
|
||||
let next = peekToken()
|
||||
if !isBlockEntry(next), !isKey(next), !isValue(next), !isBlockEnd(next) {
|
||||
states.append(.indentlessSequenceEntry)
|
||||
return try parseNode(block: true, indentlessSequence: false)
|
||||
}
|
||||
state = .indentlessSequenceEntry
|
||||
return processEmptyScalar(at: mark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_block_mapping_key`.
|
||||
mutating func parseBlockMappingKey(first: Bool) throws -> Event {
|
||||
if first {
|
||||
marks.append(peekToken().startMark)
|
||||
skipToken()
|
||||
}
|
||||
let token = peekToken()
|
||||
if case .key = token.type {
|
||||
let mark = token.endMark
|
||||
skipToken()
|
||||
let next = peekToken()
|
||||
if !isKey(next), !isValue(next), !isBlockEnd(next) {
|
||||
states.append(.blockMappingValue)
|
||||
return try parseNode(block: true, indentlessSequence: true)
|
||||
}
|
||||
state = .blockMappingValue
|
||||
return processEmptyScalar(at: mark)
|
||||
}
|
||||
if case .blockEnd = token.type {
|
||||
state = states.removeLast()
|
||||
_ = marks.popLast()
|
||||
skipToken()
|
||||
return .mappingEnd(mark: token.startMark)
|
||||
}
|
||||
// YAML 1.2 permits an empty (null) key: a VALUE indicator with no
|
||||
// preceding key denotes a mapping entry whose key is empty. libyaml
|
||||
// rejects this here; instead emit the empty scalar key and leave the
|
||||
// VALUE token for `parseBlockMappingValue` to consume.
|
||||
if case .value = token.type {
|
||||
state = .blockMappingValue
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
throw ParserError(
|
||||
context: "while parsing a block mapping",
|
||||
problem: "did not find expected key",
|
||||
mark: marks.popLast() ?? token.startMark,
|
||||
)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_block_mapping_value`.
|
||||
mutating func parseBlockMappingValue() throws -> Event {
|
||||
let token = peekToken()
|
||||
guard case .value = token.type else {
|
||||
state = .blockMappingKey
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
let mark = token.endMark
|
||||
skipToken()
|
||||
let next = peekToken()
|
||||
if !isKey(next), !isValue(next), !isBlockEnd(next) {
|
||||
states.append(.blockMappingKey)
|
||||
return try parseNode(block: true, indentlessSequence: true)
|
||||
}
|
||||
state = .blockMappingKey
|
||||
return processEmptyScalar(at: mark)
|
||||
}
|
||||
}
|
||||
70
Sources/Parsing/LibYAML.Parser.Directives.swift
Normal file
70
Sources/Parsing/LibYAML.Parser.Directives.swift
Normal file
@ -0,0 +1,70 @@
|
||||
extension PureYAML.Parsing.LibYAML.Parser {
|
||||
/// The implied tag handles, mirroring libyaml's `default_tag_directives`.
|
||||
private var defaultTagDirectives: [(handle: String, prefix: String)] {
|
||||
[("!", "!"), ("!!", "tag:yaml.org,2002:")]
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_process_directives`: consume the `%YAML` and `%TAG`
|
||||
/// directive tokens at the head of a document, validating the version and
|
||||
/// recording the tag handles, then install the default tag handles. The
|
||||
/// version is checked against a local so a duplicate `%YAML` is rejected.
|
||||
mutating func processDirectives() throws {
|
||||
var version: (major: Int, minor: Int)?
|
||||
var token = peekToken()
|
||||
while true {
|
||||
if case let .versionDirective(major, minor) = token.type {
|
||||
try recordVersion(major: major, minor: minor, into: &version, mark: token.startMark)
|
||||
} else if case let .tagDirective(handle, prefix) = token.type {
|
||||
try appendTagDirective(handle: handle, prefix: prefix, allowDuplicates: false, mark: token.startMark)
|
||||
} else if case .reservedDirective = token.type {
|
||||
// A reserved directive is ignored (YAML 1.2 §6.8.1) but still
|
||||
// consumed here so the explicit-document path requires a `---`.
|
||||
} else {
|
||||
break
|
||||
}
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
}
|
||||
for directive in defaultTagDirectives {
|
||||
try appendTagDirective(handle: directive.handle, prefix: directive.prefix, allowDuplicates: true, mark: token.startMark)
|
||||
}
|
||||
}
|
||||
|
||||
private mutating func recordVersion(major: Int, minor: Int, into version: inout (major: Int, minor: Int)?, mark: Mark) throws {
|
||||
guard version == nil else {
|
||||
throw ParserError(context: nil, problem: "found duplicate %YAML directive", mark: mark)
|
||||
}
|
||||
// YAML 1.2 §6.8.1: a document whose major version is 1 is parsed even when
|
||||
// the minor version is newer than 1.2 (with a warning), rather than
|
||||
// libyaml's hard rejection of any minor other than 1 or 2.
|
||||
guard major == 1 else {
|
||||
throw ParserError(context: nil, problem: "found incompatible YAML document", mark: mark)
|
||||
}
|
||||
version = (major, minor)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_append_tag_directive`: append a handle/prefix mapping,
|
||||
/// rejecting a redefinition unless duplicates are allowed (the defaults are).
|
||||
mutating func appendTagDirective(handle: String, prefix: String, allowDuplicates: Bool, mark: Mark) throws {
|
||||
if tagDirectives.contains(where: { $0.handle == handle }) {
|
||||
if allowDuplicates {
|
||||
return
|
||||
}
|
||||
throw ParserError(context: nil, problem: "found duplicate %TAG directive", mark: mark)
|
||||
}
|
||||
tagDirectives.append((handle, prefix))
|
||||
}
|
||||
|
||||
/// Port of the tag-resolution block in `yaml_parser_parse_node`: an empty
|
||||
/// handle means the suffix is a verbatim tag; otherwise substitute the handle's
|
||||
/// registered prefix, failing on an undefined handle.
|
||||
func resolveTag(handle: String, suffix: String, startMark _: Mark, tagMark: Mark) throws -> String {
|
||||
guard !handle.isEmpty else {
|
||||
return suffix
|
||||
}
|
||||
guard let directive = tagDirectives.first(where: { $0.handle == handle }) else {
|
||||
throw ParserError(context: "while parsing a node", problem: "found undefined tag handle", mark: tagMark)
|
||||
}
|
||||
return directive.prefix + suffix
|
||||
}
|
||||
}
|
||||
112
Sources/Parsing/LibYAML.Parser.Document.swift
Normal file
112
Sources/Parsing/LibYAML.Parser.Document.swift
Normal file
@ -0,0 +1,112 @@
|
||||
extension PureYAML.Parsing.LibYAML.Parser {
|
||||
/// Port of `yaml_parser_parse_stream_start`.
|
||||
mutating func parseStreamStart() throws -> Event {
|
||||
let token = peekToken()
|
||||
guard case .streamStart = token.type else {
|
||||
throw ParserError(context: nil, problem: "did not find expected <stream-start>", mark: token.startMark)
|
||||
}
|
||||
state = .implicitDocumentStart
|
||||
skipToken()
|
||||
return .streamStart(mark: token.startMark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_document_start`, extended for YAML 1.2: stray
|
||||
/// `...` markers are skipped, then a bare document is opened when one is allowed
|
||||
/// (the first document or after an explicit `...`), an explicit document is
|
||||
/// opened after directives and a `---`, or STREAM-END is emitted. libyaml only
|
||||
/// allows a bare document as the very first one.
|
||||
mutating func parseDocumentStart() throws -> Event {
|
||||
var token = peekToken()
|
||||
while case .documentEnd = token.type {
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
}
|
||||
// Directives apply to the document that follows, so the previous document
|
||||
// must be closed with an explicit `...` end marker before them. After an
|
||||
// implicit end (`allowBareDocument` is false) a directive is an error.
|
||||
if isDirective(token), !allowBareDocument {
|
||||
throw ParserError(
|
||||
context: nil,
|
||||
problem: "directives require a preceding document end marker",
|
||||
mark: token.startMark,
|
||||
)
|
||||
}
|
||||
if allowBareDocument, !isDirectiveOrBoundary(token) {
|
||||
try processDirectives()
|
||||
states.append(.documentEnd)
|
||||
state = .blockNode
|
||||
return .documentStart(explicit: false, mark: token.startMark)
|
||||
}
|
||||
if case .streamEnd = token.type {
|
||||
state = .end
|
||||
skipToken()
|
||||
return .streamEnd(mark: token.startMark)
|
||||
}
|
||||
return try openExplicitDocument(startMark: token.startMark)
|
||||
}
|
||||
|
||||
/// The explicit-document branch: directives then a required `---`.
|
||||
private mutating func openExplicitDocument(startMark: Mark) throws -> Event {
|
||||
try processDirectives()
|
||||
let token = peekToken()
|
||||
guard case .documentStart = token.type else {
|
||||
throw ParserError(context: nil, problem: "did not find expected <document start>", mark: token.startMark)
|
||||
}
|
||||
states.append(.documentEnd)
|
||||
state = .documentContent
|
||||
skipToken()
|
||||
return .documentStart(explicit: true, mark: startMark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_document_content`: an empty scalar stands in for
|
||||
/// a document with no node, otherwise parse the root node.
|
||||
mutating func parseDocumentContent() throws -> Event {
|
||||
let token = peekToken()
|
||||
if isDirectiveOrBoundary(token) || isDocumentEnd(token) {
|
||||
state = states.removeLast()
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
return try parseNode(block: true, indentlessSequence: false)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_document_end`: reset the per-document tag
|
||||
/// directives and emit DOCUMENT-END, explicit only when `...` was present.
|
||||
mutating func parseDocumentEnd() throws -> Event {
|
||||
let token = peekToken()
|
||||
var explicit = false
|
||||
if case .documentEnd = token.type {
|
||||
skipToken()
|
||||
explicit = true
|
||||
}
|
||||
tagDirectives.removeAll()
|
||||
// A bare document may follow only an explicit `...` end marker.
|
||||
allowBareDocument = explicit
|
||||
state = .documentStart
|
||||
return .documentEnd(explicit: explicit, mark: token.startMark)
|
||||
}
|
||||
|
||||
private func isDirectiveOrBoundary(_ token: Token) -> Bool {
|
||||
switch token.type {
|
||||
case .versionDirective, .tagDirective, .reservedDirective, .documentStart, .streamEnd:
|
||||
true
|
||||
default:
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
private func isDirective(_ token: Token) -> Bool {
|
||||
switch token.type {
|
||||
case .versionDirective, .tagDirective, .reservedDirective:
|
||||
true
|
||||
default:
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
private func isDocumentEnd(_ token: Token) -> Bool {
|
||||
if case .documentEnd = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
}
|
||||
157
Sources/Parsing/LibYAML.Parser.Flow.swift
Normal file
157
Sources/Parsing/LibYAML.Parser.Flow.swift
Normal file
@ -0,0 +1,157 @@
|
||||
extension PureYAML.Parsing.LibYAML.Parser {
|
||||
/// Port of `yaml_parser_parse_flow_sequence_entry`. A `?` key inside a flow
|
||||
/// sequence opens a single-pair flow mapping.
|
||||
mutating func parseFlowSequenceEntry(first: Bool) throws -> Event {
|
||||
if first {
|
||||
marks.append(peekToken().startMark)
|
||||
skipToken()
|
||||
}
|
||||
var token = peekToken()
|
||||
if !isFlowSequenceEnd(token) {
|
||||
if !first {
|
||||
guard isFlowEntry(token) else {
|
||||
throw ParserError(
|
||||
context: "while parsing a flow sequence",
|
||||
problem: "did not find expected ',' or ']'",
|
||||
mark: marks.popLast() ?? token.startMark,
|
||||
)
|
||||
}
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
}
|
||||
if isKey(token) {
|
||||
state = .flowSequenceEntryMappingKey
|
||||
skipToken()
|
||||
return .mappingStart(anchor: nil, tag: nil, style: .flow, mark: token.startMark)
|
||||
}
|
||||
if isValue(token) {
|
||||
// A leading `:` opens a single-pair mapping with an empty key
|
||||
// (`[ : v ]`). Keep the VALUE token for the mapping-key state.
|
||||
state = .flowSequenceEntryMappingKey
|
||||
return .mappingStart(anchor: nil, tag: nil, style: .flow, mark: token.startMark)
|
||||
}
|
||||
if !isFlowSequenceEnd(token) {
|
||||
states.append(.flowSequenceEntry)
|
||||
return try parseNode(block: false, indentlessSequence: false)
|
||||
}
|
||||
}
|
||||
state = states.removeLast()
|
||||
_ = marks.popLast()
|
||||
skipToken()
|
||||
return .sequenceEnd(mark: token.startMark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_flow_sequence_entry_mapping_key`.
|
||||
mutating func parseFlowSequenceEntryMappingKey() throws -> Event {
|
||||
let token = peekToken()
|
||||
if !isValue(token), !isFlowEntry(token), !isFlowSequenceEnd(token) {
|
||||
states.append(.flowSequenceEntryMappingValue)
|
||||
return try parseNode(block: false, indentlessSequence: false)
|
||||
}
|
||||
// An empty key: leave a VALUE token for the value state to consume (so the
|
||||
// value is parsed); a closing `]` ends the pair with an empty value too.
|
||||
if isValue(token) || isFlowSequenceEnd(token) {
|
||||
state = .flowSequenceEntryMappingValue
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
let mark = token.endMark
|
||||
skipToken()
|
||||
state = .flowSequenceEntryMappingValue
|
||||
return processEmptyScalar(at: mark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_flow_sequence_entry_mapping_value`.
|
||||
mutating func parseFlowSequenceEntryMappingValue() throws -> Event {
|
||||
var token = peekToken()
|
||||
if isValue(token) {
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
if !isFlowEntry(token), !isFlowSequenceEnd(token) {
|
||||
states.append(.flowSequenceEntryMappingEnd)
|
||||
return try parseNode(block: false, indentlessSequence: false)
|
||||
}
|
||||
}
|
||||
state = .flowSequenceEntryMappingEnd
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_flow_sequence_entry_mapping_end`.
|
||||
mutating func parseFlowSequenceEntryMappingEnd() throws -> Event {
|
||||
let token = peekToken()
|
||||
state = .flowSequenceEntry
|
||||
return .mappingEnd(mark: token.startMark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_flow_mapping_key`.
|
||||
mutating func parseFlowMappingKey(first: Bool) throws -> Event {
|
||||
if first {
|
||||
marks.append(peekToken().startMark)
|
||||
skipToken()
|
||||
}
|
||||
var token = peekToken()
|
||||
if !isFlowMappingEnd(token) {
|
||||
if !first {
|
||||
guard isFlowEntry(token) else {
|
||||
throw ParserError(
|
||||
context: "while parsing a flow mapping",
|
||||
problem: "did not find expected ',' or '}'",
|
||||
mark: marks.popLast() ?? token.startMark,
|
||||
)
|
||||
}
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
}
|
||||
if isKey(token) {
|
||||
return try flowMappingKeyEvent()
|
||||
}
|
||||
// YAML 1.2 permits an empty (null) key in a flow mapping: a `:` with
|
||||
// no preceding key denotes an entry whose key is empty. libyaml hands
|
||||
// the VALUE token to `parse_node` and fails; instead emit the empty
|
||||
// scalar key and let `parseFlowMappingValue` consume the indicator.
|
||||
if isValue(token) {
|
||||
state = .flowMappingValue
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
if !isFlowMappingEnd(token) {
|
||||
states.append(.flowMappingEmptyValue)
|
||||
return try parseNode(block: false, indentlessSequence: false)
|
||||
}
|
||||
}
|
||||
state = states.removeLast()
|
||||
_ = marks.popLast()
|
||||
skipToken()
|
||||
return .mappingEnd(mark: token.startMark)
|
||||
}
|
||||
|
||||
/// The KEY branch of `parse_flow_mapping_key`: parse the key node, or an empty
|
||||
/// scalar when a value/entry/end follows the `?` immediately.
|
||||
private mutating func flowMappingKeyEvent() throws -> Event {
|
||||
skipToken()
|
||||
let token = peekToken()
|
||||
if !isValue(token), !isFlowEntry(token), !isFlowMappingEnd(token) {
|
||||
states.append(.flowMappingValue)
|
||||
return try parseNode(block: false, indentlessSequence: false)
|
||||
}
|
||||
state = .flowMappingValue
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_parse_flow_mapping_value`.
|
||||
mutating func parseFlowMappingValue(empty: Bool) throws -> Event {
|
||||
var token = peekToken()
|
||||
if empty {
|
||||
state = .flowMappingKey
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
if isValue(token) {
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
if !isFlowEntry(token), !isFlowMappingEnd(token) {
|
||||
states.append(.flowMappingKey)
|
||||
return try parseNode(block: false, indentlessSequence: false)
|
||||
}
|
||||
}
|
||||
state = .flowMappingKey
|
||||
return processEmptyScalar(at: token.startMark)
|
||||
}
|
||||
}
|
||||
50
Sources/Parsing/LibYAML.Parser.Helpers.swift
Normal file
50
Sources/Parsing/LibYAML.Parser.Helpers.swift
Normal file
@ -0,0 +1,50 @@
|
||||
extension PureYAML.Parsing.LibYAML.Parser {
|
||||
func isBlockEntry(_ token: Token) -> Bool {
|
||||
if case .blockEntry = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isBlockEnd(_ token: Token) -> Bool {
|
||||
if case .blockEnd = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isKey(_ token: Token) -> Bool {
|
||||
if case .key = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isValue(_ token: Token) -> Bool {
|
||||
if case .value = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isFlowEntry(_ token: Token) -> Bool {
|
||||
if case .flowEntry = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isFlowSequenceEnd(_ token: Token) -> Bool {
|
||||
if case .flowSequenceEnd = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isFlowMappingEnd(_ token: Token) -> Bool {
|
||||
if case .flowMappingEnd = token.type {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
}
|
||||
99
Sources/Parsing/LibYAML.Parser.Node.swift
Normal file
99
Sources/Parsing/LibYAML.Parser.Node.swift
Normal file
@ -0,0 +1,99 @@
|
||||
extension PureYAML.Parsing.LibYAML.Parser {
|
||||
typealias NodeProperties = PureYAML.Parsing.LibYAML.NodeProperties
|
||||
|
||||
/// Port of `yaml_parser_parse_node`. An ALIAS is emitted directly; otherwise
|
||||
/// the node's anchor/tag properties are consumed, the tag is resolved, and the
|
||||
/// content token selects a scalar, a flow/block collection start, or an empty
|
||||
/// scalar standing in for a property-only node.
|
||||
mutating func parseNode(block: Bool, indentlessSequence: Bool) throws -> Event {
|
||||
let token = peekToken()
|
||||
if case let .alias(name) = token.type {
|
||||
state = states.removeLast()
|
||||
skipToken()
|
||||
return .alias(anchor: name, mark: token.startMark)
|
||||
}
|
||||
let props = consumeNodeProperties()
|
||||
let tag = try props.tagHandle.map {
|
||||
try resolveTag(handle: $0, suffix: props.tagSuffix ?? "", startMark: props.startMark, tagMark: props.tagMark)
|
||||
}
|
||||
return try dispatchNodeContent(block: block, indentlessSequence: indentlessSequence, props: props, tag: tag)
|
||||
}
|
||||
|
||||
/// Consume `properties ::= TAG ANCHOR? | ANCHOR TAG?` ahead of the content.
|
||||
private mutating func consumeNodeProperties() -> NodeProperties {
|
||||
var token = peekToken()
|
||||
var props = NodeProperties(anchor: nil, tagHandle: nil, tagSuffix: nil, startMark: token.startMark, tagMark: token.startMark)
|
||||
if case let .anchor(name) = token.type {
|
||||
props.anchor = name
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
if case let .tag(handle, suffix) = token.type {
|
||||
props.tagHandle = handle
|
||||
props.tagSuffix = suffix
|
||||
props.tagMark = token.startMark
|
||||
skipToken()
|
||||
}
|
||||
} else if case let .tag(handle, suffix) = token.type {
|
||||
props.tagHandle = handle
|
||||
props.tagSuffix = suffix
|
||||
props.tagMark = token.startMark
|
||||
skipToken()
|
||||
token = peekToken()
|
||||
if case let .anchor(name) = token.type {
|
||||
props.anchor = name
|
||||
skipToken()
|
||||
}
|
||||
}
|
||||
return props
|
||||
}
|
||||
|
||||
/// Select the event for the content token following a node's properties.
|
||||
private mutating func dispatchNodeContent(block: Bool, indentlessSequence: Bool, props: NodeProperties, tag: String?) throws -> Event {
|
||||
let token = peekToken()
|
||||
if indentlessSequence, case .blockEntry = token.type {
|
||||
state = .indentlessSequenceEntry
|
||||
return .sequenceStart(anchor: props.anchor, tag: tag, style: .block, mark: props.startMark)
|
||||
}
|
||||
if let event = contentEvent(token: token, block: block, props: props, tag: tag) {
|
||||
return event
|
||||
}
|
||||
return try emptyNodeOrError(token: token, block: block, props: props, tag: tag)
|
||||
}
|
||||
|
||||
/// A scalar or a flow/block collection start, or nil if the token is neither.
|
||||
private mutating func contentEvent(token: Token, block: Bool, props: NodeProperties, tag: String?) -> Event? {
|
||||
switch token.type {
|
||||
case let .scalar(value, style):
|
||||
state = states.removeLast()
|
||||
skipToken()
|
||||
return .scalar(value: value, anchor: props.anchor, tag: tag, style: style, mark: props.startMark, endMark: token.endMark)
|
||||
case .flowSequenceStart:
|
||||
state = .flowSequenceFirstEntry
|
||||
return .sequenceStart(anchor: props.anchor, tag: tag, style: .flow, mark: props.startMark)
|
||||
case .flowMappingStart:
|
||||
state = .flowMappingFirstKey
|
||||
return .mappingStart(anchor: props.anchor, tag: tag, style: .flow, mark: props.startMark)
|
||||
case .blockSequenceStart where block:
|
||||
state = .blockSequenceFirstEntry
|
||||
return .sequenceStart(anchor: props.anchor, tag: tag, style: .block, mark: props.startMark)
|
||||
case .blockMappingStart where block:
|
||||
state = .blockMappingFirstKey
|
||||
return .mappingStart(anchor: props.anchor, tag: tag, style: .block, mark: props.startMark)
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
/// An empty scalar when the node carried only properties, else a parse error.
|
||||
private mutating func emptyNodeOrError(token: Token, block: Bool, props: NodeProperties, tag: String?) throws -> Event {
|
||||
if props.anchor != nil || tag != nil {
|
||||
state = states.removeLast()
|
||||
return .scalar(value: "", anchor: props.anchor, tag: tag, style: .plain, mark: props.startMark, endMark: props.startMark)
|
||||
}
|
||||
throw ParserError(
|
||||
context: block ? "while parsing a block node" : "while parsing a flow node",
|
||||
problem: "did not find expected node content",
|
||||
mark: token.startMark,
|
||||
)
|
||||
}
|
||||
}
|
||||
148
Sources/Parsing/LibYAML.Parser.swift
Normal file
148
Sources/Parsing/LibYAML.Parser.swift
Normal file
@ -0,0 +1,148 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// A faithful port of libyaml's `parser.c`: a one-event-at-a-time state
|
||||
/// machine that consumes the scanner's token stream and produces the YAML
|
||||
/// event stream. The token queue is materialized up front by the scanner, so
|
||||
/// `peekToken`/`skipToken` index into a fixed array rather than pulling lazily,
|
||||
/// which is equivalent because simple-key resolution lives entirely in the
|
||||
/// scanner.
|
||||
struct Parser {
|
||||
let tokens: [Token]
|
||||
var cursor = 0
|
||||
var state: ParserState = .streamStart
|
||||
var states: [ParserState] = []
|
||||
var marks: [Mark] = []
|
||||
var tagDirectives: [(handle: String, prefix: String)] = []
|
||||
/// Whether a bare (implicit) document may begin here: true for the first
|
||||
/// document and after an explicit `...` end marker, false otherwise. YAML
|
||||
/// 1.2 allows a bare document to follow `...`, which libyaml rejects.
|
||||
var allowBareDocument = true
|
||||
|
||||
init(_ input: String = "") throws {
|
||||
var scanner = Scanner(input)
|
||||
tokens = try scanner.scanAll()
|
||||
}
|
||||
|
||||
init(tokens: [Token]) {
|
||||
self.tokens = tokens
|
||||
}
|
||||
|
||||
/// Drive the state machine until STREAM-END, collecting every event.
|
||||
mutating func parse() throws -> [Event] {
|
||||
var events: [Event] = []
|
||||
while state != .end {
|
||||
try events.append(stateMachine())
|
||||
}
|
||||
return events
|
||||
}
|
||||
|
||||
/// The head of the token queue. STREAM-END is always the final token and
|
||||
/// the machine halts once it has been consumed, so the clamp is never the
|
||||
/// value actually parsed; it only avoids an out-of-range index.
|
||||
func peekToken() -> Token {
|
||||
tokens[min(cursor, tokens.count - 1)]
|
||||
}
|
||||
|
||||
mutating func skipToken() {
|
||||
cursor += 1
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_state_machine`: dispatch on the current state. The
|
||||
/// flat 24-way switch is split into document/block/flow groups to keep each
|
||||
/// dispatcher's branching within the complexity budget.
|
||||
mutating func stateMachine() throws -> Event {
|
||||
switch state {
|
||||
case .streamStart, .implicitDocumentStart, .documentStart, .documentContent, .documentEnd:
|
||||
try documentStateMachine()
|
||||
case .blockNode, .blockNodeOrIndentlessSequence, .flowNode,
|
||||
.blockSequenceFirstEntry, .blockSequenceEntry, .indentlessSequenceEntry,
|
||||
.blockMappingFirstKey, .blockMappingKey, .blockMappingValue:
|
||||
try nodeStateMachine()
|
||||
case .flowSequenceFirstEntry, .flowSequenceEntry, .flowSequenceEntryMappingKey,
|
||||
.flowSequenceEntryMappingValue, .flowSequenceEntryMappingEnd,
|
||||
.flowMappingFirstKey, .flowMappingKey, .flowMappingValue, .flowMappingEmptyValue:
|
||||
try flowStateMachine()
|
||||
case .end:
|
||||
throw endError()
|
||||
}
|
||||
}
|
||||
|
||||
private mutating func documentStateMachine() throws -> Event {
|
||||
switch state {
|
||||
case .streamStart:
|
||||
try parseStreamStart()
|
||||
case .implicitDocumentStart:
|
||||
try parseDocumentStart()
|
||||
case .documentStart:
|
||||
try parseDocumentStart()
|
||||
case .documentContent:
|
||||
try parseDocumentContent()
|
||||
default:
|
||||
try parseDocumentEnd()
|
||||
}
|
||||
}
|
||||
|
||||
private mutating func nodeStateMachine() throws -> Event {
|
||||
switch state {
|
||||
case .blockNode:
|
||||
try parseNode(block: true, indentlessSequence: false)
|
||||
case .blockNodeOrIndentlessSequence:
|
||||
try parseNode(block: true, indentlessSequence: true)
|
||||
case .flowNode:
|
||||
try parseNode(block: false, indentlessSequence: false)
|
||||
case .blockSequenceFirstEntry:
|
||||
try parseBlockSequenceEntry(first: true)
|
||||
case .blockSequenceEntry:
|
||||
try parseBlockSequenceEntry(first: false)
|
||||
case .indentlessSequenceEntry:
|
||||
try parseIndentlessSequenceEntry()
|
||||
case .blockMappingFirstKey:
|
||||
try parseBlockMappingKey(first: true)
|
||||
case .blockMappingKey:
|
||||
try parseBlockMappingKey(first: false)
|
||||
default:
|
||||
try parseBlockMappingValue()
|
||||
}
|
||||
}
|
||||
|
||||
private mutating func flowStateMachine() throws -> Event {
|
||||
switch state {
|
||||
case .flowSequenceFirstEntry:
|
||||
try parseFlowSequenceEntry(first: true)
|
||||
case .flowSequenceEntry:
|
||||
try parseFlowSequenceEntry(first: false)
|
||||
case .flowSequenceEntryMappingKey:
|
||||
try parseFlowSequenceEntryMappingKey()
|
||||
case .flowSequenceEntryMappingValue:
|
||||
try parseFlowSequenceEntryMappingValue()
|
||||
case .flowSequenceEntryMappingEnd:
|
||||
try parseFlowSequenceEntryMappingEnd()
|
||||
case .flowMappingFirstKey:
|
||||
try parseFlowMappingKey(first: true)
|
||||
case .flowMappingKey:
|
||||
try parseFlowMappingKey(first: false)
|
||||
case .flowMappingValue:
|
||||
try parseFlowMappingValue(empty: false)
|
||||
default:
|
||||
try parseFlowMappingValue(empty: true)
|
||||
}
|
||||
}
|
||||
|
||||
private func endError() -> ParserError {
|
||||
ParserError(context: nil, problem: "parsed past the end of the stream", mark: peekToken().startMark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_process_empty_scalar`: an empty plain scalar event
|
||||
/// stands in for an absent node.
|
||||
func processEmptyScalar(at mark: Mark) -> Event {
|
||||
.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: mark, endMark: mark)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
extension PureYAML.Parsing.LibYAML.Parser {
|
||||
typealias Token = PureYAML.Parsing.LibYAML.Token
|
||||
typealias Mark = PureYAML.Parsing.Mark
|
||||
typealias Event = PureYAML.Parsing.Event
|
||||
typealias ParserError = PureYAML.Parsing.LibYAML.ParserError
|
||||
typealias ParserState = PureYAML.Parsing.LibYAML.ParserState
|
||||
}
|
||||
10
Sources/Parsing/LibYAML.ParserError.swift
Normal file
10
Sources/Parsing/LibYAML.ParserError.swift
Normal file
@ -0,0 +1,10 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// A parser error, mirroring `yaml_parser_set_parser_error[_context]`: an
|
||||
/// optional context describing the surrounding production, the immediate
|
||||
/// problem, and the mark where it was detected.
|
||||
struct ParserError: Error, Equatable {
|
||||
var context: String?
|
||||
var problem: String
|
||||
var mark: PureYAML.Parsing.Mark
|
||||
}
|
||||
}
|
||||
30
Sources/Parsing/LibYAML.ParserState.swift
Normal file
30
Sources/Parsing/LibYAML.ParserState.swift
Normal file
@ -0,0 +1,30 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// The parser state, mirroring `yaml_parser_state_e`. Each case names the
|
||||
/// production the state machine is in the middle of recognizing.
|
||||
enum ParserState: Equatable {
|
||||
case streamStart
|
||||
case implicitDocumentStart
|
||||
case documentStart
|
||||
case documentContent
|
||||
case documentEnd
|
||||
case blockNode
|
||||
case blockNodeOrIndentlessSequence
|
||||
case flowNode
|
||||
case blockSequenceFirstEntry
|
||||
case blockSequenceEntry
|
||||
case indentlessSequenceEntry
|
||||
case blockMappingFirstKey
|
||||
case blockMappingKey
|
||||
case blockMappingValue
|
||||
case flowSequenceFirstEntry
|
||||
case flowSequenceEntry
|
||||
case flowSequenceEntryMappingKey
|
||||
case flowSequenceEntryMappingValue
|
||||
case flowSequenceEntryMappingEnd
|
||||
case flowMappingFirstKey
|
||||
case flowMappingKey
|
||||
case flowMappingValue
|
||||
case flowMappingEmptyValue
|
||||
case end
|
||||
}
|
||||
}
|
||||
57
Sources/Parsing/LibYAML.Reader.swift
Normal file
57
Sources/Parsing/LibYAML.Reader.swift
Normal file
@ -0,0 +1,57 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// Port of libyaml's input buffer and position tracking (the `buffer`/`mark`
|
||||
/// fields with the `PEEK`, `SKIP`, and `SKIP_LINE` macros). It iterates
|
||||
/// `Unicode.Scalar`s rather than grapheme clusters so a CR and an LF are
|
||||
/// distinct, matching libyaml's byte-oriented scanning, and tracks the mark in
|
||||
/// libyaml's 0-based line and column convention; conversion to the package's
|
||||
/// 1-based `Mark` happens at token emission during integration.
|
||||
struct Reader {
|
||||
private let scalars: [Unicode.Scalar]
|
||||
private(set) var index = 0
|
||||
private(set) var line = 0
|
||||
private(set) var column = 0
|
||||
|
||||
init(_ string: String) {
|
||||
scalars = Array(string.unicodeScalars)
|
||||
}
|
||||
|
||||
/// `PEEK_TOKEN` / `CHECK_AT`: the scalar `offset` ahead, or nil past the
|
||||
/// end (libyaml's `\0`).
|
||||
func peek(_ offset: Int = 0) -> Unicode.Scalar? {
|
||||
let position = index + offset
|
||||
return position < scalars.count ? scalars[position] : nil
|
||||
}
|
||||
|
||||
var isAtEnd: Bool {
|
||||
index >= scalars.count
|
||||
}
|
||||
|
||||
/// The current position. Column is 0-based, as in libyaml.
|
||||
var mark: PureYAML.Parsing.Mark {
|
||||
PureYAML.Parsing.Mark(line: line, column: column, index: index)
|
||||
}
|
||||
|
||||
/// `SKIP`: consume one non-break scalar, advancing the column.
|
||||
mutating func skip() {
|
||||
guard index < scalars.count else {
|
||||
return
|
||||
}
|
||||
index += 1
|
||||
column += 1
|
||||
}
|
||||
|
||||
/// `SKIP_LINE`: consume a line break and start a new line. A CR+LF pair is
|
||||
/// one break, matching libyaml.
|
||||
mutating func skipLine() {
|
||||
if peek() == "\r", peek(1) == "\n" {
|
||||
index += 2
|
||||
} else if CharacterClass.isBreak(peek()) {
|
||||
index += 1
|
||||
} else {
|
||||
return
|
||||
}
|
||||
line += 1
|
||||
column = 0
|
||||
}
|
||||
}
|
||||
}
|
||||
45
Sources/Parsing/LibYAML.Scanner.Anchor.swift
Normal file
45
Sources/Parsing/LibYAML.Scanner.Anchor.swift
Normal file
@ -0,0 +1,45 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
|
||||
/// Port of `yaml_parser_scan_anchor`, widened to YAML 1.2: an anchor (`&name`)
|
||||
/// or alias (`*name`) name is `ns-anchor-char+`, i.e. any printable character
|
||||
/// that is not whitespace, a line break, or a flow indicator (`,[]{}`). libyaml
|
||||
/// restricts the name to alphanumerics, which rejects Unicode and punctuation
|
||||
/// anchor names that YAML 1.2 allows.
|
||||
mutating func scanAnchor(isAnchor: Bool) throws -> PureYAML.Parsing.LibYAML.Token {
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
var name = ""
|
||||
while let character = reader.peek(), isAnchorCharacter(character) {
|
||||
name.unicodeScalars.append(character)
|
||||
reader.skip()
|
||||
}
|
||||
let endMark = reader.mark
|
||||
guard !name.isEmpty else {
|
||||
throw PureYAML.Parsing.LibYAML.ScannerError(
|
||||
context: isAnchor ? "while scanning an anchor" : "while scanning an alias",
|
||||
problem: "did not find expected alphabetic or numeric character",
|
||||
mark: startMark,
|
||||
)
|
||||
}
|
||||
return PureYAML.Parsing.LibYAML.Token(
|
||||
type: isAnchor ? .anchor(name: name) : .alias(name: name),
|
||||
startMark: startMark,
|
||||
endMark: endMark,
|
||||
)
|
||||
}
|
||||
|
||||
/// `ns-anchor-char`: a printable, non-blank, non-break character that is not a
|
||||
/// flow indicator or the byte-order mark.
|
||||
private func isAnchorCharacter(_ scalar: Unicode.Scalar) -> Bool {
|
||||
guard Classify.isPrintable(scalar), !Classify.isBlankZ(scalar), scalar != "\u{FEFF}" else {
|
||||
return false
|
||||
}
|
||||
switch scalar {
|
||||
case ",", "[", "]", "{", "}":
|
||||
return false
|
||||
default:
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
205
Sources/Parsing/LibYAML.Scanner.BlockScalar.swift
Normal file
205
Sources/Parsing/LibYAML.Scanner.BlockScalar.swift
Normal file
@ -0,0 +1,205 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
|
||||
/// The accumulation state for `scan_block_scalar`.
|
||||
private struct BlockState {
|
||||
var string = ""
|
||||
var leadingBreak = ""
|
||||
var trailingBreaks = ""
|
||||
var leadingBlank = false
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_block_scalar`. Scans a literal (`|`) or folded
|
||||
/// (`>`) block scalar: parses the chomping (`+`/`-`) and indentation indicators
|
||||
/// from the header, auto-detects the indentation if not given, then folds the
|
||||
/// content (literal keeps breaks, folded joins equally-indented non-empty lines
|
||||
/// with a space) and applies clip/strip/keep chomping to the trailing breaks.
|
||||
mutating func scanBlockScalar(literal: Bool) throws -> PureYAML.Parsing.LibYAML.Token {
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
let header = try readBlockScalarHeader(startMark: startMark)
|
||||
try consumeBlockHeaderTail(startMark: startMark)
|
||||
|
||||
var endMark = reader.mark
|
||||
// -1 is the "indentation not yet auto-detected" sentinel. libyaml uses 0,
|
||||
// but a root-level block scalar legitimately detects indentation 0, so 0
|
||||
// must be a representable detected value rather than the sentinel.
|
||||
var indent = -1
|
||||
if header.increment != 0 {
|
||||
indent = indentEngine.indent >= 0 ? indentEngine.indent + header.increment : header.increment
|
||||
}
|
||||
var state = BlockState()
|
||||
try scanBlockScalarBreaks(indent: &indent, state: &state, startMark: startMark, endMark: &endMark)
|
||||
|
||||
while reader.column == indent, reader.peek() != nil {
|
||||
// A root block scalar (indent 0) ends at a `---`/`...` document marker
|
||||
// in column 0. libyaml never needs this because it floors the indent
|
||||
// at 1, but indent 0 lets the loop reach a column-0 marker line.
|
||||
if reader.column == 0, documentMarkerAhead() {
|
||||
break
|
||||
}
|
||||
foldBlockLine(&state, literal: literal)
|
||||
while let character = reader.peek(), !Classify.isBreakZ(character) {
|
||||
state.string.unicodeScalars.append(character)
|
||||
reader.skip()
|
||||
}
|
||||
state.leadingBreak += readBlockLine()
|
||||
try scanBlockScalarBreaks(indent: &indent, state: &state, startMark: startMark, endMark: &endMark)
|
||||
}
|
||||
|
||||
if header.chomping != -1 {
|
||||
state.string += state.leadingBreak
|
||||
}
|
||||
if header.chomping == 1 {
|
||||
state.string += state.trailingBreaks
|
||||
}
|
||||
return PureYAML.Parsing.LibYAML.Token(
|
||||
type: .scalar(value: state.string, style: literal ? .literal : .folded),
|
||||
startMark: startMark,
|
||||
endMark: endMark,
|
||||
)
|
||||
}
|
||||
|
||||
/// Parse the chomping and indentation indicators in either order.
|
||||
private mutating func readBlockScalarHeader(startMark: PureYAML.Parsing.Mark) throws -> (chomping: Int, increment: Int) {
|
||||
var chomping = 0
|
||||
var increment = 0
|
||||
if reader.peek() == "+" || reader.peek() == "-" {
|
||||
chomping = reader.peek() == "+" ? 1 : -1
|
||||
reader.skip()
|
||||
if Classify.isDigit(reader.peek()) {
|
||||
increment = try blockIncrement(startMark: startMark)
|
||||
}
|
||||
} else if Classify.isDigit(reader.peek()) {
|
||||
increment = try blockIncrement(startMark: startMark)
|
||||
if reader.peek() == "+" || reader.peek() == "-" {
|
||||
chomping = reader.peek() == "+" ? 1 : -1
|
||||
reader.skip()
|
||||
}
|
||||
}
|
||||
return (chomping, increment)
|
||||
}
|
||||
|
||||
private mutating func blockIncrement(startMark: PureYAML.Parsing.Mark) throws -> Int {
|
||||
if reader.peek() == "0" {
|
||||
throw blockError("found an indentation indicator equal to 0", startMark)
|
||||
}
|
||||
let value = digitValue(reader.peek())
|
||||
reader.skip()
|
||||
return value
|
||||
}
|
||||
|
||||
/// Eat trailing blanks and a comment to the end of the header line, then the
|
||||
/// line break.
|
||||
private mutating func consumeBlockHeaderTail(startMark: PureYAML.Parsing.Mark) throws {
|
||||
while reader.peek() == " " {
|
||||
reader.skip()
|
||||
}
|
||||
if reader.peek() == "#" {
|
||||
// A comment needs a preceding space, so `>#` (a `#` adjacent to the
|
||||
// block scalar indicator) is malformed, not a comment. libyaml accepts it.
|
||||
if reader.column > 0, let previous = reader.peek(-1), !Classify.isBlank(previous) {
|
||||
throw blockError("found a comment without a preceding space", startMark)
|
||||
}
|
||||
while !Classify.isBreakZ(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
}
|
||||
guard Classify.isBreakZ(reader.peek()) else {
|
||||
throw blockError("did not find expected comment or line break", startMark)
|
||||
}
|
||||
if Classify.isBreak(reader.peek()) {
|
||||
reader.skipLine()
|
||||
}
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_block_scalar_breaks`: eat indentation spaces and
|
||||
/// blank lines, auto-detecting the indentation level when none was given.
|
||||
private mutating func scanBlockScalarBreaks(indent: inout Int, state: inout BlockState, startMark: PureYAML.Parsing.Mark, endMark: inout PureYAML.Parsing.Mark) throws {
|
||||
var maxIndent = 0
|
||||
endMark = reader.mark
|
||||
// A tab is content (or separation) once the column reaches the content
|
||||
// indentation; it is illegal only while still inside the required
|
||||
// indentation. During auto-detection the floor is the parent indent + 1.
|
||||
let tabFloor = indent >= 0 ? indent : max(0, indentEngine.indent + 1)
|
||||
while true {
|
||||
let lineStart = reader.column
|
||||
while indent < 0 || reader.column < indent, reader.peek() == " " {
|
||||
reader.skip()
|
||||
}
|
||||
if reader.column > maxIndent {
|
||||
maxIndent = reader.column
|
||||
}
|
||||
if reader.column < tabFloor, Classify.isTab(reader.peek()) {
|
||||
throw blockError("found a tab character where an indentation space is expected", startMark)
|
||||
}
|
||||
if !Classify.isBreak(reader.peek()) {
|
||||
// A trailing whitespace-only line that ends at end of input without
|
||||
// a break still counts as one line break for keep chomping (as the
|
||||
// test-suite reference does), even with no break byte present.
|
||||
if reader.peek() == nil, reader.column > lineStart {
|
||||
state.trailingBreaks += "\n"
|
||||
}
|
||||
break
|
||||
}
|
||||
state.trailingBreaks += readBlockLine()
|
||||
endMark = reader.mark
|
||||
}
|
||||
// A leading empty line may not be more indented than the first content
|
||||
// line; libyaml silently detects the deeper indent and drops the content.
|
||||
if indent < 0, !Classify.isBreakZ(reader.peek()), reader.column < maxIndent {
|
||||
throw blockError("found a leading empty line more indented than the content", startMark)
|
||||
}
|
||||
if indent < 0 {
|
||||
// Auto-detected indentation. A root-level block scalar has a parent
|
||||
// indent of -1, so the floor is 0 (allowing column-0 content), not
|
||||
// libyaml's 1. maxIndent >= 0 keeps the result non-negative.
|
||||
indent = max(maxIndent, indentEngine.indent + 1)
|
||||
}
|
||||
}
|
||||
|
||||
/// Fold the pending breaks at the start of a content line (folded style joins a
|
||||
/// single break between non-blank lines into a space; otherwise breaks are
|
||||
/// kept).
|
||||
private func foldBlockLine(_ state: inout BlockState, literal: Bool) {
|
||||
let trailingBlank = Classify.isBlank(reader.peek())
|
||||
if !literal, state.leadingBreak.first == "\n", !state.leadingBlank, !trailingBlank {
|
||||
if state.trailingBreaks.isEmpty {
|
||||
state.string += " "
|
||||
}
|
||||
state.leadingBreak = ""
|
||||
} else {
|
||||
state.string += state.leadingBreak
|
||||
state.leadingBreak = ""
|
||||
}
|
||||
state.string += state.trailingBreaks
|
||||
state.trailingBreaks = ""
|
||||
state.leadingBlank = Classify.isBlank(reader.peek())
|
||||
}
|
||||
|
||||
private mutating func readBlockLine() -> String {
|
||||
reader.skipLine()
|
||||
return "\n"
|
||||
}
|
||||
|
||||
/// Whether a `---` or `...` document marker (three repeats then a blank, break,
|
||||
/// or end of input) starts at the current position.
|
||||
private func documentMarkerAhead() -> Bool {
|
||||
let first = reader.peek()
|
||||
guard first == "-" || first == "." else {
|
||||
return false
|
||||
}
|
||||
return reader.peek(1) == first && reader.peek(2) == first && Classify.isBlankZ(reader.peek(3))
|
||||
}
|
||||
|
||||
private func digitValue(_ scalar: Unicode.Scalar?) -> Int {
|
||||
guard let scalar, scalar >= "0", scalar <= "9" else {
|
||||
return 0
|
||||
}
|
||||
return Int(scalar.value - Unicode.Scalar("0").value)
|
||||
}
|
||||
|
||||
private func blockError(_ problem: String, _ mark: PureYAML.Parsing.Mark) -> PureYAML.Parsing.LibYAML.ScannerError {
|
||||
PureYAML.Parsing.LibYAML.ScannerError(context: "while scanning a block scalar", problem: problem, mark: mark)
|
||||
}
|
||||
}
|
||||
153
Sources/Parsing/LibYAML.Scanner.Directive.swift
Normal file
153
Sources/Parsing/LibYAML.Scanner.Directive.swift
Normal file
@ -0,0 +1,153 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
private typealias Token = PureYAML.Parsing.LibYAML.Token
|
||||
private typealias Mark = PureYAML.Parsing.Mark
|
||||
|
||||
/// The longest version number libyaml accepts (`MAX_NUMBER_LENGTH`).
|
||||
private var maxVersionNumberLength: Int {
|
||||
9
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_directive`: reset the indentation and simple keys,
|
||||
/// then scan a `%YAML` or `%TAG` directive token.
|
||||
mutating func fetchDirective() throws {
|
||||
indentEngine.unrollIndent(column: -1, mark: reader.mark)
|
||||
try removeSimpleKey()
|
||||
simpleKeyAllowed = false
|
||||
let token = try scanDirective()
|
||||
indentEngine.enqueue(token)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_directive`, extended to YAML 1.2 §6.8.1: a `%YAML`
|
||||
/// or `%TAG` directive produces its token; any other (reserved) directive is
|
||||
/// ignored with a warning, producing a `reservedDirective` token whose content
|
||||
/// the parser drops, rather than libyaml's hard rejection. The reserved token
|
||||
/// still marks the document as a directives document, so it must be followed by
|
||||
/// a `---`.
|
||||
private mutating func scanDirective() throws -> Token {
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
let name = try scanDirectiveName(startMark: startMark)
|
||||
if name == "YAML" {
|
||||
let version = try scanVersionDirectiveValue(startMark: startMark)
|
||||
let token = Token(type: .versionDirective(major: version.major, minor: version.minor), startMark: startMark, endMark: reader.mark)
|
||||
try consumeDirectiveTail(startMark: startMark)
|
||||
return token
|
||||
}
|
||||
if name == "TAG" {
|
||||
let value = try scanTagDirectiveValue(startMark: startMark)
|
||||
let token = Token(type: .tagDirective(handle: value.handle, prefix: value.prefix), startMark: startMark, endMark: reader.mark)
|
||||
try consumeDirectiveTail(startMark: startMark)
|
||||
return token
|
||||
}
|
||||
skipReservedDirectiveLine()
|
||||
return Token(type: .reservedDirective, startMark: startMark, endMark: reader.mark)
|
||||
}
|
||||
|
||||
/// Consume a reserved directive's parameters through the end of the line; the
|
||||
/// whole line is ignored per YAML 1.2 §6.8.1.
|
||||
private mutating func skipReservedDirectiveLine() {
|
||||
while !Classify.isBreakZ(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
if Classify.isBreak(reader.peek()) {
|
||||
reader.skipLine()
|
||||
}
|
||||
}
|
||||
|
||||
/// Eat trailing blanks and an optional comment, require a line break, eat it.
|
||||
private mutating func consumeDirectiveTail(startMark: Mark) throws {
|
||||
while Classify.isBlank(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
if reader.peek() == "#" {
|
||||
while !Classify.isBreakZ(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
}
|
||||
guard Classify.isBreakZ(reader.peek()) else {
|
||||
throw directiveError("while scanning a directive", "did not find expected comment or line break", startMark)
|
||||
}
|
||||
if Classify.isBreak(reader.peek()) {
|
||||
reader.skipLine()
|
||||
}
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_directive_name`: the alphanumeric name after `%`,
|
||||
/// which must be non-empty and followed by a blank or line break.
|
||||
private mutating func scanDirectiveName(startMark: Mark) throws -> String {
|
||||
var name = ""
|
||||
while let character = reader.peek(), Classify.isAlpha(character) {
|
||||
name.unicodeScalars.append(character)
|
||||
reader.skip()
|
||||
}
|
||||
guard !name.isEmpty else {
|
||||
throw directiveError("while scanning a directive", "could not find expected directive name", startMark)
|
||||
}
|
||||
guard Classify.isBlankZ(reader.peek()) else {
|
||||
throw directiveError("while scanning a directive", "found unexpected non-alphabetical character", startMark)
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_version_directive_value`: `major.minor`.
|
||||
private mutating func scanVersionDirectiveValue(startMark: Mark) throws -> (major: Int, minor: Int) {
|
||||
while Classify.isBlank(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
let major = try scanVersionDirectiveNumber(startMark: startMark)
|
||||
guard reader.peek() == "." else {
|
||||
throw directiveError("while scanning a %YAML directive", "did not find expected digit or '.' character", startMark)
|
||||
}
|
||||
reader.skip()
|
||||
let minor = try scanVersionDirectiveNumber(startMark: startMark)
|
||||
// The version must be followed by separation; a comment needs a space
|
||||
// before its `#`, so `%YAML 1.1#...` (no space) is malformed. libyaml
|
||||
// accepts it, treating the `#` as an adjacent comment.
|
||||
guard Classify.isBlankZ(reader.peek()) else {
|
||||
throw directiveError("while scanning a %YAML directive", "did not find expected digit or '.' character", startMark)
|
||||
}
|
||||
return (major, minor)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_version_directive_number`: a bounded run of digits.
|
||||
private mutating func scanVersionDirectiveNumber(startMark: Mark) throws -> Int {
|
||||
var value = 0
|
||||
var length = 0
|
||||
while let character = reader.peek(), Classify.isDigit(character) {
|
||||
length += 1
|
||||
guard length <= maxVersionNumberLength else {
|
||||
throw directiveError("while scanning a %YAML directive", "found extremely long version number", startMark)
|
||||
}
|
||||
value = value * 10 + Int(character.value - Unicode.Scalar("0").value)
|
||||
reader.skip()
|
||||
}
|
||||
guard length > 0 else {
|
||||
throw directiveError("while scanning a %YAML directive", "did not find expected version number", startMark)
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_tag_directive_value`: `!handle! prefix`.
|
||||
private mutating func scanTagDirectiveValue(startMark: Mark) throws -> (handle: String, prefix: String) {
|
||||
while Classify.isBlank(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
let handle = try scanTagHandle(directive: true, startMark: startMark)
|
||||
guard Classify.isBlank(reader.peek()) else {
|
||||
throw directiveError("while scanning a %TAG directive", "did not find expected whitespace", startMark)
|
||||
}
|
||||
while Classify.isBlank(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
let prefix = try scanTagURI(uriChar: true, directive: true, head: "", startMark: startMark)
|
||||
guard Classify.isBlankZ(reader.peek()) else {
|
||||
throw directiveError("while scanning a %TAG directive", "did not find expected whitespace or line break", startMark)
|
||||
}
|
||||
return (handle, prefix)
|
||||
}
|
||||
|
||||
private func directiveError(_ context: String, _ problem: String, _ mark: Mark) -> PureYAML.Parsing.LibYAML.ScannerError {
|
||||
PureYAML.Parsing.LibYAML.ScannerError(context: context, problem: problem, mark: mark)
|
||||
}
|
||||
}
|
||||
361
Sources/Parsing/LibYAML.Scanner.Fetch.swift
Normal file
361
Sources/Parsing/LibYAML.Scanner.Fetch.swift
Normal file
@ -0,0 +1,361 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
private typealias Token = PureYAML.Parsing.LibYAML.Token
|
||||
private typealias ScannerError = PureYAML.Parsing.LibYAML.ScannerError
|
||||
|
||||
/// Produce every token for the input, mirroring repeated calls to libyaml's
|
||||
/// `yaml_parser_fetch_next_token` until STREAM-END. Tokens are never dequeued,
|
||||
/// so `tokens_parsed` stays 0 and `QUEUE_INSERT` positions are absolute, which
|
||||
/// matches the simple-key splice.
|
||||
mutating func scanAll() throws -> [PureYAML.Parsing.LibYAML.Token] {
|
||||
while !streamEndProduced {
|
||||
try fetchNextToken()
|
||||
}
|
||||
return indentEngine.tokens
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_next_token`: emit STREAM-START first, otherwise
|
||||
/// skip to the next token, expire stale simple keys, unroll the indentation to
|
||||
/// the current column, and dispatch on the leading character. Constructs not
|
||||
/// yet ported throw an explicit error rather than mis-scan.
|
||||
mutating func fetchNextToken() throws {
|
||||
if !streamStartProduced {
|
||||
fetchStreamStart()
|
||||
return
|
||||
}
|
||||
|
||||
try scanToNextToken()
|
||||
try checkFlowContinuationIndent()
|
||||
mark = reader.mark
|
||||
try staleSimpleKeys()
|
||||
indentEngine.unrollIndent(column: reader.column, mark: reader.mark)
|
||||
mark = reader.mark
|
||||
try dispatchToken()
|
||||
}
|
||||
|
||||
/// Dispatch on the leading character: stream end, directives and document
|
||||
/// markers at column 0, then the indicators.
|
||||
private mutating func dispatchToken() throws {
|
||||
let character = reader.peek()
|
||||
if Classify.isZ(character) {
|
||||
fetchStreamEnd()
|
||||
return
|
||||
}
|
||||
if reader.column == 0, character == "%" {
|
||||
try fetchDirective()
|
||||
return
|
||||
}
|
||||
if reader.column == 0, isDocumentMarker("-") {
|
||||
try fetchDocumentIndicator(.documentStart)
|
||||
return
|
||||
}
|
||||
if reader.column == 0, isDocumentMarker(".") {
|
||||
try fetchDocumentIndicator(.documentEnd)
|
||||
return
|
||||
}
|
||||
guard let character else {
|
||||
return
|
||||
}
|
||||
try dispatchIndicator(character)
|
||||
}
|
||||
|
||||
/// Dispatch the block-structure indicators and plain scalars; unported
|
||||
/// constructs throw an explicit error.
|
||||
private mutating func dispatchIndicator(_ character: Unicode.Scalar) throws {
|
||||
if try fetchFlowToken(character) {
|
||||
return
|
||||
}
|
||||
if try fetchNodeToken(character) {
|
||||
return
|
||||
}
|
||||
switch character {
|
||||
case "-" where Classify.isBlankZ(reader.peek(1)):
|
||||
try fetchBlockEntry()
|
||||
case "-" where flowLevel > 0 && Classify.isFlowIndicator(reader.peek(1)):
|
||||
// Per `ns-plain-first` a `-` begins a plain scalar in flow only when a
|
||||
// flow-safe character follows; before a flow indicator (`[-]`, `[-,`)
|
||||
// it cannot start a node. libyaml reads it as the scalar `-`.
|
||||
throw ScannerError(
|
||||
context: "while scanning for the next token",
|
||||
problem: "found unexpected '-' indicator",
|
||||
mark: reader.mark,
|
||||
)
|
||||
case "?" where Classify.isBlankZ(reader.peek(1)) || (flowLevel > 0 && Classify.isFlowIndicator(reader.peek(1))):
|
||||
// In flow context libyaml treats every `?` as a key indicator, but per
|
||||
// `ns-plain-first` a `?` followed by a flow-safe character (anything but
|
||||
// whitespace, end, or a flow indicator) begins a plain scalar (`?foo`).
|
||||
try fetchKey()
|
||||
case ":" where Classify.isBlankZ(reader.peek(1))
|
||||
|| (flowLevel > 0 && (Classify.isFlowIndicator(reader.peek(1)) || flowKeyIsJSON)):
|
||||
// In flow context libyaml makes every `:` a value indicator. Per the
|
||||
// spec a `:` only ends a node when followed by whitespace, end of
|
||||
// input, or a flow indicator, except an adjacent `:` after a JSON
|
||||
// (quoted) key. Otherwise the `:` is plain-scalar content (`:x`).
|
||||
try fetchValue()
|
||||
default:
|
||||
// A plain scalar may begin with any non-blank character; a blank (a
|
||||
// tab here, since spaces and breaks were already consumed) cannot
|
||||
// start a token, matching libyaml's final guard in fetch_next_token.
|
||||
guard !Classify.isBlankZ(character) else {
|
||||
throw ScannerError(
|
||||
context: "while scanning for the next token",
|
||||
problem: "found character that cannot start any token",
|
||||
mark: reader.mark,
|
||||
)
|
||||
}
|
||||
try fetchPlainScalar()
|
||||
}
|
||||
}
|
||||
|
||||
/// Fetch a flow-collection indicator (`[` `]` `{` `}` `,`); returns false
|
||||
/// otherwise.
|
||||
private mutating func fetchFlowToken(_ character: Unicode.Scalar) throws -> Bool {
|
||||
switch character {
|
||||
case "[": fetchFlowCollectionStart(.flowSequenceStart)
|
||||
case "{": fetchFlowCollectionStart(.flowMappingStart)
|
||||
case "]": try fetchFlowCollectionEnd(.flowSequenceEnd)
|
||||
case "}": try fetchFlowCollectionEnd(.flowMappingEnd)
|
||||
case ",": try fetchFlowEntry()
|
||||
default: return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
/// Fetch a quoted scalar, an anchor/alias, or a block scalar; returns false
|
||||
/// otherwise.
|
||||
private mutating func fetchNodeToken(_ character: Unicode.Scalar) throws -> Bool {
|
||||
switch character {
|
||||
case "'": try fetchFlowScalar(single: true)
|
||||
case "\"": try fetchFlowScalar(single: false)
|
||||
case "&": try fetchAnchor(isAnchor: true)
|
||||
case "*": try fetchAnchor(isAnchor: false)
|
||||
case "!": try fetchTag()
|
||||
case "|" where flowLevel == 0: try fetchBlockScalar(literal: true)
|
||||
case ">" where flowLevel == 0: try fetchBlockScalar(literal: false)
|
||||
default: return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_tag`.
|
||||
private mutating func fetchTag() throws {
|
||||
mark = reader.mark
|
||||
try saveSimpleKey()
|
||||
simpleKeyAllowed = false
|
||||
let token = try scanTag()
|
||||
indentEngine.enqueue(token)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_block_scalar`.
|
||||
private mutating func fetchBlockScalar(literal: Bool) throws {
|
||||
try removeSimpleKey()
|
||||
simpleKeyAllowed = true
|
||||
let token = try scanBlockScalar(literal: literal)
|
||||
indentEngine.enqueue(token)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_flow_scalar`.
|
||||
private mutating func fetchFlowScalar(single: Bool) throws {
|
||||
mark = reader.mark
|
||||
let saved = simpleKeyAllowed
|
||||
try saveSimpleKey()
|
||||
// A quoted scalar that became this level's simple key is a JSON node, so a
|
||||
// later adjacent `:` (no space) still opens its value.
|
||||
if saved {
|
||||
simpleKeys[simpleKeys.count - 1].isJSON = true
|
||||
}
|
||||
simpleKeyAllowed = false
|
||||
let token = try scanFlowScalar(single: single)
|
||||
indentEngine.enqueue(token)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_anchor`.
|
||||
private mutating func fetchAnchor(isAnchor: Bool) throws {
|
||||
mark = reader.mark
|
||||
try saveSimpleKey()
|
||||
simpleKeyAllowed = false
|
||||
let token = try scanAnchor(isAnchor: isAnchor)
|
||||
indentEngine.enqueue(token)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_flow_collection_start`.
|
||||
private mutating func fetchFlowCollectionStart(_ type: PureYAML.Parsing.LibYAML.TokenType) {
|
||||
// saveSimpleKey cannot throw in flow: a required key needs block context.
|
||||
let saved = simpleKeyAllowed
|
||||
try? saveSimpleKey()
|
||||
// A flow collection is a JSON node, so when it is this level's simple key
|
||||
// an adjacent `:` after its closing `]`/`}` still opens its value.
|
||||
if saved {
|
||||
simpleKeys[simpleKeys.count - 1].isJSON = true
|
||||
}
|
||||
increaseFlowLevel(isSequence: type == .flowSequenceStart)
|
||||
simpleKeyAllowed = true
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
indentEngine.enqueue(Token(type: type, startMark: startMark, endMark: reader.mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_flow_collection_end`.
|
||||
private mutating func fetchFlowCollectionEnd(_ type: PureYAML.Parsing.LibYAML.TokenType) throws {
|
||||
try removeSimpleKey()
|
||||
decreaseFlowLevel()
|
||||
simpleKeyAllowed = false
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
indentEngine.enqueue(Token(type: type, startMark: startMark, endMark: reader.mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_flow_entry`.
|
||||
private mutating func fetchFlowEntry() throws {
|
||||
try removeSimpleKey()
|
||||
simpleKeyAllowed = true
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
indentEngine.enqueue(Token(type: .flowEntry, startMark: startMark, endMark: reader.mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_stream_start`.
|
||||
private mutating func fetchStreamStart() {
|
||||
// indent is already -1 and simpleKeys already holds the block-context slot.
|
||||
simpleKeyAllowed = true
|
||||
streamStartProduced = true
|
||||
indentEngine.enqueue(Token(type: .streamStart, startMark: reader.mark, endMark: reader.mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_stream_end`.
|
||||
private mutating func fetchStreamEnd() {
|
||||
if reader.column != 0 {
|
||||
// Force a new line in the mark, as libyaml does.
|
||||
mark = PureYAML.Parsing.Mark(line: reader.line + 1, column: 0, index: reader.index)
|
||||
}
|
||||
indentEngine.unrollIndent(column: -1, mark: mark)
|
||||
// remove_simple_key cannot throw here: a required key would already have
|
||||
// been resolved before stream end in the block + plain core.
|
||||
simpleKeys[simpleKeys.count - 1].possible = false
|
||||
simpleKeyAllowed = false
|
||||
streamEndProduced = true
|
||||
indentEngine.enqueue(Token(type: .streamEnd, startMark: mark, endMark: mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_document_indicator`.
|
||||
private mutating func fetchDocumentIndicator(_ type: PureYAML.Parsing.LibYAML.TokenType) throws {
|
||||
indentEngine.unrollIndent(column: -1, mark: reader.mark)
|
||||
simpleKeys[simpleKeys.count - 1].possible = false
|
||||
simpleKeyAllowed = false
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
indentEngine.enqueue(Token(type: type, startMark: startMark, endMark: reader.mark))
|
||||
if case .documentEnd = type {
|
||||
// Only whitespace and a comment may follow `...` on its line; a `--- x`
|
||||
// document start can carry inline content, but `... x` cannot.
|
||||
while reader.peek() == " " || Classify.isTab(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
if let next = reader.peek(), next != "#", !Classify.isBreak(next) {
|
||||
throw ScannerError(
|
||||
context: "while scanning a document end marker",
|
||||
problem: "found content after the document end marker",
|
||||
mark: reader.mark,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_block_entry`.
|
||||
private mutating func fetchBlockEntry() throws {
|
||||
if tabbedSeparation {
|
||||
throw tabIndentationError()
|
||||
}
|
||||
// A block sequence must begin its own line. A `-` mid-line after a mapping
|
||||
// value (`key: - a`) or a node property (`&a - x`) is invalid, but a `-`
|
||||
// after another `-` is a valid compact sequence (`- - a`). libyaml accepts
|
||||
// all three.
|
||||
if flowLevel == 0, !crossedLineBreak, lastTokenForbidsInlineSequence() {
|
||||
throw ScannerError(
|
||||
context: "while scanning a node",
|
||||
problem: "found a block sequence on the same line as an earlier node",
|
||||
mark: reader.mark,
|
||||
)
|
||||
}
|
||||
if flowLevel == 0 {
|
||||
indentEngine.rollIndent(column: reader.column, number: -1, type: .blockSequenceStart, mark: reader.mark)
|
||||
}
|
||||
simpleKeys[simpleKeys.count - 1].possible = false
|
||||
simpleKeyAllowed = true
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
indentEngine.enqueue(Token(type: .blockEntry, startMark: startMark, endMark: reader.mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_key`.
|
||||
private mutating func fetchKey() throws {
|
||||
if tabbedSeparation {
|
||||
throw tabIndentationError()
|
||||
}
|
||||
if flowLevel == 0 {
|
||||
indentEngine.rollIndent(column: reader.column, number: -1, type: .blockMappingStart, mark: reader.mark)
|
||||
}
|
||||
simpleKeys[simpleKeys.count - 1].possible = false
|
||||
simpleKeyAllowed = flowLevel == 0
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
indentEngine.enqueue(Token(type: .key, startMark: startMark, endMark: reader.mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_value`, including the simple-key splice: a
|
||||
/// possible simple key inserts its `KEY` token at the recorded position and
|
||||
/// rolls the block mapping indent to the key's column.
|
||||
private mutating func fetchValue() throws {
|
||||
let top = simpleKeys.count - 1
|
||||
// A pending simple key means this `:` closes an inline implicit key (key
|
||||
// and `:` on one line). An explicit `?` key's `:` has no pending simple key
|
||||
// (it went stale across the line break), so its value may open a sequence.
|
||||
lastValueWasInline = simpleKeys[top].possible
|
||||
// A flow-sequence single-pair implicit key must be single-line, so its `:`
|
||||
// must sit on the key's line (`[ "k"\n :v ]` and `[ k\n : v ]` are both
|
||||
// invalid); a flow-mapping adjacent value may span lines (`{ "k"\n :v }`).
|
||||
if simpleKeys[top].possible, flowIsSequence.last == true, simpleKeys[top].mark.line != mark.line {
|
||||
throw ScannerError(
|
||||
context: "while scanning a simple key",
|
||||
problem: "could not find expected ':'",
|
||||
mark: simpleKeys[top].mark,
|
||||
)
|
||||
}
|
||||
if simpleKeys[top].possible {
|
||||
if simpleKeys[top].tabbed {
|
||||
throw tabIndentationError()
|
||||
}
|
||||
let key = simpleKeys[top]
|
||||
indentEngine.insert(Token(type: .key, startMark: key.mark, endMark: key.mark), atTokenNumber: key.tokenNumber)
|
||||
indentEngine.rollIndent(column: key.mark.column, number: key.tokenNumber, type: .blockMappingStart, mark: key.mark)
|
||||
simpleKeys[top].possible = false
|
||||
simpleKeyAllowed = false
|
||||
} else {
|
||||
if flowLevel == 0 {
|
||||
indentEngine.rollIndent(column: reader.column, number: -1, type: .blockMappingStart, mark: reader.mark)
|
||||
}
|
||||
simpleKeyAllowed = flowLevel == 0
|
||||
}
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
indentEngine.enqueue(Token(type: .value, startMark: startMark, endMark: reader.mark))
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_fetch_plain_scalar`.
|
||||
private mutating func fetchPlainScalar() throws {
|
||||
mark = reader.mark
|
||||
try saveSimpleKey()
|
||||
simpleKeyAllowed = false
|
||||
let token = try scanPlainScalar()
|
||||
indentEngine.enqueue(token)
|
||||
}
|
||||
|
||||
private func isDocumentMarker(_ character: Unicode.Scalar) -> Bool {
|
||||
reader.peek(0) == character
|
||||
&& reader.peek(1) == character
|
||||
&& reader.peek(2) == character
|
||||
&& Classify.isBlankZ(reader.peek(3))
|
||||
}
|
||||
}
|
||||
225
Sources/Parsing/LibYAML.Scanner.FlowScalar.swift
Normal file
225
Sources/Parsing/LibYAML.Scanner.FlowScalar.swift
Normal file
@ -0,0 +1,225 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
|
||||
/// The folding state carried through `scan_flow_scalar`, the same shape as the
|
||||
/// plain-scalar state.
|
||||
private struct FlowState {
|
||||
var string = ""
|
||||
var leadingBreak = ""
|
||||
var trailingBreaks = ""
|
||||
var whitespaces = ""
|
||||
var leadingBlanks = false
|
||||
}
|
||||
|
||||
/// Decoded single-character escapes for double-quoted scalars (libyaml's escape
|
||||
/// switch). libyaml emits UTF-8 bytes; here each maps to its Unicode scalar.
|
||||
private static let doubleQuotedEscapes: [Unicode.Scalar: Unicode.Scalar] = [
|
||||
"0": "\u{0}", "a": "\u{07}", "b": "\u{08}", "t": "\u{09}", "\t": "\u{09}",
|
||||
"n": "\u{0A}", "v": "\u{0B}", "f": "\u{0C}", "r": "\u{0D}", "e": "\u{1B}",
|
||||
" ": "\u{20}", "\"": "\"", "/": "/", "\\": "\\",
|
||||
"N": "\u{85}", "_": "\u{A0}", "L": "\u{2028}", "P": "\u{2029}",
|
||||
]
|
||||
|
||||
/// Port of `yaml_parser_scan_flow_scalar`. Scans a single- or double-quoted
|
||||
/// scalar, folding line breaks like a plain scalar and, for double quotes,
|
||||
/// decoding escape sequences.
|
||||
mutating func scanFlowScalar(single: Bool) throws -> PureYAML.Parsing.LibYAML.Token {
|
||||
let startMark = reader.mark
|
||||
reader.skip()
|
||||
var state = FlowState()
|
||||
let quote: Unicode.Scalar = single ? "'" : "\""
|
||||
let indent = indentEngine.indent + 1
|
||||
// A continuation indented below `indent` is an error, but it is recorded
|
||||
// and only raised once the quote is known to close; an unterminated quote
|
||||
// reaches end of input first and reports that instead.
|
||||
var indentError: PureYAML.Parsing.LibYAML.ScannerError?
|
||||
while true {
|
||||
if reader.column == 0, documentMarkerAhead() {
|
||||
throw flowError("found unexpected document indicator", startMark)
|
||||
}
|
||||
if reader.peek() == nil {
|
||||
throw flowError("found unexpected end of stream", startMark)
|
||||
}
|
||||
try consumeFlowContent(single: single, &state, startMark: startMark)
|
||||
if reader.peek() == quote {
|
||||
break
|
||||
}
|
||||
consumeFlowBreaks(&state, indent: indent, quote: quote, startMark: startMark, error: &indentError)
|
||||
foldFlow(&state)
|
||||
}
|
||||
reader.skip()
|
||||
if let indentError {
|
||||
throw indentError
|
||||
}
|
||||
return PureYAML.Parsing.LibYAML.Token(
|
||||
type: .scalar(value: state.string, style: single ? .singleQuoted : .doubleQuoted),
|
||||
startMark: startMark,
|
||||
endMark: reader.mark,
|
||||
)
|
||||
}
|
||||
|
||||
/// Consume the run of non-blank characters, decoding escapes and stopping at
|
||||
/// the closing quote or an escaped line break.
|
||||
private mutating func consumeFlowContent(single: Bool, _ state: inout FlowState, startMark: PureYAML.Parsing.Mark) throws {
|
||||
state.leadingBlanks = false
|
||||
let quote: Unicode.Scalar = single ? "'" : "\""
|
||||
while let character = reader.peek(), !Classify.isBlankZ(character) {
|
||||
if single, character == "'", reader.peek(1) == "'" {
|
||||
state.string.unicodeScalars.append("'")
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
} else if character == quote {
|
||||
return
|
||||
} else if !single, character == "\\", Classify.isBreak(reader.peek(1)) {
|
||||
reader.skip()
|
||||
reader.skipLine()
|
||||
state.leadingBlanks = true
|
||||
return
|
||||
} else if !single, character == "\\" {
|
||||
try appendEscape(&state, startMark: startMark)
|
||||
} else {
|
||||
state.string.unicodeScalars.append(character)
|
||||
reader.skip()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Decode a double-quoted escape sequence (`\` already at the reader).
|
||||
private mutating func appendEscape(_ state: inout FlowState, startMark: PureYAML.Parsing.Mark) throws {
|
||||
let escape = reader.peek(1)
|
||||
if let escape, let decoded = Self.doubleQuotedEscapes[escape] {
|
||||
state.string.unicodeScalars.append(decoded)
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
return
|
||||
}
|
||||
let codeLength = try hexEscapeLength(escape, startMark: startMark)
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
var value = try readHexEscape(count: codeLength, startMark: startMark)
|
||||
if codeLength == 4 {
|
||||
value = try combineSurrogateIfNeeded(high: value, startMark: startMark)
|
||||
}
|
||||
guard let scalar = Unicode.Scalar(value) else {
|
||||
throw flowError("found invalid Unicode character escape code", startMark)
|
||||
}
|
||||
state.string.unicodeScalars.append(scalar)
|
||||
}
|
||||
|
||||
/// The hex digit count for `\x` (2), `\u` (4), and `\U` (8) escapes.
|
||||
private func hexEscapeLength(_ escape: Unicode.Scalar?, startMark: PureYAML.Parsing.Mark) throws -> Int {
|
||||
switch escape {
|
||||
case "x": 2
|
||||
case "u": 4
|
||||
case "U": 8
|
||||
default: throw flowError("found unknown escape character", startMark)
|
||||
}
|
||||
}
|
||||
|
||||
/// Read and consume `count` hexadecimal digits, returning their value.
|
||||
private mutating func readHexEscape(count: Int, startMark: PureYAML.Parsing.Mark) throws -> UInt32 {
|
||||
var value: UInt32 = 0
|
||||
for _ in 0 ..< count {
|
||||
guard let digit = reader.peek(), Classify.isHex(digit) else {
|
||||
throw flowError("did not find expected hexdecimal number", startMark)
|
||||
}
|
||||
value = (value << 4) + hexValue(digit)
|
||||
reader.skip()
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/// Combine a UTF-16 surrogate pair (a high surrogate `\u` escape immediately
|
||||
/// followed by a low surrogate `\u` escape) into a single astral code point,
|
||||
/// for JSON compatibility. A lone high surrogate is left for the scalar check.
|
||||
private mutating func combineSurrogateIfNeeded(high: UInt32, startMark: PureYAML.Parsing.Mark) throws -> UInt32 {
|
||||
guard (0xD800 ... 0xDBFF).contains(high), reader.peek() == "\\", reader.peek(1) == "u" else {
|
||||
return high
|
||||
}
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
let low = try readHexEscape(count: 4, startMark: startMark)
|
||||
guard (0xDC00 ... 0xDFFF).contains(low) else {
|
||||
throw flowError("found invalid Unicode character escape code", startMark)
|
||||
}
|
||||
return 0x10000 + ((high - 0xD800) << 10) + (low - 0xDC00)
|
||||
}
|
||||
|
||||
/// Consume blanks and line breaks between content, recording pending
|
||||
/// whitespace and folded breaks.
|
||||
private mutating func consumeFlowBreaks(
|
||||
_ state: inout FlowState,
|
||||
indent: Int,
|
||||
quote: Unicode.Scalar,
|
||||
startMark: PureYAML.Parsing.Mark,
|
||||
error: inout PureYAML.Parsing.LibYAML.ScannerError?,
|
||||
) {
|
||||
while Classify.isBlank(reader.peek()) || Classify.isBreak(reader.peek()) {
|
||||
if Classify.isBlank(reader.peek()) {
|
||||
if error == nil, state.leadingBlanks, reader.column < indent, Classify.isTab(reader.peek()), !blankLineAhead() {
|
||||
error = flowError("found a tab character that violates indentation", startMark)
|
||||
}
|
||||
if !state.leadingBlanks, let blank = reader.peek() {
|
||||
state.whitespaces.unicodeScalars.append(blank)
|
||||
}
|
||||
reader.skip()
|
||||
} else if !state.leadingBlanks {
|
||||
state.whitespaces = ""
|
||||
reader.skipLine()
|
||||
state.leadingBreak = "\n"
|
||||
state.leadingBlanks = true
|
||||
} else {
|
||||
reader.skipLine()
|
||||
state.trailingBreaks += "\n"
|
||||
}
|
||||
}
|
||||
// The continuation content must be indented at least `indent`; a less
|
||||
// indented (non-closing, non-empty) line is malformed.
|
||||
if error == nil, state.leadingBlanks, let next = reader.peek(), next != quote, !Classify.isBreakZ(next), reader.column < indent {
|
||||
error = flowError("found wrong indentation in a multi-line quoted scalar", startMark)
|
||||
}
|
||||
}
|
||||
|
||||
/// Fold the pending breaks or whitespace into the value.
|
||||
private func foldFlow(_ state: inout FlowState) {
|
||||
guard state.leadingBlanks else {
|
||||
state.string += state.whitespaces
|
||||
state.whitespaces = ""
|
||||
return
|
||||
}
|
||||
if state.leadingBreak.first == "\n" {
|
||||
if state.trailingBreaks.isEmpty {
|
||||
state.string += " "
|
||||
} else {
|
||||
state.string += state.trailingBreaks
|
||||
state.trailingBreaks = ""
|
||||
}
|
||||
state.leadingBreak = ""
|
||||
} else {
|
||||
state.string += state.leadingBreak
|
||||
state.string += state.trailingBreaks
|
||||
state.leadingBreak = ""
|
||||
state.trailingBreaks = ""
|
||||
}
|
||||
}
|
||||
|
||||
private func hexValue(_ scalar: Unicode.Scalar) -> UInt32 {
|
||||
switch scalar {
|
||||
case "0" ... "9": scalar.value - Unicode.Scalar("0").value
|
||||
case "A" ... "F": scalar.value - Unicode.Scalar("A").value + 10
|
||||
default: scalar.value - Unicode.Scalar("a").value + 10
|
||||
}
|
||||
}
|
||||
|
||||
private func flowError(_ problem: String, _ mark: PureYAML.Parsing.Mark) -> PureYAML.Parsing.LibYAML.ScannerError {
|
||||
PureYAML.Parsing.LibYAML.ScannerError(context: "while scanning a quoted scalar", problem: problem, mark: mark)
|
||||
}
|
||||
|
||||
private func documentMarkerAhead() -> Bool {
|
||||
let first = reader.peek()
|
||||
guard first == "-" || first == "." else {
|
||||
return false
|
||||
}
|
||||
return reader.peek(1) == first && reader.peek(2) == first && Classify.isBlankZ(reader.peek(3))
|
||||
}
|
||||
}
|
||||
59
Sources/Parsing/LibYAML.Scanner.Guards.swift
Normal file
59
Sources/Parsing/LibYAML.Scanner.Guards.swift
Normal file
@ -0,0 +1,59 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias ScannerError = PureYAML.Parsing.LibYAML.ScannerError
|
||||
|
||||
/// Reject a flow-collection continuation line that is not indented past the
|
||||
/// enclosing block context. YAML requires `s-flow-line-prefix` to be more
|
||||
/// indented than the parent block; libyaml does not enforce it, so multi-line
|
||||
/// flow content in column 0 (`[a,\nb]` under a key) wrongly parses.
|
||||
mutating func checkFlowContinuationIndent() throws {
|
||||
guard flowLevel > 0, crossedLineBreak, let next = reader.peek() else {
|
||||
return
|
||||
}
|
||||
// A closing `]`/`}` (or a `,` separator) may sit at any indentation; only
|
||||
// flow CONTENT must be indented past the parent block.
|
||||
if next == "]" || next == "}" || next == "," {
|
||||
return
|
||||
}
|
||||
// A tab in the leading whitespace is never valid indentation, even where
|
||||
// spaces of the same width would be (the tab inflates the column).
|
||||
if reader.column <= indentEngine.indent || flowLeadingTab {
|
||||
throw ScannerError(
|
||||
context: "while parsing a flow node",
|
||||
problem: "found insufficiently indented line in a flow collection",
|
||||
mark: reader.mark,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the last emitted token bars a block sequence from opening on the
|
||||
/// same line: a mapping value indicator or a node property (anchor/tag). After
|
||||
/// another block entry a compact sequence is allowed, so that is not barred.
|
||||
func lastTokenForbidsInlineSequence() -> Bool {
|
||||
switch indentEngine.tokens.last?.type {
|
||||
case .anchor, .tag:
|
||||
true
|
||||
case .value:
|
||||
lastValueWasInline
|
||||
default:
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/// The error raised when a tab is used where indentation is required (before a
|
||||
/// block indicator or an implicit mapping key). A tab is valid separation but
|
||||
/// never indentation.
|
||||
func tabIndentationError() -> PureYAML.Parsing.LibYAML.ScannerError {
|
||||
ScannerError(
|
||||
context: "while scanning for the next token",
|
||||
problem: "found a tab character where an indentation space is expected",
|
||||
mark: reader.mark,
|
||||
)
|
||||
}
|
||||
|
||||
/// Whether this flow level's pending simple key is a JSON (quoted) node, which
|
||||
/// lets an adjacent `:` open its value without a separating space.
|
||||
var flowKeyIsJSON: Bool {
|
||||
let key = simpleKeys[simpleKeys.count - 1]
|
||||
return key.possible && key.isJSON
|
||||
}
|
||||
}
|
||||
142
Sources/Parsing/LibYAML.Scanner.PlainScalar.swift
Normal file
142
Sources/Parsing/LibYAML.Scanner.PlainScalar.swift
Normal file
@ -0,0 +1,142 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
|
||||
/// The folding state carried through `scan_plain_scalar`: the accumulated
|
||||
/// value plus the pending leading break, trailing breaks, and whitespace that
|
||||
/// are folded in before the next content character.
|
||||
private struct PlainState {
|
||||
var string = ""
|
||||
var leadingBreak = ""
|
||||
var trailingBreaks = ""
|
||||
var whitespaces = ""
|
||||
var leadingBlanks = false
|
||||
var endMark: PureYAML.Parsing.Mark
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_plain_scalar`. Consumes a plain scalar, folding
|
||||
/// line breaks: a single break between non-empty lines becomes a space, blank
|
||||
/// lines become literal newlines, and leading/trailing whitespace on folded
|
||||
/// lines is dropped. Ends at a `: ` value indicator, a comment, a less-indented
|
||||
/// line (block context), a flow indicator (flow context), or a document marker.
|
||||
mutating func scanPlainScalar() throws -> PureYAML.Parsing.LibYAML.Token {
|
||||
let indent = indentEngine.indent + 1
|
||||
let startMark = reader.mark
|
||||
var state = PlainState(endMark: reader.mark)
|
||||
|
||||
while true {
|
||||
if reader.column == 0, documentMarkerAhead() {
|
||||
break
|
||||
}
|
||||
if reader.peek() == "#" {
|
||||
break
|
||||
}
|
||||
try consumePlainContent(&state, startMark: startMark)
|
||||
guard Classify.isBlank(reader.peek()) || Classify.isBreak(reader.peek()) else {
|
||||
break
|
||||
}
|
||||
try consumePlainBreaks(&state, indent: indent, startMark: startMark)
|
||||
if flowLevel == 0, reader.column < indent {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if state.leadingBlanks {
|
||||
simpleKeyAllowed = true
|
||||
}
|
||||
return PureYAML.Parsing.LibYAML.Token(
|
||||
type: .scalar(value: state.string, style: .plain),
|
||||
startMark: startMark,
|
||||
endMark: state.endMark,
|
||||
)
|
||||
}
|
||||
|
||||
/// Consume the run of non-blank characters, folding any pending breaks or
|
||||
/// whitespace ahead of each one.
|
||||
private mutating func consumePlainContent(_ state: inout PlainState, startMark _: PureYAML.Parsing.Mark) throws {
|
||||
while let character = reader.peek(), !Classify.isBlankZ(character) {
|
||||
if character == ":", Classify.isBlankZ(reader.peek(1)) {
|
||||
break
|
||||
}
|
||||
if flowLevel > 0, character == ":", Classify.isFlowIndicator(reader.peek(1)) {
|
||||
// A `:` before a flow indicator ends the key; the value is empty.
|
||||
// libyaml rejects this as an unexpected ':', but it is a valid
|
||||
// value indicator in flow context.
|
||||
break
|
||||
}
|
||||
if flowLevel > 0, Classify.isFlowIndicator(character) {
|
||||
break
|
||||
}
|
||||
foldPending(&state)
|
||||
state.string.unicodeScalars.append(character)
|
||||
reader.skip()
|
||||
state.endMark = reader.mark
|
||||
}
|
||||
}
|
||||
|
||||
/// Fold the pending leading break, trailing breaks, and whitespace into the
|
||||
/// accumulated value (the join logic from libyaml's inner loop).
|
||||
private func foldPending(_ state: inout PlainState) {
|
||||
guard state.leadingBlanks || !state.whitespaces.isEmpty else {
|
||||
return
|
||||
}
|
||||
guard state.leadingBlanks else {
|
||||
state.string += state.whitespaces
|
||||
state.whitespaces = ""
|
||||
return
|
||||
}
|
||||
if state.leadingBreak.first == "\n" {
|
||||
if state.trailingBreaks.isEmpty {
|
||||
state.string += " "
|
||||
} else {
|
||||
state.string += state.trailingBreaks
|
||||
state.trailingBreaks = ""
|
||||
}
|
||||
state.leadingBreak = ""
|
||||
} else {
|
||||
state.string += state.leadingBreak
|
||||
state.string += state.trailingBreaks
|
||||
state.leadingBreak = ""
|
||||
state.trailingBreaks = ""
|
||||
}
|
||||
state.leadingBlanks = false
|
||||
}
|
||||
|
||||
/// Consume the run of blanks and line breaks between content, recording them as
|
||||
/// pending whitespace or folded breaks.
|
||||
private mutating func consumePlainBreaks(_ state: inout PlainState, indent: Int, startMark: PureYAML.Parsing.Mark) throws {
|
||||
while Classify.isBlank(reader.peek()) || Classify.isBreak(reader.peek()) {
|
||||
if Classify.isBlank(reader.peek()) {
|
||||
if state.leadingBlanks, reader.column < indent, Classify.isTab(reader.peek()), !blankLineAhead() {
|
||||
// A tab below the indentation is illegal only when it indents
|
||||
// real content; on an otherwise-blank line it is just a blank
|
||||
// line the scalar folds over. libyaml rejects it either way.
|
||||
throw scannerError("found a tab character that violates indentation", at: startMark)
|
||||
}
|
||||
if !state.leadingBlanks, let blank = reader.peek() {
|
||||
state.whitespaces.unicodeScalars.append(blank)
|
||||
}
|
||||
reader.skip()
|
||||
} else if !state.leadingBlanks {
|
||||
state.whitespaces = ""
|
||||
reader.skipLine()
|
||||
state.leadingBreak = "\n"
|
||||
state.leadingBlanks = true
|
||||
} else {
|
||||
reader.skipLine()
|
||||
state.trailingBreaks += "\n"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func scannerError(_ problem: String, at mark: PureYAML.Parsing.Mark) -> PureYAML.Parsing.LibYAML.ScannerError {
|
||||
PureYAML.Parsing.LibYAML.ScannerError(context: "while scanning a plain scalar", problem: problem, mark: mark)
|
||||
}
|
||||
|
||||
private func documentMarkerAhead() -> Bool {
|
||||
let first = reader.peek()
|
||||
guard first == "-" || first == "." else {
|
||||
return false
|
||||
}
|
||||
return reader.peek(1) == first && reader.peek(2) == first && Classify.isBlankZ(reader.peek(3))
|
||||
}
|
||||
}
|
||||
88
Sources/Parsing/LibYAML.Scanner.SkipToToken.swift
Normal file
88
Sources/Parsing/LibYAML.Scanner.SkipToToken.swift
Normal file
@ -0,0 +1,88 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
private typealias ScannerError = PureYAML.Parsing.LibYAML.ScannerError
|
||||
|
||||
/// Port of `yaml_parser_scan_to_next_token`: skip a leading BOM, spaces and
|
||||
/// tabs, a comment, and a single line break, which re-allows a simple key in
|
||||
/// the block context. A tab is valid in-line separation, so it is skipped
|
||||
/// here; `tabbedSeparation` records when one preceded the token so a block
|
||||
/// indicator or implicit key (where the tab would be indentation) can reject.
|
||||
mutating func scanToNextToken() throws {
|
||||
crossedLineBreak = false
|
||||
flowLeadingTab = false
|
||||
while true {
|
||||
tabbedSeparation = false
|
||||
if reader.column == 0, reader.peek() == "\u{FEFF}" {
|
||||
reader.skip()
|
||||
}
|
||||
while reader.peek() == " " || Classify.isTab(reader.peek()) {
|
||||
if Classify.isTab(reader.peek()) {
|
||||
noteLeadingTab()
|
||||
}
|
||||
reader.skip()
|
||||
}
|
||||
// YAML 1.2: a line containing only whitespace is blank even when that
|
||||
// whitespace includes tabs. libyaml leaves a leading tab in place and
|
||||
// later rejects it; instead consume the trailing blanks so the line
|
||||
// break below is reached and the blank line is skipped.
|
||||
if blankLineAhead() {
|
||||
while reader.peek() == " " || Classify.isTab(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
}
|
||||
if reader.peek() == "#" {
|
||||
try consumeComment()
|
||||
}
|
||||
if Classify.isBreak(reader.peek()) {
|
||||
reader.skipLine()
|
||||
crossedLineBreak = true
|
||||
if flowLevel == 0 {
|
||||
simpleKeyAllowed = true
|
||||
}
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Record what a tab in the leading whitespace implies. In block context with a
|
||||
/// simple key allowed it is potential indentation before an indicator. In flow
|
||||
/// context, below the continuation indent on a non-blank line it is indentation
|
||||
/// abuse (a tab is never indentation); at or above the indent it is separation.
|
||||
private mutating func noteLeadingTab() {
|
||||
if flowLevel == 0, simpleKeyAllowed {
|
||||
tabbedSeparation = true
|
||||
}
|
||||
if flowLevel > 0, crossedLineBreak, reader.column < indentEngine.indent + 1, !blankLineAhead() {
|
||||
flowLeadingTab = true
|
||||
}
|
||||
}
|
||||
|
||||
/// Consume a comment to the end of the line. A comment must be preceded by
|
||||
/// whitespace or start the line; a `#` adjacent to the previous token (e.g.
|
||||
/// `"x"#`, `]#`, `,#`) is not a comment. The plain-scalar scanner consumes its
|
||||
/// own trailing blanks, so the character just before the `#` is the reliable
|
||||
/// signal. libyaml accepts the adjacent form.
|
||||
private mutating func consumeComment() throws {
|
||||
if reader.column > 0, let previous = reader.peek(-1), !Classify.isBlank(previous) {
|
||||
throw ScannerError(
|
||||
context: "while scanning for the next token",
|
||||
problem: "found a comment without a preceding space",
|
||||
mark: reader.mark,
|
||||
)
|
||||
}
|
||||
while !Classify.isBreakZ(reader.peek()) {
|
||||
reader.skip()
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether the rest of the current line, from the current position, is only
|
||||
/// spaces and tabs up to a line break or end of stream (a blank line).
|
||||
func blankLineAhead() -> Bool {
|
||||
var offset = 0
|
||||
while let character = reader.peek(offset), character == " " || Classify.isTab(character) {
|
||||
offset += 1
|
||||
}
|
||||
return Classify.isBreakZ(reader.peek(offset))
|
||||
}
|
||||
}
|
||||
155
Sources/Parsing/LibYAML.Scanner.Tag.swift
Normal file
155
Sources/Parsing/LibYAML.Scanner.Tag.swift
Normal file
@ -0,0 +1,155 @@
|
||||
extension PureYAML.Parsing.LibYAML.Scanner {
|
||||
private typealias Classify = PureYAML.Parsing.LibYAML.CharacterClass
|
||||
|
||||
/// Port of `yaml_parser_scan_tag`. Scans a tag in verbatim `!<uri>`, shorthand
|
||||
/// `!handle!suffix`, or `!suffix` form, producing the handle and suffix; the
|
||||
/// parser resolves them against the tag directives.
|
||||
mutating func scanTag() throws -> PureYAML.Parsing.LibYAML.Token {
|
||||
let startMark = reader.mark
|
||||
var handle = ""
|
||||
var suffix = ""
|
||||
if reader.peek(1) == "<" {
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
suffix = try scanTagURI(uriChar: true, directive: false, head: "", startMark: startMark)
|
||||
guard reader.peek() == ">" else {
|
||||
throw tagError("did not find the expected '>'", startMark)
|
||||
}
|
||||
reader.skip()
|
||||
} else {
|
||||
handle = try scanTagHandle(directive: false, startMark: startMark)
|
||||
if handle.first == "!", handle.count > 1, handle.last == "!" {
|
||||
suffix = try scanTagURI(uriChar: false, directive: false, head: "", startMark: startMark)
|
||||
} else {
|
||||
suffix = try scanTagURI(uriChar: false, directive: false, head: handle, startMark: startMark)
|
||||
handle = "!"
|
||||
if suffix.isEmpty {
|
||||
swap(&handle, &suffix)
|
||||
}
|
||||
}
|
||||
}
|
||||
guard Classify.isBlankZ(reader.peek()) || (flowLevel > 0 && reader.peek() == ",") else {
|
||||
throw tagError("did not find expected whitespace or line break", startMark)
|
||||
}
|
||||
return PureYAML.Parsing.LibYAML.Token(type: .tag(handle: handle, suffix: suffix), startMark: startMark, endMark: reader.mark)
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_tag_handle`. A handle is `!`, optionally followed
|
||||
/// by alphanumeric characters and a closing `!`.
|
||||
mutating func scanTagHandle(directive: Bool, startMark: PureYAML.Parsing.Mark) throws -> String {
|
||||
guard reader.peek() == "!" else {
|
||||
throw tagError("did not find expected '!'", startMark)
|
||||
}
|
||||
var handle = "!"
|
||||
reader.skip()
|
||||
while let character = reader.peek(), Classify.isAlpha(character) {
|
||||
handle.unicodeScalars.append(character)
|
||||
reader.skip()
|
||||
}
|
||||
if reader.peek() == "!" {
|
||||
handle.unicodeScalars.append("!")
|
||||
reader.skip()
|
||||
} else if directive, handle != "!" {
|
||||
throw tagError("did not find expected '!'", startMark)
|
||||
}
|
||||
return handle
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_tag_uri`. Consumes the URI character set (plus the
|
||||
/// flow indicators inside a verbatim tag), decoding `%`-escapes.
|
||||
mutating func scanTagURI(uriChar: Bool, directive: Bool, head: String, startMark: PureYAML.Parsing.Mark) throws -> String {
|
||||
var string = head.count > 1 ? String(head.dropFirst()) : ""
|
||||
var length = head.isEmpty ? 0 : head.count
|
||||
while let character = reader.peek(), isURICharacter(character, uriChar: uriChar) {
|
||||
if character == "%" {
|
||||
try scanURIEscapes(directive: directive, startMark: startMark, into: &string)
|
||||
} else {
|
||||
string.unicodeScalars.append(character)
|
||||
reader.skip()
|
||||
}
|
||||
length += 1
|
||||
}
|
||||
guard length > 0 else {
|
||||
throw tagError("did not find expected tag URI", startMark)
|
||||
}
|
||||
return string
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_scan_uri_escapes`. Decodes one UTF-8 character from a
|
||||
/// run of `%XX` octets.
|
||||
private mutating func scanURIEscapes(directive _: Bool, startMark: PureYAML.Parsing.Mark, into string: inout String) throws {
|
||||
var bytes: [UInt8] = []
|
||||
var width = 0
|
||||
repeat {
|
||||
guard reader.peek() == "%", let high = reader.peek(1), Classify.isHex(high), let low = reader.peek(2), Classify.isHex(low) else {
|
||||
throw tagError("did not find URI escaped octet", startMark)
|
||||
}
|
||||
let octet = UInt8((hexDigit(high) << 4) + hexDigit(low))
|
||||
if width == 0 {
|
||||
width = utf8Width(of: octet)
|
||||
guard width > 0 else {
|
||||
throw tagError("found an incorrect leading UTF-8 octet", startMark)
|
||||
}
|
||||
} else if octet & 0xC0 != 0x80 {
|
||||
throw tagError("found an incorrect trailing UTF-8 octet", startMark)
|
||||
}
|
||||
bytes.append(octet)
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
reader.skip()
|
||||
width -= 1
|
||||
} while width > 0
|
||||
guard let scalar = Unicode.Scalar(decodeUTF8(bytes)) else {
|
||||
throw tagError("found invalid UTF-8 in a URI escape", startMark)
|
||||
}
|
||||
string.unicodeScalars.append(scalar)
|
||||
}
|
||||
|
||||
/// Assemble a code point from the validated 1-to-4 UTF-8 octets of one escape.
|
||||
private func decodeUTF8(_ bytes: [UInt8]) -> UInt32 {
|
||||
switch bytes.count {
|
||||
case 1:
|
||||
UInt32(bytes[0])
|
||||
case 2:
|
||||
(UInt32(bytes[0] & 0x1F) << 6) | UInt32(bytes[1] & 0x3F)
|
||||
case 3:
|
||||
(UInt32(bytes[0] & 0x0F) << 12) | (UInt32(bytes[1] & 0x3F) << 6) | UInt32(bytes[2] & 0x3F)
|
||||
default:
|
||||
(UInt32(bytes[0] & 0x07) << 18) | (UInt32(bytes[1] & 0x3F) << 12) | (UInt32(bytes[2] & 0x3F) << 6) | UInt32(bytes[3] & 0x3F)
|
||||
}
|
||||
}
|
||||
|
||||
private func utf8Width(of octet: UInt8) -> Int {
|
||||
if octet & 0x80 == 0x00 { return 1 }
|
||||
if octet & 0xE0 == 0xC0 { return 2 }
|
||||
if octet & 0xF0 == 0xE0 { return 3 }
|
||||
if octet & 0xF8 == 0xF0 { return 4 }
|
||||
return 0
|
||||
}
|
||||
|
||||
private func isURICharacter(_ scalar: Unicode.Scalar, uriChar: Bool) -> Bool {
|
||||
if Classify.isAlpha(scalar) {
|
||||
return true
|
||||
}
|
||||
switch scalar {
|
||||
case ";", "/", "?", ":", "@", "&", "=", "+", "$", ".", "%", "!", "~", "*", "'", "(", ")":
|
||||
return true
|
||||
case ",", "[", "]":
|
||||
return uriChar
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
private func hexDigit(_ scalar: Unicode.Scalar) -> Int {
|
||||
switch scalar {
|
||||
case "0" ... "9": Int(scalar.value - Unicode.Scalar("0").value)
|
||||
case "A" ... "F": Int(scalar.value - Unicode.Scalar("A").value) + 10
|
||||
default: Int(scalar.value - Unicode.Scalar("a").value) + 10
|
||||
}
|
||||
}
|
||||
|
||||
private func tagError(_ problem: String, _ mark: PureYAML.Parsing.Mark) -> PureYAML.Parsing.LibYAML.ScannerError {
|
||||
PureYAML.Parsing.LibYAML.ScannerError(context: "while scanning a tag", problem: problem, mark: mark)
|
||||
}
|
||||
}
|
||||
139
Sources/Parsing/LibYAML.Scanner.swift
Normal file
139
Sources/Parsing/LibYAML.Scanner.swift
Normal file
@ -0,0 +1,139 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// Aggregate scanner state, mirroring the fields of `yaml_parser_t` that the
|
||||
/// ported functions read and write. It is built up as the port proceeds;
|
||||
/// currently it holds the indentation engine and the simple-key mechanism.
|
||||
/// Column values follow libyaml's 0-based convention internally; conversion to
|
||||
/// the package's 1-based `Mark` happens at token emission during integration.
|
||||
struct Scanner {
|
||||
/// The input buffer and position (`parser->buffer` / `parser->mark`).
|
||||
var reader: Reader
|
||||
/// The indentation stack and `roll_indent` / `unroll_indent` (also the
|
||||
/// owner of `flow_level` and the token queue).
|
||||
var indentEngine = IndentEngine()
|
||||
/// `parser->simple_keys`. Always holds at least the block-context level.
|
||||
var simpleKeys: [SimpleKey] = [.empty]
|
||||
/// `parser->simple_key_allowed`. True at stream start.
|
||||
var simpleKeyAllowed = true
|
||||
/// Whether a tab appeared in the indentation before the pending token. A
|
||||
/// tab is valid in-line separation but never indentation, so a block
|
||||
/// indicator or implicit mapping key reached through one is an error.
|
||||
var tabbedSeparation = false
|
||||
/// Whether reaching the pending token crossed at least one line break. In
|
||||
/// flow context a continuation line must be indented past the parent block,
|
||||
/// so a token on a new line at or below that indent is an error.
|
||||
var crossedLineBreak = false
|
||||
/// Whether a tab appeared in the leading whitespace of a flow continuation
|
||||
/// line. A tab is never indentation, so flow content reached through one is
|
||||
/// an error even though the tab inflates the column.
|
||||
var flowLeadingTab = false
|
||||
/// Whether the most recent `:` value indicator sat on the same line as its
|
||||
/// key (an inline implicit-key value). Such a value may not open a block
|
||||
/// sequence on that line (`key: - a`), whereas an explicit key whose `:` is
|
||||
/// on a later line may (`? k`\n`: - a`).
|
||||
var lastValueWasInline = false
|
||||
/// For each open flow level, whether it is a sequence (`[`) rather than a
|
||||
/// mapping (`{`). A flow-sequence single-pair implicit key is single-line
|
||||
/// (`c-s-implicit-json-key`), so its `:` may not cross a line break, unlike
|
||||
/// a flow-mapping adjacent value (`{ "a"`\n`: b }` is allowed).
|
||||
var flowIsSequence: [Bool] = []
|
||||
/// `parser->mark`, kept in sync with the reader at each fetch step.
|
||||
var mark: PureYAML.Parsing.Mark = .start
|
||||
/// `parser->stream_start_produced`.
|
||||
var streamStartProduced = false
|
||||
/// `parser->stream_end_produced`.
|
||||
var streamEndProduced = false
|
||||
|
||||
init(_ input: String = "") {
|
||||
reader = Reader(input)
|
||||
}
|
||||
|
||||
/// `parser->flow_level`, kept on the indent engine.
|
||||
var flowLevel: Int {
|
||||
indentEngine.flowLevel
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_stale_simple_keys`. A potential simple key is
|
||||
/// invalid once it is on an earlier line or more than 1024 characters back
|
||||
/// (a simple key is single-line and bounded); a required one going stale is
|
||||
/// an error.
|
||||
mutating func staleSimpleKeys() throws {
|
||||
for index in simpleKeys.indices {
|
||||
let key = simpleKeys[index]
|
||||
guard key.possible else {
|
||||
continue
|
||||
}
|
||||
// A block-context key (index 0) is single-line. A flow-context key
|
||||
// (index >= 1) may be separated from its ':' by line breaks under
|
||||
// YAML 1.2, so a new line alone does not invalidate it; both remain
|
||||
// bounded to 1024 characters. libyaml invalidates every key on a new
|
||||
// line, which wrongly rejects multi-line flow keys.
|
||||
let tooFarBack = key.mark.index + 1024 < mark.index
|
||||
let stale = index == 0 ? (key.mark.line < mark.line || tooFarBack) : tooFarBack
|
||||
guard stale else {
|
||||
continue
|
||||
}
|
||||
if key.required {
|
||||
throw ScannerError(
|
||||
context: "while scanning a simple key",
|
||||
problem: "could not find expected ':'",
|
||||
mark: key.mark,
|
||||
)
|
||||
}
|
||||
simpleKeys[index].possible = false
|
||||
}
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_save_simple_key`. Record the current position as a
|
||||
/// potential simple key for this flow level when one is allowed. A key is
|
||||
/// required (a `:` must follow) when it sits exactly at the block indent.
|
||||
mutating func saveSimpleKey() throws {
|
||||
let required = flowLevel == 0 && indentEngine.indent == mark.column
|
||||
guard simpleKeyAllowed else {
|
||||
return
|
||||
}
|
||||
let key = SimpleKey(
|
||||
possible: true,
|
||||
required: required,
|
||||
tokenNumber: indentEngine.tokensParsed + indentEngine.tokens.count,
|
||||
mark: mark,
|
||||
tabbed: tabbedSeparation,
|
||||
)
|
||||
try removeSimpleKey()
|
||||
simpleKeys[simpleKeys.count - 1] = key
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_remove_simple_key`. Drop the potential key at the
|
||||
/// current flow level; removing a required key is an error.
|
||||
mutating func removeSimpleKey() throws {
|
||||
let top = simpleKeys.count - 1
|
||||
if simpleKeys[top].possible, simpleKeys[top].required {
|
||||
throw ScannerError(
|
||||
context: "while scanning a simple key",
|
||||
problem: "could not find expected ':'",
|
||||
mark: simpleKeys[top].mark,
|
||||
)
|
||||
}
|
||||
simpleKeys[top].possible = false
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_increase_flow_level`. Push an empty simple key for
|
||||
/// the new flow level, recording whether the level is a sequence.
|
||||
mutating func increaseFlowLevel(isSequence: Bool) {
|
||||
simpleKeys.append(.empty)
|
||||
flowIsSequence.append(isSequence)
|
||||
indentEngine.flowLevel += 1
|
||||
}
|
||||
|
||||
/// Port of `yaml_parser_decrease_flow_level`. Pop the flow level's simple
|
||||
/// key.
|
||||
mutating func decreaseFlowLevel() {
|
||||
if indentEngine.flowLevel > 0 {
|
||||
indentEngine.flowLevel -= 1
|
||||
simpleKeys.removeLast()
|
||||
if !flowIsSequence.isEmpty {
|
||||
flowIsSequence.removeLast()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
10
Sources/Parsing/LibYAML.ScannerError.swift
Normal file
10
Sources/Parsing/LibYAML.ScannerError.swift
Normal file
@ -0,0 +1,10 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// A scanning error, mirroring libyaml's `yaml_parser_set_scanner_error`
|
||||
/// (`context` plus `problem` at a mark). It is mapped to the package's public
|
||||
/// `ParseError` when the port is integrated.
|
||||
struct ScannerError: Swift.Error, Equatable {
|
||||
var context: String
|
||||
var problem: String
|
||||
var mark: PureYAML.Parsing.Mark
|
||||
}
|
||||
}
|
||||
28
Sources/Parsing/LibYAML.SimpleKey.swift
Normal file
28
Sources/Parsing/LibYAML.SimpleKey.swift
Normal file
@ -0,0 +1,28 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// Port of `yaml_simple_key_t`. A potential implicit ("simple") mapping key:
|
||||
/// a node that may turn out to be a key once a `:` is seen on the same line.
|
||||
/// `tokenNumber` records where its `KEY` token must be spliced into the queue.
|
||||
struct SimpleKey: Equatable {
|
||||
/// Whether a simple key is still possible at this flow level.
|
||||
var possible: Bool
|
||||
/// Whether a `:` is required (a key sitting exactly at the block indent).
|
||||
var required: Bool
|
||||
/// The absolute token number where the `KEY` token is inserted.
|
||||
var tokenNumber: Int
|
||||
/// The position the potential key started at.
|
||||
var mark: PureYAML.Parsing.Mark
|
||||
/// Whether this key is a JSON-style node (a quoted flow scalar). A `:`
|
||||
/// adjacent to such a key is a value indicator even without a following
|
||||
/// space, per the YAML flow "adjacent value" rule.
|
||||
var isJSON = false
|
||||
/// Whether a tab preceded this key in its indentation. A tab may be
|
||||
/// separation but never indentation, so promoting such a key to a block
|
||||
/// mapping key (a `:` follows) is an error.
|
||||
var tabbed = false
|
||||
|
||||
/// The empty entry pushed for a new flow level (`{ 0, 0, 0, {0,0,0} }`).
|
||||
static var empty: Self {
|
||||
Self(possible: false, required: false, tokenNumber: 0, mark: .start)
|
||||
}
|
||||
}
|
||||
}
|
||||
10
Sources/Parsing/LibYAML.Token.swift
Normal file
10
Sources/Parsing/LibYAML.Token.swift
Normal file
@ -0,0 +1,10 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// A scanner token, mirroring `yaml_token_t`: a kind plus its start and end
|
||||
/// marks. Token-specific payloads (scalar value, anchor name, tag) are added
|
||||
/// as their fetchers are ported.
|
||||
struct Token: Equatable {
|
||||
var type: TokenType
|
||||
var startMark: PureYAML.Parsing.Mark
|
||||
var endMark: PureYAML.Parsing.Mark
|
||||
}
|
||||
}
|
||||
29
Sources/Parsing/LibYAML.TokenType.swift
Normal file
29
Sources/Parsing/LibYAML.TokenType.swift
Normal file
@ -0,0 +1,29 @@
|
||||
extension PureYAML.Parsing.LibYAML {
|
||||
/// A scanner token kind, mirroring `yaml_token_type_t`. The flow-collection,
|
||||
/// anchor, tag, alias, and directive kinds are added as their fetchers are
|
||||
/// ported; the block + plain-scalar core is present.
|
||||
enum TokenType: Equatable {
|
||||
case streamStart
|
||||
case streamEnd
|
||||
case documentStart
|
||||
case documentEnd
|
||||
case blockSequenceStart
|
||||
case blockMappingStart
|
||||
case blockEnd
|
||||
case blockEntry
|
||||
case flowSequenceStart
|
||||
case flowMappingStart
|
||||
case flowSequenceEnd
|
||||
case flowMappingEnd
|
||||
case flowEntry
|
||||
case key
|
||||
case value
|
||||
case versionDirective(major: Int, minor: Int)
|
||||
case tagDirective(handle: String, prefix: String)
|
||||
case reservedDirective
|
||||
case anchor(name: String)
|
||||
case alias(name: String)
|
||||
case tag(handle: String, suffix: String)
|
||||
case scalar(value: String, style: PureYAML.Parsing.ScalarStyle)
|
||||
}
|
||||
}
|
||||
10
Sources/Parsing/LibYAML.swift
Normal file
10
Sources/Parsing/LibYAML.swift
Normal file
@ -0,0 +1,10 @@
|
||||
extension PureYAML.Parsing {
|
||||
/// Faithful Swift port of libyaml's scanner (`CYaml/src/scanner.c`). It is
|
||||
/// built as a parallel implementation, verified token-for-token against
|
||||
/// libyaml through the differential harness, to replace the hand-rolled
|
||||
/// approximation whose indentation and simple-key handling the official test
|
||||
/// suite surfaced as incorrect. The port proceeds function by function; each
|
||||
/// piece mirrors its C counterpart so correctness is by construction rather
|
||||
/// than re-derived. Members live under this namespace in their own files.
|
||||
enum LibYAML {}
|
||||
}
|
||||
@ -1,17 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
struct NodeProperties: Equatable {
|
||||
var anchor: String?
|
||||
var tag: String?
|
||||
var mark: Mark?
|
||||
|
||||
static var none: Self {
|
||||
Self(anchor: nil, tag: nil, mark: nil)
|
||||
}
|
||||
|
||||
mutating func mergeUnset(from other: Self) {
|
||||
anchor = anchor ?? other.anchor
|
||||
tag = tag ?? other.tag
|
||||
mark = mark ?? other.mark
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,6 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
struct NodeResult: Equatable {
|
||||
var kind: NodeKind
|
||||
var endMark: Mark
|
||||
}
|
||||
}
|
||||
39
Sources/Parsing/ParseError.LibYAML.swift
Normal file
39
Sources/Parsing/ParseError.LibYAML.swift
Normal file
@ -0,0 +1,39 @@
|
||||
extension PureYAML.Parsing.ParseError {
|
||||
/// Translate a ported-scanner error into the package's diagnostic-aware error,
|
||||
/// converting libyaml's 0-based mark to the 1-based convention and mapping the
|
||||
/// well-known scanner failures to precise diagnostic codes.
|
||||
init(translating error: PureYAML.Parsing.LibYAML.ScannerError) {
|
||||
let line = error.mark.line + 1
|
||||
let column = error.mark.column + 1
|
||||
let context = error.context ?? ""
|
||||
if error.problem.hasPrefix("found a tab") {
|
||||
self = .tabIndentation(line: line)
|
||||
} else if context == "while scanning a quoted scalar", error.problem.hasSuffix("end of stream") {
|
||||
self = .unterminatedQuotedString(line: line)
|
||||
} else if context.hasPrefix("while scanning a tag") || context.hasSuffix("TAG directive") {
|
||||
self = .unterminatedTag(line: line)
|
||||
} else {
|
||||
self = .malformed(reason: Self.reason(context: error.context, problem: error.problem), line: line, column: column)
|
||||
}
|
||||
}
|
||||
|
||||
/// Translate a ported-parser error into the package's diagnostic-aware error.
|
||||
init(translating error: PureYAML.Parsing.LibYAML.ParserError) {
|
||||
let line = error.mark.line + 1
|
||||
let column = error.mark.column + 1
|
||||
if error.problem == "found incompatible YAML document" {
|
||||
self = .incompatibleYAMLDirective(line: line)
|
||||
} else {
|
||||
self = .malformed(reason: Self.reason(context: error.context, problem: error.problem), line: line, column: column)
|
||||
}
|
||||
}
|
||||
|
||||
/// Join a libyaml context and problem into one readable reason, e.g.
|
||||
/// "while parsing a flow sequence, did not find expected ',' or ']'".
|
||||
private static func reason(context: String?, problem: String) -> String {
|
||||
guard let context else {
|
||||
return problem
|
||||
}
|
||||
return "\(context), \(problem)"
|
||||
}
|
||||
}
|
||||
@ -1,13 +1,17 @@
|
||||
public extension PureYAML.Parsing {
|
||||
/// Parser error with source line information when available.
|
||||
enum ParseError: Swift.Error, Equatable, Sendable, CustomStringConvertible {
|
||||
case emptyDocument
|
||||
case tabIndentation(line: Int)
|
||||
case unexpectedIndentation(line: Int)
|
||||
case mixedCollectionStyles(line: Int)
|
||||
case expectedMappingKey(line: Int)
|
||||
case expectedAnchorName(line: Int)
|
||||
case expectedAliasName(line: Int)
|
||||
case expectedNode(line: Int, column: Int)
|
||||
case expectedScalarKey(line: Int, column: Int)
|
||||
case incompatibleYAMLDirective(line: Int)
|
||||
case directiveWithoutDocument(line: Int)
|
||||
case invalidTaggedScalar(tag: String, value: String, line: Int, column: Int)
|
||||
case invalidMergeValue(line: Int, column: Int)
|
||||
case unterminatedTag(line: Int)
|
||||
@ -18,25 +22,20 @@ public extension PureYAML.Parsing {
|
||||
case unexpectedEvent(expected: String, actual: String, line: Int, column: Int)
|
||||
case unexpectedToken(expected: String, actual: String, line: Int, column: Int)
|
||||
case nestingTooDeep(line: Int, column: Int)
|
||||
case commentNotPrecededByWhitespace(line: Int, column: Int)
|
||||
case invalidEscape(sequence: String, line: Int)
|
||||
case malformedDirective(name: String, line: Int)
|
||||
case duplicateDirective(name: String, line: Int)
|
||||
case directiveWithoutDocument(line: Int)
|
||||
case contentAfterDocumentEnd(line: Int)
|
||||
case blockSequenceAsInlineMappingValue(line: Int)
|
||||
case multilineImplicitKey(line: Int)
|
||||
case commentInMultilineScalar(line: Int)
|
||||
case aliasWithProperties(line: Int, column: Int)
|
||||
case inlineBlockSequenceAfterProperty(line: Int, column: Int)
|
||||
case documentMarkerInScalar(line: Int)
|
||||
case malformed(reason: String, line: Int, column: Int)
|
||||
|
||||
public var description: String {
|
||||
switch self {
|
||||
case .emptyDocument:
|
||||
"document is empty"
|
||||
case let .tabIndentation(line):
|
||||
"tabs are not allowed for indentation at line \(line)"
|
||||
case let .unexpectedIndentation(line):
|
||||
"unexpected indentation at line \(line)"
|
||||
case let .mixedCollectionStyles(line):
|
||||
"mapping and sequence entries are mixed at line \(line)"
|
||||
case let .expectedMappingKey(line):
|
||||
"expected a mapping key at line \(line)"
|
||||
case let .expectedAnchorName(line):
|
||||
"expected an anchor name at line \(line)"
|
||||
case let .expectedAliasName(line):
|
||||
@ -47,6 +46,8 @@ public extension PureYAML.Parsing {
|
||||
"expected a scalar mapping key at line \(line), column \(column)"
|
||||
case let .incompatibleYAMLDirective(line):
|
||||
"incompatible YAML directive at line \(line)"
|
||||
case let .directiveWithoutDocument(line):
|
||||
"directive is not followed by a document start at line \(line)"
|
||||
case let .invalidTaggedScalar(tag, value, line, column):
|
||||
"invalid scalar '\(value)' for tag '\(tag)' at line \(line), column \(column)"
|
||||
case let .invalidMergeValue(line, column):
|
||||
@ -67,39 +68,23 @@ public extension PureYAML.Parsing {
|
||||
"expected \(expected) at line \(line), column \(column), found \(actual)"
|
||||
case let .nestingTooDeep(line, column):
|
||||
"nesting is too deep at line \(line), column \(column)"
|
||||
case let .commentNotPrecededByWhitespace(line, column):
|
||||
"a comment must be preceded by whitespace at line \(line), column \(column)"
|
||||
case let .invalidEscape(sequence, line):
|
||||
"invalid escape sequence '\(sequence)' at line \(line)"
|
||||
case let .malformedDirective(name, line):
|
||||
"malformed directive '\(name)' at line \(line)"
|
||||
case let .duplicateDirective(name, line):
|
||||
"duplicate directive '\(name)' at line \(line)"
|
||||
case let .directiveWithoutDocument(line):
|
||||
"a directive must be followed by a document at line \(line)"
|
||||
case let .contentAfterDocumentEnd(line):
|
||||
"no content may follow a document end marker at line \(line)"
|
||||
case let .blockSequenceAsInlineMappingValue(line):
|
||||
"a block sequence cannot start on the same line as a mapping value at line \(line)"
|
||||
case let .multilineImplicitKey(line):
|
||||
"an implicit mapping key must not span multiple lines at line \(line)"
|
||||
case let .commentInMultilineScalar(line):
|
||||
"a comment cannot interrupt a multi-line plain scalar at line \(line)"
|
||||
case let .aliasWithProperties(line, column):
|
||||
"an alias node cannot have an anchor or tag at line \(line), column \(column)"
|
||||
case let .inlineBlockSequenceAfterProperty(line, column):
|
||||
"a block sequence cannot start on the same line as its anchor or tag at line \(line), column \(column)"
|
||||
case let .documentMarkerInScalar(line):
|
||||
"a document marker cannot appear inside a multi-line scalar at line \(line)"
|
||||
case let .malformed(reason, line, column):
|
||||
"\(reason) at line \(line), column \(column)"
|
||||
}
|
||||
}
|
||||
|
||||
public var diagnosticCode: String {
|
||||
switch self {
|
||||
case .emptyDocument:
|
||||
"emptyDocument"
|
||||
case .tabIndentation:
|
||||
"tabIndentation"
|
||||
case .unexpectedIndentation:
|
||||
"unexpectedIndentation"
|
||||
case .mixedCollectionStyles:
|
||||
"mixedCollectionStyles"
|
||||
case .expectedMappingKey:
|
||||
"expectedMappingKey"
|
||||
case .expectedAnchorName:
|
||||
"expectedAnchorName"
|
||||
case .expectedAliasName:
|
||||
@ -110,6 +95,8 @@ public extension PureYAML.Parsing {
|
||||
"expectedScalarKey"
|
||||
case .incompatibleYAMLDirective:
|
||||
"incompatibleYAMLDirective"
|
||||
case .directiveWithoutDocument:
|
||||
"directiveWithoutDocument"
|
||||
case .invalidTaggedScalar:
|
||||
"invalidTaggedScalar"
|
||||
case .invalidMergeValue:
|
||||
@ -130,53 +117,27 @@ public extension PureYAML.Parsing {
|
||||
"unexpectedToken"
|
||||
case .nestingTooDeep:
|
||||
"nestingTooDeep"
|
||||
case .commentNotPrecededByWhitespace:
|
||||
"commentNotPrecededByWhitespace"
|
||||
case .invalidEscape:
|
||||
"invalidEscape"
|
||||
case .malformedDirective:
|
||||
"malformedDirective"
|
||||
case .duplicateDirective:
|
||||
"duplicateDirective"
|
||||
case .directiveWithoutDocument:
|
||||
"directiveWithoutDocument"
|
||||
case .contentAfterDocumentEnd:
|
||||
"contentAfterDocumentEnd"
|
||||
case .blockSequenceAsInlineMappingValue:
|
||||
"blockSequenceAsInlineMappingValue"
|
||||
case .multilineImplicitKey:
|
||||
"multilineImplicitKey"
|
||||
case .commentInMultilineScalar:
|
||||
"commentInMultilineScalar"
|
||||
case .aliasWithProperties:
|
||||
"aliasWithProperties"
|
||||
case .inlineBlockSequenceAfterProperty:
|
||||
"inlineBlockSequenceAfterProperty"
|
||||
case .documentMarkerInScalar:
|
||||
"documentMarkerInScalar"
|
||||
case .malformed:
|
||||
"malformed"
|
||||
}
|
||||
}
|
||||
|
||||
public var sourceLine: Int? {
|
||||
switch self {
|
||||
case let .unexpectedIndentation(line),
|
||||
case .emptyDocument:
|
||||
nil
|
||||
case let .tabIndentation(line),
|
||||
let .unexpectedIndentation(line),
|
||||
let .mixedCollectionStyles(line),
|
||||
let .expectedMappingKey(line),
|
||||
let .expectedAnchorName(line),
|
||||
let .expectedAliasName(line),
|
||||
let .incompatibleYAMLDirective(line),
|
||||
let .directiveWithoutDocument(line),
|
||||
let .unterminatedTag(line),
|
||||
let .unterminatedQuotedString(line),
|
||||
let .unsupportedDirective(_, line),
|
||||
let .unsupportedMultiDocumentStream(line),
|
||||
let .documentMarkerInScalar(line),
|
||||
let .invalidEscape(_, line),
|
||||
let .malformedDirective(_, line),
|
||||
let .duplicateDirective(_, line),
|
||||
let .directiveWithoutDocument(line),
|
||||
let .contentAfterDocumentEnd(line),
|
||||
let .blockSequenceAsInlineMappingValue(line),
|
||||
let .multilineImplicitKey(line),
|
||||
let .commentInMultilineScalar(line):
|
||||
let .unsupportedMultiDocumentStream(line):
|
||||
line
|
||||
case let .expectedNode(line, _),
|
||||
let .expectedScalarKey(line, _),
|
||||
@ -186,33 +147,26 @@ public extension PureYAML.Parsing {
|
||||
let .unexpectedEvent(_, _, line, _),
|
||||
let .unexpectedToken(_, _, line, _),
|
||||
let .nestingTooDeep(line, _),
|
||||
let .commentNotPrecededByWhitespace(line, _),
|
||||
let .aliasWithProperties(line, _),
|
||||
let .inlineBlockSequenceAfterProperty(line, _):
|
||||
let .malformed(_, line, _):
|
||||
line
|
||||
}
|
||||
}
|
||||
|
||||
public var sourceColumn: Int? {
|
||||
switch self {
|
||||
case .unexpectedIndentation,
|
||||
case .emptyDocument,
|
||||
.tabIndentation,
|
||||
.unexpectedIndentation,
|
||||
.mixedCollectionStyles,
|
||||
.expectedMappingKey,
|
||||
.expectedAnchorName,
|
||||
.expectedAliasName,
|
||||
.incompatibleYAMLDirective,
|
||||
.directiveWithoutDocument,
|
||||
.unterminatedTag,
|
||||
.unterminatedQuotedString,
|
||||
.unsupportedDirective,
|
||||
.unsupportedMultiDocumentStream,
|
||||
.documentMarkerInScalar,
|
||||
.invalidEscape,
|
||||
.malformedDirective,
|
||||
.duplicateDirective,
|
||||
.directiveWithoutDocument,
|
||||
.contentAfterDocumentEnd,
|
||||
.blockSequenceAsInlineMappingValue,
|
||||
.multilineImplicitKey,
|
||||
.commentInMultilineScalar:
|
||||
.unsupportedMultiDocumentStream:
|
||||
nil
|
||||
case let .expectedNode(_, column),
|
||||
let .expectedScalarKey(_, column),
|
||||
@ -222,9 +176,7 @@ public extension PureYAML.Parsing {
|
||||
let .unexpectedEvent(_, _, _, column),
|
||||
let .unexpectedToken(_, _, _, column),
|
||||
let .nestingTooDeep(_, column),
|
||||
let .commentNotPrecededByWhitespace(_, column),
|
||||
let .aliasWithProperties(_, column),
|
||||
let .inlineBlockSequenceAfterProperty(_, column):
|
||||
let .malformed(_, _, column):
|
||||
column
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,8 +1,46 @@
|
||||
extension PureYAML.Parsing.Parser {
|
||||
func parseEvents(_ yaml: String) throws -> [PureYAML.Parsing.Event] {
|
||||
let tokens = try PureYAML.Parsing.Scanner().scan(yaml)
|
||||
var parser = PureYAML.Parsing.TokenEventParser(tokens: tokens, scalarParser: self)
|
||||
return try parser.parse()
|
||||
let events: [PureYAML.Parsing.Event]
|
||||
do {
|
||||
var parser = try PureYAML.Parsing.LibYAML.Parser(yaml)
|
||||
events = try parser.parse().map { $0.bumpingMark() }
|
||||
} catch let error as PureYAML.Parsing.LibYAML.ScannerError {
|
||||
throw PureYAML.Parsing.ParseError(translating: error)
|
||||
} catch let error as PureYAML.Parsing.LibYAML.ParserError {
|
||||
throw PureYAML.Parsing.ParseError(translating: error)
|
||||
}
|
||||
try enforceNestingLimit(events)
|
||||
return events
|
||||
}
|
||||
|
||||
/// The deepest collection nesting the value/event composers will build. The
|
||||
/// ported parser produces events iteratively without recursion, but the
|
||||
/// composers, equality, hashing, and `Model/Value` teardown all recurse with
|
||||
/// the nesting depth, so the event stream is bounded here before any recursive
|
||||
/// consumer runs. 64 accepts every realistic document yet rejects pathological
|
||||
/// input well before it can overflow the stack.
|
||||
static var maxNestingDepth: Int {
|
||||
64
|
||||
}
|
||||
|
||||
/// Reject an event stream that nests collections past ``maxNestingDepth`` with
|
||||
/// a ``ParseError/nestingTooDeep(line:column:)`` so deeply nested input fails
|
||||
/// cleanly instead of overflowing a recursive consumer's stack.
|
||||
private func enforceNestingLimit(_ events: [PureYAML.Parsing.Event]) throws {
|
||||
var depth = 0
|
||||
for event in events {
|
||||
switch event {
|
||||
case .sequenceStart, .mappingStart:
|
||||
depth += 1
|
||||
if depth > Self.maxNestingDepth {
|
||||
throw PureYAML.Parsing.ParseError.nestingTooDeep(line: event.mark.line, column: event.mark.column)
|
||||
}
|
||||
case .sequenceEnd, .mappingEnd:
|
||||
depth -= 1
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func parseStreaming(_ yaml: String) throws -> PureYAML.Model.Value {
|
||||
@ -10,14 +48,9 @@ extension PureYAML.Parsing.Parser {
|
||||
scalarParser: self,
|
||||
mode: .singleDocument,
|
||||
)
|
||||
var parser = PureYAML.Parsing.TokenEventParser(
|
||||
yaml: yaml,
|
||||
scalarParser: self,
|
||||
eventSink: { event in
|
||||
try composer.consume(event)
|
||||
},
|
||||
)
|
||||
try parser.parseStreaming()
|
||||
for event in try parseEvents(yaml) {
|
||||
try composer.consume(event)
|
||||
}
|
||||
return try composer.composedValue()
|
||||
}
|
||||
|
||||
@ -26,14 +59,9 @@ extension PureYAML.Parsing.Parser {
|
||||
scalarParser: self,
|
||||
mode: .stream,
|
||||
)
|
||||
var parser = PureYAML.Parsing.TokenEventParser(
|
||||
yaml: yaml,
|
||||
scalarParser: self,
|
||||
eventSink: { event in
|
||||
try composer.consume(event)
|
||||
},
|
||||
)
|
||||
try parser.parseStreaming()
|
||||
for event in try parseEvents(yaml) {
|
||||
try composer.consume(event)
|
||||
}
|
||||
return try composer.composedStream()
|
||||
}
|
||||
|
||||
@ -43,14 +71,9 @@ extension PureYAML.Parsing.Parser {
|
||||
mode: .singleDocument,
|
||||
tracksLocations: true,
|
||||
)
|
||||
var parser = PureYAML.Parsing.TokenEventParser(
|
||||
yaml: yaml,
|
||||
scalarParser: self,
|
||||
eventSink: { event in
|
||||
try composer.consume(event)
|
||||
},
|
||||
)
|
||||
try parser.parseStreaming()
|
||||
for event in try parseEvents(yaml) {
|
||||
try composer.consume(event)
|
||||
}
|
||||
return try composer.composedLocated()
|
||||
}
|
||||
|
||||
@ -60,14 +83,9 @@ extension PureYAML.Parsing.Parser {
|
||||
mode: .stream,
|
||||
tracksLocations: true,
|
||||
)
|
||||
var parser = PureYAML.Parsing.TokenEventParser(
|
||||
yaml: yaml,
|
||||
scalarParser: self,
|
||||
eventSink: { event in
|
||||
try composer.consume(event)
|
||||
},
|
||||
)
|
||||
try parser.parseStreaming()
|
||||
for event in try parseEvents(yaml) {
|
||||
try composer.consume(event)
|
||||
}
|
||||
return try composer.composedLocatedStream()
|
||||
}
|
||||
}
|
||||
|
||||
@ -35,6 +35,10 @@ extension PureYAML.Parsing.Parser {
|
||||
return nil
|
||||
}
|
||||
switch tag {
|
||||
case "!":
|
||||
// The non-specific `!` tag resolves a scalar by kind alone: a scalar is
|
||||
// a string, with no implicit typing, so `! 12` is "12" not the int 12.
|
||||
return .string(value)
|
||||
case "tag:yaml.org,2002:str":
|
||||
return .string(value)
|
||||
case "tag:yaml.org,2002:int":
|
||||
|
||||
@ -110,27 +110,8 @@ extension PureYAML.Parsing.Parser {
|
||||
guard text.count >= 2, text.last == "'" else {
|
||||
throw PureYAML.Parsing.ParseError.unterminatedQuotedString(line: line)
|
||||
}
|
||||
let inner = text.dropFirst().dropLast()
|
||||
var output = ""
|
||||
var index = inner.startIndex
|
||||
while index < inner.endIndex {
|
||||
let character = inner[index]
|
||||
if character == "'" {
|
||||
// A doubled quote escapes to a single quote; a lone quote (not normally present
|
||||
// inside the stripped body) is kept as-is.
|
||||
inner.formIndex(after: &index)
|
||||
output.append("'")
|
||||
if index < inner.endIndex, inner[index] == "'" {
|
||||
inner.formIndex(after: &index)
|
||||
}
|
||||
} else if character == "\n" || character == "\r" {
|
||||
foldFlowScalarLineBreaks(in: inner, index: &index, to: &output)
|
||||
} else {
|
||||
output.append(character)
|
||||
inner.formIndex(after: &index)
|
||||
}
|
||||
}
|
||||
return output
|
||||
let inner = Array(text.dropFirst().dropLast())
|
||||
return foldFlowScalar(inner, single: true)
|
||||
}
|
||||
|
||||
func parseDoubleQuoted(
|
||||
@ -140,184 +121,238 @@ extension PureYAML.Parsing.Parser {
|
||||
guard text.count >= 2, text.last == "\"" else {
|
||||
throw PureYAML.Parsing.ParseError.unterminatedQuotedString(line: line)
|
||||
}
|
||||
let inner = text.dropFirst().dropLast()
|
||||
let inner = Array(text.dropFirst().dropLast())
|
||||
return foldFlowScalar(inner, single: false)
|
||||
}
|
||||
|
||||
/// Flow (quoted) scalar line folding, mirroring libyaml's
|
||||
/// yaml_parser_scan_flow_scalar (single and double quoted). Trailing
|
||||
/// whitespace before a break is dropped, leading whitespace on a
|
||||
/// continuation line is dropped, a single break between non-empty lines
|
||||
/// folds to one space, and N consecutive breaks yield N-1 line feeds.
|
||||
/// For double quoted scalars a backslash at end of line suppresses the
|
||||
/// break entirely.
|
||||
func foldFlowScalar(
|
||||
_ inner: [Character],
|
||||
single: Bool,
|
||||
) -> String {
|
||||
var state = FlowFoldState()
|
||||
let count = inner.count
|
||||
while state.index < count {
|
||||
state.leadingBlanks = false
|
||||
consumeFlowContent(inner, single: single, state: &state)
|
||||
consumeFlowBlanks(inner, state: &state)
|
||||
joinFlowFold(state: &state)
|
||||
}
|
||||
return state.output
|
||||
}
|
||||
|
||||
/// Mutable accumulators for flow scalar folding, matching the C reference's
|
||||
/// string / leading_break / trailing_breaks / whitespaces buffers.
|
||||
struct FlowFoldState {
|
||||
var output = ""
|
||||
var index = inner.startIndex
|
||||
while index < inner.endIndex {
|
||||
let character = inner[index]
|
||||
if character == "\\" {
|
||||
inner.formIndex(after: &index)
|
||||
guard index < inner.endIndex else {
|
||||
output.append("\\")
|
||||
var whitespaces = ""
|
||||
var leadingBreak = ""
|
||||
var trailingBreaks = ""
|
||||
var leadingBlanks = false
|
||||
var index = 0
|
||||
}
|
||||
|
||||
/// Consume the run of non-blank characters at `state.index`, decoding
|
||||
/// escapes. A double quoted backslash at end of line sets `leadingBlanks`
|
||||
/// and stops, so the following break is suppressed during folding.
|
||||
func consumeFlowContent(
|
||||
_ inner: [Character],
|
||||
single: Bool,
|
||||
state: inout FlowFoldState,
|
||||
) {
|
||||
let count = inner.count
|
||||
while state.index < count, !isFlowBlank(inner[state.index]), !isFlowBreak(inner[state.index]) {
|
||||
if single, inner[state.index] == "'" {
|
||||
state.output.append("'")
|
||||
let isEscapedQuote = state.index + 1 < count && inner[state.index + 1] == "'"
|
||||
state.index += isEscapedQuote ? 2 : 1
|
||||
continue
|
||||
}
|
||||
if !single, inner[state.index] == "\\" {
|
||||
if state.index + 1 < count, isFlowBreak(inner[state.index + 1]) {
|
||||
state.index += 2
|
||||
state.leadingBlanks = true
|
||||
break
|
||||
}
|
||||
try appendDoubleQuotedEscape(in: inner, index: &index, to: &output, line: line)
|
||||
} else if character == "\n" || character == "\r" {
|
||||
foldFlowScalarLineBreaks(in: inner, index: &index, to: &output)
|
||||
} else {
|
||||
output.append(character)
|
||||
inner.formIndex(after: &index)
|
||||
state.index += 1
|
||||
appendDoubleQuotedEscape(inner, index: &state.index, to: &state.output)
|
||||
continue
|
||||
}
|
||||
state.output.append(inner[state.index])
|
||||
state.index += 1
|
||||
}
|
||||
return output
|
||||
}
|
||||
|
||||
/// Flow line folding for a quoted (single- or double-) scalar at a line break. Whitespace
|
||||
/// adjacent to the break is stripped (the current line's trailing, each following line's
|
||||
/// leading), then a single break folds to one space and a run of breaks (blank lines) folds to
|
||||
/// one fewer newline. The scalar's first-line leading whitespace and last-line trailing
|
||||
/// whitespace are untouched, since they sit next to a quote, not a break.
|
||||
func foldFlowScalarLineBreaks(
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
/// Consume the run of blanks and line breaks at `state.index`, sorting them
|
||||
/// into the whitespace / leading-break / trailing-break buffers.
|
||||
func consumeFlowBlanks(
|
||||
_ inner: [Character],
|
||||
state: inout FlowFoldState,
|
||||
) {
|
||||
let count = inner.count
|
||||
while state.index < count, isFlowBlank(inner[state.index]) || isFlowBreak(inner[state.index]) {
|
||||
if isFlowBlank(inner[state.index]) {
|
||||
if !state.leadingBlanks {
|
||||
state.whitespaces.append(inner[state.index])
|
||||
}
|
||||
} else if !state.leadingBlanks {
|
||||
state.whitespaces = ""
|
||||
state.leadingBreak = "\n"
|
||||
state.leadingBlanks = true
|
||||
} else {
|
||||
state.trailingBreaks.append("\n")
|
||||
}
|
||||
state.index += 1
|
||||
}
|
||||
}
|
||||
|
||||
/// Join the captured whitespace or fold the captured line breaks into the
|
||||
/// output, per YAML 1.2 flow folding.
|
||||
func joinFlowFold(state: inout FlowFoldState) {
|
||||
guard state.leadingBlanks else {
|
||||
state.output.append(state.whitespaces)
|
||||
state.whitespaces = ""
|
||||
return
|
||||
}
|
||||
if state.leadingBreak.first == "\n" {
|
||||
if state.trailingBreaks.isEmpty {
|
||||
state.output.append(" ")
|
||||
} else {
|
||||
state.output.append(state.trailingBreaks)
|
||||
state.trailingBreaks = ""
|
||||
}
|
||||
state.leadingBreak = ""
|
||||
} else {
|
||||
state.output.append(state.leadingBreak)
|
||||
state.output.append(state.trailingBreaks)
|
||||
state.leadingBreak = ""
|
||||
state.trailingBreaks = ""
|
||||
}
|
||||
}
|
||||
|
||||
func isFlowBlank(_ character: Character) -> Bool {
|
||||
character == " " || character == "\t"
|
||||
}
|
||||
|
||||
func isFlowBreak(_ character: Character) -> Bool {
|
||||
character == "\n" || character == "\r" || character == "\r\n"
|
||||
}
|
||||
|
||||
/// Decode a double quoted escape. On entry index points at the escape
|
||||
/// character (the position just after the backslash). On return index has
|
||||
/// advanced past every character the escape consumed. The escape set
|
||||
/// mirrors libyaml's yaml_parser_scan_flow_scalar.
|
||||
func appendDoubleQuotedEscape(
|
||||
_ text: [Character],
|
||||
index: inout Int,
|
||||
to output: inout String,
|
||||
) {
|
||||
while let last = output.last, last == " " || last == "\t" {
|
||||
output.removeLast()
|
||||
guard index < text.count else {
|
||||
output.append("\\")
|
||||
return
|
||||
}
|
||||
var breaks = 0
|
||||
while index < text.endIndex {
|
||||
let character = text[index]
|
||||
if character == "\r" {
|
||||
text.formIndex(after: &index)
|
||||
if index < text.endIndex, text[index] == "\n" {
|
||||
text.formIndex(after: &index)
|
||||
}
|
||||
breaks += 1
|
||||
} else if character == "\n" {
|
||||
text.formIndex(after: &index)
|
||||
breaks += 1
|
||||
} else if character == " " || character == "\t" {
|
||||
text.formIndex(after: &index)
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
output.append(breaks == 1 ? " " : String(repeating: "\n", count: breaks - 1))
|
||||
}
|
||||
|
||||
/// Decodes one double-quoted escape, the character after a `\`. The set is exactly YAML 1.2's
|
||||
/// escape table; anything else is rejected rather than silently passed through. The index enters
|
||||
/// at the escape character and is left on the last consumed character, so the caller's trailing
|
||||
/// advance steps past it.
|
||||
func appendDoubleQuotedEscape(
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
to output: inout String,
|
||||
line: Int,
|
||||
) throws {
|
||||
let character = text[index]
|
||||
if let replacement = Self.simpleDoubleQuotedEscapes[character] {
|
||||
output.append(replacement)
|
||||
} else {
|
||||
switch character {
|
||||
case "x": try appendHexEscape(in: text, index: &index, to: &output, digits: 2, line: line)
|
||||
case "u": try appendUnicodeEscape(in: text, index: &index, to: &output, line: line)
|
||||
case "U": try appendHexEscape(in: text, index: &index, to: &output, digits: 8, line: line)
|
||||
case "\n", "\r": consumeEscapedLineBreak(in: text, index: &index)
|
||||
default:
|
||||
throw PureYAML.Parsing.ParseError.invalidEscape(sequence: "\\\(character)", line: line)
|
||||
}
|
||||
if let simple = Self.simpleDoubleQuotedEscapes[character] {
|
||||
output.append(simple)
|
||||
index += 1
|
||||
return
|
||||
}
|
||||
switch character {
|
||||
case "x":
|
||||
index += 1
|
||||
appendHexEscape(text, index: &index, digits: 2, marker: "x", to: &output)
|
||||
case "u":
|
||||
index += 1
|
||||
appendUnicodeEscape(text, index: &index, to: &output)
|
||||
case "U":
|
||||
index += 1
|
||||
appendHexEscape(text, index: &index, digits: 8, marker: "U", to: &output)
|
||||
default:
|
||||
output.append(character)
|
||||
index += 1
|
||||
}
|
||||
text.formIndex(after: &index)
|
||||
}
|
||||
|
||||
/// The YAML 1.2 escapes that map a single character after `\` to one character. Escapes that
|
||||
/// consume following characters (`\x`, `\u`, `\U`, an escaped line break) are handled separately.
|
||||
/// Single character double quoted escapes, matching libyaml's escape table.
|
||||
static let simpleDoubleQuotedEscapes: [Character: Character] = [
|
||||
"0": "\u{00}", "a": "\u{07}", "b": "\u{08}", "t": "\t", "\t": "\t",
|
||||
"n": "\n", "v": "\u{0B}", "f": "\u{0C}", "r": "\r", "e": "\u{1B}",
|
||||
" ": " ", "\"": "\"", "/": "/", "\\": "\\",
|
||||
"N": "\u{85}", "_": "\u{A0}", "L": "\u{2028}", "P": "\u{2029}",
|
||||
"0": "\u{00}",
|
||||
"a": "\u{07}",
|
||||
"b": "\u{08}",
|
||||
"t": "\t",
|
||||
"\t": "\t",
|
||||
"n": "\n",
|
||||
"v": "\u{0B}",
|
||||
"f": "\u{0C}",
|
||||
"r": "\r",
|
||||
"e": "\u{1B}",
|
||||
" ": " ",
|
||||
"\"": "\"",
|
||||
"/": "/",
|
||||
"\\": "\\",
|
||||
"N": "\u{85}",
|
||||
"_": "\u{A0}",
|
||||
"L": "\u{2028}",
|
||||
"P": "\u{2029}",
|
||||
]
|
||||
|
||||
/// Handles a `\` immediately before a line break: an escaped line break, where the two lines
|
||||
/// join with no separator. Consumes the break (LF, CR, or CRLF) and the continuation line's
|
||||
/// leading spaces and tabs, emitting nothing, and leaves the index on the last consumed
|
||||
/// character so the caller's advance steps to the continuation's first content character.
|
||||
func consumeEscapedLineBreak(
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
) {
|
||||
if text[index] == "\r" {
|
||||
var next = index
|
||||
text.formIndex(after: &next)
|
||||
if next < text.endIndex, text[next] == "\n" {
|
||||
index = next
|
||||
}
|
||||
}
|
||||
while true {
|
||||
var next = index
|
||||
text.formIndex(after: &next)
|
||||
guard next < text.endIndex, text[next] == " " || text[next] == "\t" else {
|
||||
break
|
||||
}
|
||||
index = next
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads a fixed-width hex escape (`\xHH`, `\UHHHHHHHH`) and appends the scalar it denotes,
|
||||
/// rejecting a truncated sequence or a value that is not a valid Unicode scalar.
|
||||
/// Decode \xXX or \UXXXXXXXX. On entry index points at the first hex digit.
|
||||
func appendHexEscape(
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
to output: inout String,
|
||||
_ text: [Character],
|
||||
index: inout Int,
|
||||
digits: Int,
|
||||
line: Int,
|
||||
) throws {
|
||||
let typeCharacter = text[index]
|
||||
guard let value = readHexEscape(in: text, index: &index, digits: digits),
|
||||
let scalar = UnicodeScalar(value)
|
||||
else {
|
||||
throw PureYAML.Parsing.ParseError.invalidEscape(sequence: "\\\(typeCharacter)", line: line)
|
||||
marker: Character,
|
||||
to output: inout String,
|
||||
) {
|
||||
guard let value = readHex(text, index: &index, digits: digits) else {
|
||||
output.append(marker)
|
||||
return
|
||||
}
|
||||
if let scalar = UnicodeScalar(value) {
|
||||
output.unicodeScalars.append(scalar)
|
||||
} else {
|
||||
output.append(marker)
|
||||
}
|
||||
output.unicodeScalars.append(scalar)
|
||||
}
|
||||
|
||||
/// Decode \uXXXX, including surrogate pairs. On entry index points at the
|
||||
/// first hex digit.
|
||||
func appendUnicodeEscape(
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
_ text: [Character],
|
||||
index: inout Int,
|
||||
to output: inout String,
|
||||
line: Int,
|
||||
) throws {
|
||||
guard let codeUnit = readHexEscape(in: text, index: &index, digits: 4) else {
|
||||
throw PureYAML.Parsing.ParseError.invalidEscape(sequence: "\\u", line: line)
|
||||
) {
|
||||
let start = index
|
||||
guard let codeUnit = readHex(text, index: &index, digits: 4) else {
|
||||
output.append("u")
|
||||
index = start
|
||||
return
|
||||
}
|
||||
if let scalar = readSurrogatePairIfNeeded(high: codeUnit, in: text, index: &index) {
|
||||
if isHighSurrogate(codeUnit), let scalar = readSurrogatePair(high: codeUnit, text: text, index: &index) {
|
||||
output.unicodeScalars.append(scalar)
|
||||
} else if let scalar = UnicodeScalar(codeUnit) {
|
||||
output.unicodeScalars.append(scalar)
|
||||
} else {
|
||||
throw PureYAML.Parsing.ParseError.invalidEscape(sequence: "\\u", line: line)
|
||||
output.append("u")
|
||||
}
|
||||
}
|
||||
|
||||
func readSurrogatePairIfNeeded(
|
||||
high: UInt32,
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
) -> UnicodeScalar? {
|
||||
guard isHighSurrogate(high) else {
|
||||
return nil
|
||||
}
|
||||
return readSurrogatePair(high: high, in: text, index: &index)
|
||||
}
|
||||
|
||||
func readSurrogatePair(
|
||||
high: UInt32,
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
text: [Character],
|
||||
index: inout Int,
|
||||
) -> UnicodeScalar? {
|
||||
var probe = index
|
||||
text.formIndex(after: &probe)
|
||||
guard probe < text.endIndex, text[probe] == "\\" else {
|
||||
guard probe + 1 < text.count, text[probe] == "\\", text[probe + 1] == "u" else {
|
||||
return nil
|
||||
}
|
||||
text.formIndex(after: &probe)
|
||||
guard probe < text.endIndex, text[probe] == "u",
|
||||
let low = readHexEscape(in: text, index: &probe, digits: 4),
|
||||
isLowSurrogate(low)
|
||||
else {
|
||||
probe += 2
|
||||
guard let low = readHex(text, index: &probe, digits: 4), isLowSurrogate(low) else {
|
||||
return nil
|
||||
}
|
||||
index = probe
|
||||
@ -325,26 +360,25 @@ extension PureYAML.Parsing.Parser {
|
||||
return UnicodeScalar(value)
|
||||
}
|
||||
|
||||
/// Reads `digits` hex characters following the escape type character. The index enters at the
|
||||
/// type character and is left on the final hex digit; returns nil if fewer than `digits` hex
|
||||
/// characters are available.
|
||||
func readHexEscape(
|
||||
in text: Substring,
|
||||
index: inout Substring.Index,
|
||||
/// Read exactly `digits` hex characters starting at index. On success index
|
||||
/// advances past them, otherwise index is left unchanged.
|
||||
func readHex(
|
||||
_ text: [Character],
|
||||
index: inout Int,
|
||||
digits: Int,
|
||||
) -> UInt32? {
|
||||
var probe = index
|
||||
text.formIndex(after: &probe)
|
||||
var hex = ""
|
||||
for _ in 0 ..< digits {
|
||||
guard probe < text.endIndex, text[probe].isHexDigit else {
|
||||
guard index + digits <= text.count else {
|
||||
return nil
|
||||
}
|
||||
var value: UInt32 = 0
|
||||
for offset in 0 ..< digits {
|
||||
guard let digit = text[index + offset].hexDigitValue else {
|
||||
return nil
|
||||
}
|
||||
hex.append(text[probe])
|
||||
text.formIndex(after: &probe)
|
||||
value = (value << 4) + UInt32(digit)
|
||||
}
|
||||
index = text.index(before: probe)
|
||||
return UInt32(hex, radix: 16)
|
||||
index += digits
|
||||
return value
|
||||
}
|
||||
|
||||
func isHighSurrogate(_ value: UInt32) -> Bool {
|
||||
|
||||
@ -1,167 +0,0 @@
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
func scanDirectiveOrDocumentMarker(_ state: inout State) throws -> Bool {
|
||||
guard state.indentation.last == 0 else {
|
||||
return false
|
||||
}
|
||||
// A `%` opens a directive only outside a block scalar. Inside one (a zero-indent literal or
|
||||
// folded scalar sits at column 0, where this directive check runs) a leading `%` is literal
|
||||
// content, so let the main scanner read the line. Elsewhere the directive path still runs, so
|
||||
// a stray `%` in other content is rejected as before, not silently accepted.
|
||||
if state.reader.peek() == "%", state.blockScalarIndentation == nil {
|
||||
try scanDirective(&state)
|
||||
return true
|
||||
}
|
||||
if state.reader.peek() == "#" {
|
||||
return false
|
||||
}
|
||||
if startsWith("---", state.reader), isIndicatorBoundary(state.reader.peek(offset: 3)) {
|
||||
try scanDocumentMarker(.documentStart, state: &state)
|
||||
resetForDocumentStart(&state)
|
||||
return true
|
||||
}
|
||||
if startsWith("...", state.reader), isIndicatorBoundary(state.reader.peek(offset: 3)) {
|
||||
// A `...` after a document is already closed (only comments or blank lines since the
|
||||
// last `...`) ends nothing: it is a repeated document-end marker, which YAML allows. Skip
|
||||
// it as a no-op rather than emitting a second documentEnd (which would compose a phantom
|
||||
// null document) or rejecting a valid stream. Content after it is still an error.
|
||||
if state.isDocumentClosed {
|
||||
state.reader.advance()
|
||||
state.reader.advance()
|
||||
state.reader.advance()
|
||||
while state.reader.peek() == " " {
|
||||
state.reader.advance()
|
||||
}
|
||||
if let next = state.reader.peek(), !isLineBreak(next), next != "#" {
|
||||
throw PureYAML.Parsing.ParseError.contentAfterDocumentEnd(line: state.reader.mark.line)
|
||||
}
|
||||
skipLine(&state)
|
||||
return true
|
||||
}
|
||||
// A `...` ends a document, so a directive still awaiting its `---` never gets one.
|
||||
try enforceDirectiveHasDocument(state)
|
||||
try scanDocumentMarker(.documentEnd, state: &state)
|
||||
resetForDocumentEnd(&state)
|
||||
return true
|
||||
}
|
||||
// Content after a document-end marker (`...`) begins a new bare document, so reopen the
|
||||
// stream rather than rejecting it.
|
||||
if state.isDocumentClosed {
|
||||
state.isDocumentClosed = false
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func scanDocumentMarker(
|
||||
_ kind: PureYAML.Parsing.TokenKind,
|
||||
state: inout State,
|
||||
) throws {
|
||||
let start = state.reader.mark
|
||||
state.reader.advance()
|
||||
state.reader.advance()
|
||||
state.reader.advance()
|
||||
state.append(kind, mark: start, endMark: state.reader.mark)
|
||||
while state.reader.peek() == " " {
|
||||
state.reader.advance()
|
||||
}
|
||||
if let next = state.reader.peek(), !isLineBreak(next), next != "#" {
|
||||
// The marker's own line carries real content. A document start may carry the root node
|
||||
// here (e.g. `--- >`, `--- foo`, `--- {a: 1}`), so leave it for the main loop to scan.
|
||||
if case .documentStart = kind {
|
||||
state.isAtLineStart = false
|
||||
return
|
||||
}
|
||||
// A document end (`...`) takes no same-line node; only a comment or blank may follow it.
|
||||
throw PureYAML.Parsing.ParseError.contentAfterDocumentEnd(line: state.reader.mark.line)
|
||||
}
|
||||
skipLine(&state)
|
||||
}
|
||||
|
||||
func resetForDocumentStart(_ state: inout State) {
|
||||
state.hasDocumentContent = false
|
||||
state.hasDocumentStartMarker = true
|
||||
state.isDocumentClosed = false
|
||||
state.hasYAMLDirective = false
|
||||
state.pendingDirectiveLine = nil
|
||||
// The directives that preceded this `---` apply to this document only: merge them onto the
|
||||
// defaults, then clear them so they do not leak into the next document.
|
||||
state.tagHandles = State.defaultTagHandles.merging(state.pendingTagHandles) { _, new in new }
|
||||
state.pendingTagHandles = [:]
|
||||
}
|
||||
|
||||
/// A directive must be followed by an explicit `---` document start. Called at a document end
|
||||
/// and at stream end, where a still-pending directive means none ever came.
|
||||
func enforceDirectiveHasDocument(_ state: State) throws {
|
||||
if let line = state.pendingDirectiveLine {
|
||||
throw PureYAML.Parsing.ParseError.directiveWithoutDocument(line: line)
|
||||
}
|
||||
}
|
||||
|
||||
func resetForDocumentEnd(_ state: inout State) {
|
||||
state.hasDocumentContent = false
|
||||
state.hasDocumentStartMarker = false
|
||||
state.isDocumentClosed = true
|
||||
state.hasYAMLDirective = false
|
||||
// A document boundary ends the scope of that document's tag handles.
|
||||
state.tagHandles = State.defaultTagHandles
|
||||
state.pendingTagHandles = [:]
|
||||
}
|
||||
|
||||
func scanDirective(_ state: inout State) throws {
|
||||
let start = state.reader.mark
|
||||
let line = state.reader.consume { character in
|
||||
!isLineBreak(character)
|
||||
}
|
||||
let parts = line.split(separator: " ").map(String.init)
|
||||
guard let name = parts.first else {
|
||||
return
|
||||
}
|
||||
// A directive after a document-end marker (`...`) begins the next document's directive
|
||||
// section, so it reopens the stream for a new document rather than being rejected.
|
||||
if state.isDocumentClosed {
|
||||
state.isDocumentClosed = false
|
||||
}
|
||||
guard !state.hasDocumentContent, !state.hasDocumentStartMarker else {
|
||||
throw PureYAML.Parsing.ParseError.unsupportedDirective(name: name, line: start.line)
|
||||
}
|
||||
// Every directive must be followed by an explicit `---`; record that one is now awaiting it.
|
||||
state.pendingDirectiveLine = start.line
|
||||
switch name {
|
||||
case "%YAML":
|
||||
// Any version with major 1 is processable: a higher minor than 1.2 (`%YAML 1.3`) is
|
||||
// accepted, since the spec calls for processing it, with a warning, rather than rejecting
|
||||
// it. Only a different major version, or a malformed one, is incompatible.
|
||||
guard parts.indices.contains(1), isProcessableYAMLVersion(parts[1]) else {
|
||||
throw PureYAML.Parsing.ParseError.incompatibleYAMLDirective(line: start.line)
|
||||
}
|
||||
// `%YAML` takes exactly one parameter, the version; only a comment may follow it.
|
||||
guard parts.count == 2 || parts[2].hasPrefix("#") else {
|
||||
throw PureYAML.Parsing.ParseError.malformedDirective(name: name, line: start.line)
|
||||
}
|
||||
// At most one `%YAML` directive is allowed per document.
|
||||
guard !state.hasYAMLDirective else {
|
||||
throw PureYAML.Parsing.ParseError.duplicateDirective(name: name, line: start.line)
|
||||
}
|
||||
state.hasYAMLDirective = true
|
||||
case "%TAG":
|
||||
guard parts.count >= 3 else {
|
||||
return
|
||||
}
|
||||
state.pendingTagHandles[parts[1]] = parts[2]
|
||||
default:
|
||||
// A YAML processor ignores a directive it does not recognize (the spec calls for a
|
||||
// warning and otherwise no effect), rather than rejecting the document. The line has
|
||||
// already been consumed, so there is nothing more to do.
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a `%YAML` version is one this processor can attempt: any `major.minor` with major 1,
|
||||
/// so 1.0 through 1.2 and a higher minor like 1.3 are all accepted; a different major (2.0) or a
|
||||
/// non-numeric version is not.
|
||||
func isProcessableYAMLVersion(_ version: String) -> Bool {
|
||||
let components = version.split(separator: ".", omittingEmptySubsequences: false)
|
||||
return components.count == 2
|
||||
&& components[0] == "1"
|
||||
&& components.allSatisfy { !$0.isEmpty && $0.allSatisfy(\.isNumber) }
|
||||
}
|
||||
}
|
||||
@ -1,11 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
struct ScannerFlowContext {
|
||||
var kind: ScannerFlowContextKind
|
||||
var isExpectingKey: Bool
|
||||
/// Whether an explicit key indicator (`?`) is open at this flow level, so the following scalar
|
||||
/// is the explicit key's content and must not synthesize its own implicit key. Cleared by the
|
||||
/// pair's `:` value indicator or a `,`, and discarded with the context when the collection
|
||||
/// closes, so it tracks per-level and a nested collection's `:` does not affect an outer key.
|
||||
var hasExplicitKey = false
|
||||
}
|
||||
}
|
||||
@ -1,89 +0,0 @@
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
/// Emits the implicit-key marker before a block-level flow collection that a `: ` follows, so the
|
||||
/// flow node is parsed as a mapping key (e.g. `[a, b]: value`) through the same path a scalar key
|
||||
/// takes. Skipped inside a flow context, after a mapping value already on the line (block style
|
||||
/// allows one pair per line), and under a pending explicit `?` key, which marks its own key.
|
||||
func appendFlowImplicitKeyMarkerIfNeeded(state: inout State) {
|
||||
guard state.flowDepth == 0, !state.sawMappingValueOnLine, !state.pendingExplicitKey,
|
||||
flowCollectionIsImplicitKey(state: state)
|
||||
else {
|
||||
return
|
||||
}
|
||||
let mark = state.reader.mark
|
||||
state.append(.mappingKey, mark: mark, endMark: mark)
|
||||
}
|
||||
|
||||
/// Looks ahead from a block-level flow collection opener (`[` or `{`) to decide whether the whole
|
||||
/// collection is an implicit mapping key: a flow node immediately followed by a `: ` mapping-value
|
||||
/// indicator, e.g. `[a, b]: value` or `{a: 1}: value`. Quoted scalars are skipped so brackets and
|
||||
/// colons inside them do not affect the nesting count, and the `:` must sit on the same line as
|
||||
/// the close (only spaces between), since a block implicit key carries its `:` on its own line.
|
||||
/// Returns false if the brackets do not balance before end of input.
|
||||
func flowCollectionIsImplicitKey(state: State) -> Bool {
|
||||
var depth = 0
|
||||
var offset = 0
|
||||
while let character = state.reader.peek(offset: offset) {
|
||||
// A block implicit key must lie on a single line, so a flow collection that spans a line
|
||||
// break (outside any quoted scalar) before its `:` cannot be one: `[23\n]: 42` is an error.
|
||||
if isLineBreak(character) {
|
||||
return false
|
||||
}
|
||||
switch character {
|
||||
case "'":
|
||||
offset = skipSingleQuotedLookahead(after: offset, state: state)
|
||||
case "\"":
|
||||
offset = skipDoubleQuotedLookahead(after: offset, state: state)
|
||||
case "[", "{":
|
||||
depth += 1
|
||||
offset += 1
|
||||
case "]", "}":
|
||||
depth -= 1
|
||||
offset += 1
|
||||
guard depth > 0 else {
|
||||
while state.reader.peek(offset: offset) == " " {
|
||||
offset += 1
|
||||
}
|
||||
return state.reader.peek(offset: offset) == ":"
|
||||
&& isBlockMappingValueBoundary(state.reader.peek(offset: offset + 1))
|
||||
}
|
||||
default:
|
||||
offset += 1
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/// Returns the offset just past the closing `'` of a single-quoted scalar that opened at `open`,
|
||||
/// treating a doubled `''` as an escaped quote rather than a close.
|
||||
private func skipSingleQuotedLookahead(after open: Int, state: State) -> Int {
|
||||
var offset = open + 1
|
||||
while let character = state.reader.peek(offset: offset) {
|
||||
if character == "'" {
|
||||
if state.reader.peek(offset: offset + 1) == "'" {
|
||||
offset += 2
|
||||
continue
|
||||
}
|
||||
return offset + 1
|
||||
}
|
||||
offset += 1
|
||||
}
|
||||
return offset
|
||||
}
|
||||
|
||||
/// Returns the offset just past the closing `"` of a double-quoted scalar that opened at `open`,
|
||||
/// skipping `\`-escaped characters so an escaped quote does not end the scan early.
|
||||
private func skipDoubleQuotedLookahead(after open: Int, state: State) -> Int {
|
||||
var offset = open + 1
|
||||
while let character = state.reader.peek(offset: offset) {
|
||||
if character == "\\" {
|
||||
offset += 2
|
||||
continue
|
||||
}
|
||||
if character == "\"" {
|
||||
return offset + 1
|
||||
}
|
||||
offset += 1
|
||||
}
|
||||
return offset
|
||||
}
|
||||
}
|
||||
@ -1,379 +0,0 @@
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
/// A blank line inside a block scalar is preserved as empty content so that
|
||||
/// chomping and folding can apply to it. The guard keeps blank lines outside
|
||||
/// block scalars unaffected; `isAtLineStart` is true here only when the
|
||||
/// previous line already ended, so a content line's own terminating break
|
||||
/// does not emit a blank. Blank lines between the header and the first
|
||||
/// content line precede the indent token, so they are buffered and emitted
|
||||
/// once the content indentation is established.
|
||||
func captureBlockScalarBlankLine(_ state: inout State) {
|
||||
guard state.isAtLineStart else {
|
||||
return
|
||||
}
|
||||
if state.blockScalarIndentation != nil {
|
||||
// The prefix holds any surplus indentation past the block's content level for an
|
||||
// otherwise-empty line (see `preserveBlockScalarSurplusOnBlankLine`); empty otherwise.
|
||||
state.append(.scalar(value: state.blockScalarContentPrefix, style: .plain))
|
||||
state.blockScalarContentPrefix = ""
|
||||
} else if state.pendingBlockScalarIndentation {
|
||||
state.pendingBlockScalarLeadingBlankWhitespace.append(state.lastLeadingWhitespace)
|
||||
}
|
||||
}
|
||||
|
||||
func startsWith(
|
||||
_ prefix: String,
|
||||
_ reader: PureYAML.Parsing.Reader,
|
||||
) -> Bool {
|
||||
for (offset, character) in prefix.enumerated() {
|
||||
guard reader.peek(offset: offset) == character else {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func isLineBreak(_ character: Character?) -> Bool {
|
||||
character == "\n" || character == "\r" || character == "\r\n"
|
||||
}
|
||||
|
||||
func isBlockEntry(_ reader: PureYAML.Parsing.Reader) -> Bool {
|
||||
reader.peek() == "-" && isIndicatorBoundary(reader.peek(offset: 1))
|
||||
}
|
||||
|
||||
func isIndicatorBoundary(_ character: Character?) -> Bool {
|
||||
guard let character else {
|
||||
return true
|
||||
}
|
||||
// A tab is separation white space in YAML (only indentation forbids it), so it ends an
|
||||
// indicator just as a space does: `---\tscalar`, `-\titem`, `?\tkey`.
|
||||
return character == " " || character == "\t" || isLineBreak(character)
|
||||
}
|
||||
|
||||
func isMappingValueBoundary(_ character: Character?) -> Bool {
|
||||
guard let character else {
|
||||
return true
|
||||
}
|
||||
// A tab is separation white space, so it follows a `:` value indicator as a space does.
|
||||
return character == " "
|
||||
|| character == "\t"
|
||||
|| isLineBreak(character)
|
||||
|| character == ","
|
||||
|| character == "]"
|
||||
|| character == "}"
|
||||
}
|
||||
|
||||
func isBlockMappingValueBoundary(_ character: Character?) -> Bool {
|
||||
guard let character else {
|
||||
return true
|
||||
}
|
||||
return character == " " || character == "\t" || isLineBreak(character)
|
||||
}
|
||||
|
||||
func isTokenTerminator(_ character: Character) -> Bool {
|
||||
character == " "
|
||||
|| isLineBreak(character)
|
||||
|| character == ","
|
||||
|| character == "["
|
||||
|| character == "]"
|
||||
|| character == "{"
|
||||
|| character == "}"
|
||||
|| character == "#"
|
||||
}
|
||||
|
||||
func canStartAnchorName(_ character: Character?) -> Bool {
|
||||
guard let character else {
|
||||
return false
|
||||
}
|
||||
return !isTokenTerminator(character)
|
||||
&& character != "*"
|
||||
&& character != "&"
|
||||
&& character != "!"
|
||||
&& character != "\""
|
||||
&& character != "'"
|
||||
}
|
||||
|
||||
func hasNodeOnSameLine(afterCurrentPositionIn reader: PureYAML.Parsing.Reader) -> Bool {
|
||||
var offset = 0
|
||||
while reader.peek(offset: offset) == " " {
|
||||
offset += 1
|
||||
}
|
||||
guard let character = reader.peek(offset: offset) else {
|
||||
return false
|
||||
}
|
||||
return !isLineBreak(character) && character != "#"
|
||||
}
|
||||
|
||||
func isFlowDelimiter(_ character: Character) -> Bool {
|
||||
character == ","
|
||||
|| character == "["
|
||||
|| character == "]"
|
||||
|| character == "{"
|
||||
|| character == "}"
|
||||
}
|
||||
|
||||
func isBlockScalarHeaderCharacter(_ character: Character) -> Bool {
|
||||
character == "+"
|
||||
|| character == "-"
|
||||
|| character.isNumber
|
||||
}
|
||||
|
||||
func trimTrailingSpaces(_ value: String) -> String {
|
||||
var end = value.endIndex
|
||||
while end > value.startIndex {
|
||||
let previous = value.index(before: end)
|
||||
guard value[previous] == " " else {
|
||||
break
|
||||
}
|
||||
end = previous
|
||||
}
|
||||
return String(value[..<end])
|
||||
}
|
||||
|
||||
func scanPlainScalarContinuations(
|
||||
after firstLine: String,
|
||||
firstLineIndent: Int,
|
||||
state: inout State,
|
||||
) -> String {
|
||||
var value = firstLine
|
||||
while true {
|
||||
var probe = state.reader
|
||||
guard isLineBreak(probe.peek()) else {
|
||||
return value
|
||||
}
|
||||
|
||||
probe.advance()
|
||||
let blankLineCount = consumeBlankLines(&probe)
|
||||
|
||||
_ = probe.consume { character in
|
||||
character == " "
|
||||
}
|
||||
let indentationWidth = probe.mark.column - 1
|
||||
|
||||
guard !probe.isAtEnd, !isLineBreak(probe.peek()), probe.peek() != "#" else {
|
||||
return value
|
||||
}
|
||||
|
||||
// A plain scalar never spans a document marker (`---` / `...`); stop the continuation at
|
||||
// one so it begins (or ends) the document rather than folding into the scalar's value.
|
||||
if indentationWidth == 0, isAtDocumentMarker(probe) {
|
||||
return value
|
||||
}
|
||||
|
||||
let baseIndentation = plainScalarContinuationBaseIndentation(state: state, firstLineIndent: firstLineIndent)
|
||||
guard indentationWidth > baseIndentation else {
|
||||
return value
|
||||
}
|
||||
if isBlockEntry(probe), state.validImplicitIndentation.contains(indentationWidth) {
|
||||
return value
|
||||
}
|
||||
|
||||
var continuationState = state
|
||||
continuationState.reader = probe
|
||||
let continuation = scanPlainScalarContinuationLine(&continuationState)
|
||||
if isAtMappingValueIndicator(state: continuationState) {
|
||||
// A continuation line ending at `:` is normally a new implicit key, so the scalar
|
||||
// stops before it. Two cases keep folding instead, with the `:` left to be scanned as
|
||||
// the value indicator: a block explicit key (`? foo` then `bar :`), where the line is
|
||||
// still the key's content, and any flow scalar, since a flow key may span lines
|
||||
// (`{ multi`/`line: v }`) where a block implicit key may not.
|
||||
let foldsAcrossMappingValue = state.flowDepth > 0 || state.pendingExplicitKey
|
||||
guard foldsAcrossMappingValue else {
|
||||
return value
|
||||
}
|
||||
state = continuationState
|
||||
value += blankLineCount == 0 ? " " : String(repeating: "\n", count: blankLineCount)
|
||||
value += continuation
|
||||
return value
|
||||
}
|
||||
|
||||
state = continuationState
|
||||
if blankLineCount == 0 {
|
||||
value += " "
|
||||
} else {
|
||||
value += String(repeating: "\n", count: blankLineCount)
|
||||
}
|
||||
value += continuation
|
||||
}
|
||||
}
|
||||
|
||||
/// Advance `probe` over a run of blank lines (lines that are empty or only spaces), returning
|
||||
/// how many were consumed; the probe is left at the start of the next non-blank line.
|
||||
func consumeBlankLines(_ probe: inout PureYAML.Parsing.Reader) -> Int {
|
||||
var count = 0
|
||||
while true {
|
||||
var blankProbe = probe
|
||||
// A line of only white space is empty for folding, including one that holds a tab: a tab
|
||||
// is not indentation, but a whitespace-only line is still blank, so consume it here.
|
||||
_ = blankProbe.consume { character in
|
||||
character == " " || character == "\t"
|
||||
}
|
||||
guard isLineBreak(blankProbe.peek()) else {
|
||||
break
|
||||
}
|
||||
blankProbe.advance()
|
||||
probe = blankProbe
|
||||
count += 1
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
/// Whether the reader is positioned at a document marker (`---` or `...`) followed by an
|
||||
/// indicator boundary, i.e. the marker that starts or ends a document.
|
||||
func isAtDocumentMarker(_ reader: PureYAML.Parsing.Reader) -> Bool {
|
||||
(startsWith("---", reader) || startsWith("...", reader)) && isIndicatorBoundary(reader.peek(offset: 3))
|
||||
}
|
||||
|
||||
func plainScalarContinuationBaseIndentation(state: State, firstLineIndent: Int) -> Int {
|
||||
// When the scalar begins exactly at the current indentation top, it is a block value on its
|
||||
// own line, so its own indent was pushed onto the stack; its continuation lines sit at that
|
||||
// same indent, and the level they must exceed is the enclosing block's: the indent below on
|
||||
// the stack, or -1 at the document root, where nothing encloses the scalar and a
|
||||
// continuation may sit at column 1. For an inline value the top of the stack already is the
|
||||
// enclosing block's indent, so it is used directly.
|
||||
if state.indentation.last == firstLineIndent {
|
||||
return state.indentation.count >= 2 ? state.indentation[state.indentation.count - 2] : -1
|
||||
}
|
||||
return state.indentation.last ?? 0
|
||||
}
|
||||
|
||||
func scanPlainScalarContinuationLine(_ state: inout State) -> String {
|
||||
// Leading white space beyond the line's indentation folds away in a plain scalar. A tab is
|
||||
// not indentation, so the indentation scan leaves it here; skip it (and any spaces after it)
|
||||
// so a continuation line like `\t3rd` joins as `3rd`, not `\t3rd`.
|
||||
while let character = state.reader.peek(), character == " " || character == "\t" {
|
||||
state.reader.advance()
|
||||
}
|
||||
var value = ""
|
||||
while let character = state.reader.peek(), !isLineBreak(character) {
|
||||
if character == " ", state.reader.peek(offset: 1) == "#" {
|
||||
break
|
||||
}
|
||||
if character == "#", value.isEmpty || value.last == " " {
|
||||
break
|
||||
}
|
||||
if character == ":", shouldEndPlainScalarAtMappingValue(state: state) {
|
||||
break
|
||||
}
|
||||
if state.flowDepth > 0, isFlowDelimiter(character) {
|
||||
break
|
||||
}
|
||||
value.append(character)
|
||||
state.reader.advance()
|
||||
}
|
||||
return trimTrailingSpaces(value)
|
||||
}
|
||||
|
||||
func skipLine(_ state: inout State) {
|
||||
_ = state.reader.consume { character in
|
||||
!isLineBreak(character)
|
||||
}
|
||||
}
|
||||
|
||||
func shouldEndPlainScalar(
|
||||
_ character: Character,
|
||||
state: State,
|
||||
accumulated: String,
|
||||
) -> Bool {
|
||||
if isLineBreak(character) {
|
||||
return true
|
||||
}
|
||||
// A comment opens after white space, a tab as well as a space, so a `#` preceded by either
|
||||
// ends the plain scalar: `d\t#X` is the value `d` with a trailing comment.
|
||||
if character == " " || character == "\t", state.reader.peek(offset: 1) == "#" {
|
||||
return true
|
||||
}
|
||||
if character == "#", accumulated.isEmpty || accumulated.last == " " || accumulated.last == "\t" {
|
||||
return true
|
||||
}
|
||||
if character == ":", shouldEndPlainScalarAtMappingValue(state: state) {
|
||||
return true
|
||||
}
|
||||
if state.flowDepth > 0, isFlowDelimiter(character) {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func shouldEndPlainScalarAtMappingValue(state: State) -> Bool {
|
||||
if state.flowDepth > 0 {
|
||||
return state.isExpectingFlowMappingKey
|
||||
|| isMappingValueBoundary(state.reader.peek(offset: 1))
|
||||
}
|
||||
return isBlockMappingValueBoundary(state.reader.peek(offset: 1))
|
||||
}
|
||||
|
||||
func isAtMappingValueIndicator(state: State) -> Bool {
|
||||
state.reader.peek() == ":" && shouldEndPlainScalarAtMappingValue(state: state)
|
||||
}
|
||||
|
||||
/// Whether a mapping-value indicator (`: ` or `:` at line end) follows, allowing spaces before
|
||||
/// the colon. Used after a quoted scalar, where the reader stops at the closing quote and any
|
||||
/// separating spaces remain, unlike a plain scalar that ends at the colon itself.
|
||||
func isAtMappingValueIndicatorAfterSpaces(state: State) -> Bool {
|
||||
var offset = 0
|
||||
while state.reader.peek(offset: offset) == " " {
|
||||
offset += 1
|
||||
}
|
||||
return state.reader.peek(offset: offset) == ":"
|
||||
&& isBlockMappingValueBoundary(state.reader.peek(offset: offset + 1))
|
||||
}
|
||||
|
||||
@discardableResult
|
||||
func appendMappingKeyIfNeeded(
|
||||
start: PureYAML.Parsing.Mark,
|
||||
state: inout State,
|
||||
) -> Bool {
|
||||
guard isAtMappingValueIndicator(state: state), canSynthesizeImplicitBlockKey(state) else {
|
||||
return false
|
||||
}
|
||||
state.append(.mappingKey, mark: start, endMark: start)
|
||||
return true
|
||||
}
|
||||
|
||||
/// Whether a scalar followed by `:` may start an implicit mapping key here. In block style a
|
||||
/// line holds at most one mapping pair, so once a block-level `:` has appeared on the line a
|
||||
/// further key would form a forbidden inline nested mapping; in flow style several pairs per
|
||||
/// line are allowed, so that restriction does not apply. In flow a scalar that is the content of
|
||||
/// an open explicit key (`?`) is the key itself, not a new implicit key.
|
||||
/// Records the implicit indentation levels a block entry with an inline node introduces: the
|
||||
/// node's own indent (one past the dash), and the dash's indent as a sibling level so an inline
|
||||
/// item's plain value stops folding at a later entry that returns to it (`: -␣b` then ` - - c`).
|
||||
/// The sibling level is skipped under a pending explicit key (`? - x`), whose sequence key drives
|
||||
/// implicit indentation itself.
|
||||
func recordBlockEntryImplicitIndentation(start: PureYAML.Parsing.Mark, state: inout State) {
|
||||
guard hasNodeOnSameLine(afterCurrentPositionIn: state.reader) else {
|
||||
return
|
||||
}
|
||||
state.validImplicitIndentation.insert(start.column + 1)
|
||||
if !state.pendingExplicitKey {
|
||||
state.validImplicitIndentation.insert(start.column - 1)
|
||||
}
|
||||
}
|
||||
|
||||
func canSynthesizeImplicitBlockKey(_ state: State) -> Bool {
|
||||
if state.flowDepth > 0 {
|
||||
return state.flowContexts.last?.hasExplicitKey != true
|
||||
}
|
||||
return !state.sawMappingValueOnLine
|
||||
}
|
||||
|
||||
/// A `:` with no node before it on the line is an implicit empty mapping key, unless it is the
|
||||
/// value indicator of a pending explicit `?` key. This covers both a bare `: value` and a line
|
||||
/// whose only prior content is a node property (`&a : value`, `!!str : value`): the property
|
||||
/// decorates the empty key, which the preceding property tokens bind to. Synthesizes the empty
|
||||
/// key and its node so the block mapping machinery treats the pair uniformly.
|
||||
func synthesizeNullMappingKeyIfNeeded(
|
||||
_ character: Character,
|
||||
hadNonPropertyContentOnLine: Bool,
|
||||
state: inout State,
|
||||
) {
|
||||
guard character == ":", !hadNonPropertyContentOnLine, state.flowDepth == 0,
|
||||
!state.pendingExplicitKey, isAtMappingValueIndicator(state: state)
|
||||
else {
|
||||
return
|
||||
}
|
||||
let mark = state.reader.mark
|
||||
state.append(.mappingKey, mark: mark, endMark: mark)
|
||||
state.append(.scalar(value: "", style: .plain), mark: mark, endMark: mark)
|
||||
}
|
||||
}
|
||||
@ -1,312 +0,0 @@
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
func scanIndentation(_ state: inout State) throws {
|
||||
let start = state.reader.mark
|
||||
let width = try scanIndentationWidth(&state)
|
||||
guard shouldProcessIndentation(state) else {
|
||||
preserveBlockScalarSurplusOnBlankLine(width: width, state: &state)
|
||||
recordPendingBlockScalarBlankIndent(width: width, state: &state)
|
||||
return
|
||||
}
|
||||
|
||||
let end = state.reader.mark
|
||||
let current = state.indentation.last ?? 0
|
||||
let wasInBlockScalar = state.blockScalarIndentation != nil
|
||||
|
||||
// A document marker (`---` / `...`) at column 0 ends an active block scalar rather than
|
||||
// being its content. Close the scalar's indentation here, dedenting back to the document
|
||||
// level, so the marker is then scanned balanced at the document level. Without this the
|
||||
// scalar's indent lingers past the marker and surfaces later as a stray dedent plus a
|
||||
// phantom empty node (the marker still ends the document, but the structure is unbalanced).
|
||||
if state.blockScalarIndentation != nil, width == 0, isAtDocumentMarker(state.reader) {
|
||||
state.blockScalarIndentation = nil
|
||||
state.blockScalarContentPrefix = ""
|
||||
while state.indentation.count > 1 {
|
||||
state.indentation.removeLast()
|
||||
state.append(.dedent(width: state.indentation.last ?? 0), mark: end, endMark: end)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if consumeBlockScalarContentIndentation(width: width, state: &state) {
|
||||
return
|
||||
}
|
||||
|
||||
// An active block scalar that just ended (its content was less indented this line) onto a
|
||||
// line beginning with '#' is leaving an ordinary comment. A comment's own indentation does
|
||||
// not participate in block structure, so leave it for the main loop to scan rather than
|
||||
// forcing its column to match an indentation level, which would dedent onto a width no stack
|
||||
// level holds and raise a spurious error. The guard on `wasInBlockScalar` keeps a block
|
||||
// scalar's own first content line that begins with '#' (e.g. `|-` then `# text`) as content.
|
||||
if wasInBlockScalar, state.reader.peek() == "#" {
|
||||
return
|
||||
}
|
||||
|
||||
if let indicator = state.pendingBlockScalarIndentationIndicator {
|
||||
try applyExplicitBlockScalarIndentation(
|
||||
indicator: indicator,
|
||||
width: width,
|
||||
current: current,
|
||||
marks: (start: start, end: end),
|
||||
state: &state,
|
||||
)
|
||||
} else if width > current {
|
||||
try applyIncreasedIndentation(width: width, start: start, end: end, state: &state)
|
||||
} else if state.pendingBlockScalarIndentation, state.pendingBlockScalarIsRootNode, width == current {
|
||||
// A root block scalar (`>`/`|` as the whole document) has the document node as its
|
||||
// parent, whose indentation is -1, so content at column 0 (width == current == 0) is
|
||||
// still more indented than its parent and must establish the block scalar's content
|
||||
// indentation here, where `width > current` (with the base level read as 0) cannot fire.
|
||||
// A non-root block scalar (a sequence item or mapping value) keeps its parent's indent as
|
||||
// the floor, so its under-indented content still falls through to an error, unchanged.
|
||||
try applyIncreasedIndentation(width: width, start: start, end: end, state: &state)
|
||||
} else if width < current {
|
||||
try applyDedentation(width: width, start: start, end: end, state: &state)
|
||||
}
|
||||
|
||||
pruneImplicitIndentation(&state)
|
||||
clearPendingBlockScalarIndentation(&state)
|
||||
}
|
||||
|
||||
/// Records a leading blank line's indentation while a block scalar's content indentation is
|
||||
/// still pending (before its first content line), so an over-indented leading line can be
|
||||
/// caught once the content indentation is established.
|
||||
func recordPendingBlockScalarBlankIndent(width _: Int, state: inout State) {
|
||||
guard state.pendingBlockScalarIndentation, state.blockScalarIndentation == nil else {
|
||||
return
|
||||
}
|
||||
// Compare leading spaces, not the tab-expanded width: a tab on a blank line is not
|
||||
// indentation, so `␣\t` counts as one space, not a jump to the next tab stop.
|
||||
state.pendingBlockScalarMaxBlankIndent = max(state.pendingBlockScalarMaxBlankIndent, state.lastLeadingSpaceCount)
|
||||
}
|
||||
|
||||
/// A block scalar's leading empty lines must not be more indented than its content, so reject
|
||||
/// the scalar if the widest leading blank exceeds the content indentation.
|
||||
func enforceLeadingBlankIndentation(contentIndent: Int, line: Int, state: State) throws {
|
||||
guard state.pendingBlockScalarMaxBlankIndent > contentIndent else {
|
||||
return
|
||||
}
|
||||
throw PureYAML.Parsing.ParseError.unexpectedIndentation(line: line)
|
||||
}
|
||||
|
||||
/// A whitespace-only line inside an active block scalar that is indented past the block's
|
||||
/// content indentation carries the surplus spaces as content (e.g. a 3-space line in a 2-space
|
||||
/// block contributes `" "`). The width is consumed before the line is recognized as blank, so
|
||||
/// stash the surplus in the content prefix for the blank-line emission to pick up; otherwise the
|
||||
/// line collapses to empty and a space is dropped from the scalar's value.
|
||||
func preserveBlockScalarSurplusOnBlankLine(width: Int, state: inout State) {
|
||||
guard let blockScalarIndentation = state.blockScalarIndentation, width > blockScalarIndentation else {
|
||||
return
|
||||
}
|
||||
state.blockScalarContentPrefix = blockScalarSurplusPrefix(beyond: blockScalarIndentation, whitespace: state.lastLeadingWhitespace)
|
||||
}
|
||||
|
||||
func scanIndentationWidth(_ state: inout State) throws -> Int {
|
||||
var width = 0
|
||||
var leadingSpaces = 0
|
||||
var sawTab = false
|
||||
var whitespace = ""
|
||||
while let character = state.reader.peek() {
|
||||
if character == " " {
|
||||
width += 1
|
||||
if !sawTab {
|
||||
leadingSpaces += 1
|
||||
}
|
||||
whitespace.append(character)
|
||||
state.reader.advance()
|
||||
} else if character == "\t" {
|
||||
sawTab = true
|
||||
width = nextTabStop(after: width)
|
||||
whitespace.append(character)
|
||||
state.reader.advance()
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
state.lastLeadingSpaceCount = leadingSpaces
|
||||
state.lastLeadingWhitespace = whitespace
|
||||
return width
|
||||
}
|
||||
|
||||
func nextTabStop(after width: Int) -> Int {
|
||||
((width / 4) + 1) * 4
|
||||
}
|
||||
|
||||
/// The leading-whitespace characters that lie beyond a block scalar's space-only `indent`, kept
|
||||
/// verbatim so a tab in the content (` \ttext` under a 2-space indent) stays a tab rather than
|
||||
/// being re-expanded to spaces. Walks the recorded whitespace, counting columns the way the width
|
||||
/// scan does, and returns the remainder once the indentation columns are covered.
|
||||
func blockScalarSurplusPrefix(beyond indent: Int, whitespace: String) -> String {
|
||||
var column = 0
|
||||
var index = whitespace.startIndex
|
||||
while index < whitespace.endIndex, column < indent {
|
||||
let character = whitespace[index]
|
||||
column = character == "\t" ? nextTabStop(after: column) : column + 1
|
||||
index = whitespace.index(after: index)
|
||||
}
|
||||
return String(whitespace[index...])
|
||||
}
|
||||
|
||||
func shouldProcessIndentation(_ state: State) -> Bool {
|
||||
guard !state.reader.isAtEnd, !isLineBreak(state.reader.peek()) else {
|
||||
return false
|
||||
}
|
||||
// A leading '#' normally starts a comment line, which must not drive
|
||||
// indent/dedent processing. Inside a block scalar it is different: when
|
||||
// the header is still pending or the content is active, a leading '#' is
|
||||
// literal scalar content, so indentation must be processed so the block
|
||||
// scalar's indentation resolves and its content lines are consumed.
|
||||
// Bailing here would leave `blockScalarIndentation` unset and the main
|
||||
// loop would mis-scan the line as a comment, dropping the content.
|
||||
if state.reader.peek() == "#", !state.pendingBlockScalarIndentation, state.blockScalarIndentation == nil {
|
||||
return false
|
||||
}
|
||||
return state.flowDepth == 0
|
||||
}
|
||||
|
||||
func consumeBlockScalarContentIndentation(
|
||||
width: Int,
|
||||
state: inout State,
|
||||
) -> Bool {
|
||||
guard let blockScalarIndentation = state.blockScalarIndentation else {
|
||||
return false
|
||||
}
|
||||
guard width < blockScalarIndentation else {
|
||||
state.blockScalarContentPrefix = blockScalarSurplusPrefix(beyond: blockScalarIndentation, whitespace: state.lastLeadingWhitespace)
|
||||
return true
|
||||
}
|
||||
state.blockScalarIndentation = nil
|
||||
state.blockScalarContentPrefix = ""
|
||||
return false
|
||||
}
|
||||
|
||||
func applyExplicitBlockScalarIndentation(
|
||||
indicator: Int,
|
||||
width: Int,
|
||||
current: Int,
|
||||
marks: (start: PureYAML.Parsing.Mark, end: PureYAML.Parsing.Mark),
|
||||
state: inout State,
|
||||
) throws {
|
||||
// The indicator is the content indentation relative to the block scalar's parent node. For an
|
||||
// inline mapping value (`- aaa: |2`), the parent mapping sits at an implicit indentation that
|
||||
// never reached the main stack, so take the deepest applicable implicit level as the base:
|
||||
// `|2` under a mapping at column 2 makes the content indentation 4. A direct sequence item
|
||||
// (`- |2`) has no such parent and keeps the stack's current level.
|
||||
let parentIndentation = state.pendingBlockScalarInlineMappingValue
|
||||
? max(current, state.validImplicitIndentation.filter { $0 <= width }.max() ?? current)
|
||||
: current
|
||||
let blockScalarIndentation = parentIndentation + indicator
|
||||
guard width >= blockScalarIndentation else {
|
||||
throw PureYAML.Parsing.ParseError.unexpectedIndentation(line: marks.end.line)
|
||||
}
|
||||
try enforceLeadingBlankIndentation(contentIndent: blockScalarIndentation, line: marks.end.line, state: state)
|
||||
state.indentation.append(blockScalarIndentation)
|
||||
state.append(.indent(width: blockScalarIndentation), mark: marks.start, endMark: marks.end)
|
||||
state.blockScalarIndentation = blockScalarIndentation
|
||||
state.blockScalarContentPrefix = String(repeating: " ", count: width - blockScalarIndentation)
|
||||
emitBufferedBlockScalarLeadingBlanks(&state)
|
||||
clearPendingBlockScalarIndentation(&state)
|
||||
}
|
||||
|
||||
/// Leading blank lines (between the header and the first content line) are
|
||||
/// buffered until the indent token is emitted, then flushed as empty content
|
||||
/// so they land after the indent and before the first content line.
|
||||
func emitBufferedBlockScalarLeadingBlanks(_ state: inout State) {
|
||||
// Each leading blank line carries any whitespace past the (now known) content indentation as
|
||||
// content: an otherwise-blank `␣\t` line under a one-space indent yields the tab. With no
|
||||
// content indentation established (an empty block scalar), the line is simply empty.
|
||||
let indent = state.blockScalarIndentation
|
||||
for whitespace in state.pendingBlockScalarLeadingBlankWhitespace {
|
||||
let surplus = indent.map { blockScalarSurplusPrefix(beyond: $0, whitespace: whitespace) } ?? ""
|
||||
state.append(.scalar(value: surplus, style: .plain))
|
||||
}
|
||||
state.pendingBlockScalarLeadingBlankWhitespace = []
|
||||
}
|
||||
|
||||
func applyIncreasedIndentation(
|
||||
width: Int,
|
||||
start: PureYAML.Parsing.Mark,
|
||||
end: PureYAML.Parsing.Mark,
|
||||
state: inout State,
|
||||
) throws {
|
||||
guard state.pendingBlockScalarIndentation else {
|
||||
state.indentation.append(width)
|
||||
state.append(.indent(width: width), mark: start, endMark: end)
|
||||
return
|
||||
}
|
||||
try enforceLeadingBlankIndentation(contentIndent: width, line: end.line, state: state)
|
||||
state.indentation.append(width)
|
||||
state.append(.indent(width: width), mark: start, endMark: end)
|
||||
state.blockScalarIndentation = width
|
||||
state.blockScalarContentPrefix = ""
|
||||
emitBufferedBlockScalarLeadingBlanks(&state)
|
||||
clearPendingBlockScalarIndentation(&state)
|
||||
}
|
||||
|
||||
func applyDedentation(
|
||||
width: Int,
|
||||
start: PureYAML.Parsing.Mark,
|
||||
end: PureYAML.Parsing.Mark,
|
||||
state: inout State,
|
||||
) throws {
|
||||
while state.indentation.count > 1, width < (state.indentation.last ?? 0) {
|
||||
state.indentation.removeLast()
|
||||
state.append(.dedent(width: state.indentation.last ?? 0), mark: end, endMark: end)
|
||||
}
|
||||
if canUseImplicitIndentation(width: width, state: state) {
|
||||
state.indentation.append(width)
|
||||
state.append(.indent(width: width), mark: start, endMark: end)
|
||||
}
|
||||
guard width == state.indentation.last else {
|
||||
throw PureYAML.Parsing.ParseError.unexpectedIndentation(line: end.line)
|
||||
}
|
||||
}
|
||||
|
||||
func canUseImplicitIndentation(
|
||||
width: Int,
|
||||
state: State,
|
||||
) -> Bool {
|
||||
width != state.indentation.last
|
||||
&& width > (state.indentation.last ?? 0)
|
||||
&& state.validImplicitIndentation.contains(width)
|
||||
}
|
||||
|
||||
func pruneImplicitIndentation(_ state: inout State) {
|
||||
let current = state.indentation.last ?? 0
|
||||
state.validImplicitIndentation = state.validImplicitIndentation.filter { $0 <= current }
|
||||
}
|
||||
|
||||
func clearPendingBlockScalarIndentation(_ state: inout State) {
|
||||
state.pendingBlockScalarIndentation = false
|
||||
state.pendingBlockScalarIndentationIndicator = nil
|
||||
state.pendingBlockScalarLeadingBlankWhitespace = []
|
||||
state.pendingBlockScalarMaxBlankIndent = 0
|
||||
state.pendingBlockScalarIsRootNode = false
|
||||
state.pendingBlockScalarInlineMappingValue = false
|
||||
}
|
||||
|
||||
func closeIndentation(_ state: inout State) {
|
||||
flushEmptyBlockScalarBlanks(&state)
|
||||
let mark = state.reader.mark
|
||||
while state.indentation.count > 1 {
|
||||
state.indentation.removeLast()
|
||||
state.append(.dedent(width: state.indentation.last ?? 0), mark: mark, endMark: mark)
|
||||
}
|
||||
}
|
||||
|
||||
/// A block scalar whose only lines are blank never establishes a content indentation, so its
|
||||
/// blank lines stay buffered and are lost. At stream end, flush them as content wrapped in an
|
||||
/// indent/dedent pair, so the parser builds a block scalar of those blank lines and chomping
|
||||
/// applies: `keep: |+` over one blank line keeps a trailing line break, while clip and strip trim
|
||||
/// it away. (Only the stream-end case is handled here, where no later line constrains ordering.)
|
||||
func flushEmptyBlockScalarBlanks(_ state: inout State) {
|
||||
guard state.pendingBlockScalarIndentation, state.blockScalarIndentation == nil,
|
||||
!state.pendingBlockScalarLeadingBlankWhitespace.isEmpty
|
||||
else {
|
||||
return
|
||||
}
|
||||
state.append(.indent(width: (state.indentation.last ?? 0) + 1))
|
||||
emitBufferedBlockScalarLeadingBlanks(&state)
|
||||
state.append(.dedent(width: state.indentation.last ?? 0))
|
||||
clearPendingBlockScalarIndentation(&state)
|
||||
}
|
||||
}
|
||||
@ -1,191 +0,0 @@
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
func scanComment(_ state: inout State) {
|
||||
let start = state.reader.mark
|
||||
state.reader.advance()
|
||||
if state.reader.peek() == " " {
|
||||
state.reader.advance()
|
||||
}
|
||||
let value = state.reader.consume { character in
|
||||
!isLineBreak(character)
|
||||
}
|
||||
state.append(.comment(value), mark: start, endMark: state.reader.mark)
|
||||
}
|
||||
|
||||
func scanQuotedScalar(
|
||||
_ style: PureYAML.Parsing.ScalarStyle,
|
||||
state: inout State,
|
||||
) throws {
|
||||
let start = state.reader.mark
|
||||
let quote: Character = style == .doubleQuoted ? "\"" : "'"
|
||||
state.reader.advance()
|
||||
var value = ""
|
||||
var previousWasEscape = false
|
||||
while let character = state.reader.peek() {
|
||||
if style == .singleQuoted, character == quote, state.reader.peek(offset: 1) == quote {
|
||||
value.append(character)
|
||||
state.reader.advance()
|
||||
value.append(character)
|
||||
state.reader.advance()
|
||||
continue
|
||||
}
|
||||
if character == quote, !previousWasEscape {
|
||||
break
|
||||
}
|
||||
value.append(character)
|
||||
previousWasEscape = character == "\\" && style == .doubleQuoted && !previousWasEscape
|
||||
state.reader.advance()
|
||||
if character != "\\" {
|
||||
previousWasEscape = false
|
||||
}
|
||||
// A line that begins with a document marker (`---` / `...`) is forbidden inside a
|
||||
// multi-line scalar: the marker would be ambiguous with a document boundary.
|
||||
if isLineBreak(character), isAtDocumentMarker(state.reader) {
|
||||
throw PureYAML.Parsing.ParseError.documentMarkerInScalar(line: state.reader.mark.line)
|
||||
}
|
||||
}
|
||||
guard state.reader.peek() == quote else {
|
||||
throw PureYAML.Parsing.ParseError.unterminatedQuotedString(line: start.line)
|
||||
}
|
||||
state.reader.advance()
|
||||
// Unlike a plain scalar (which consumes the space before a `:` into itself), a quoted scalar
|
||||
// stops at its closing quote, so the reader sits at any spaces before a `:`. A space between
|
||||
// an implicit key and its mapping-value indicator is allowed, so look past them.
|
||||
if isAtMappingValueIndicatorAfterSpaces(state: state), canSynthesizeImplicitBlockKey(state) {
|
||||
// A block implicit key must lie on a single line, so a quoted scalar that spanned a line
|
||||
// break (its open and close quotes sit on different lines) cannot be one. The rule is
|
||||
// block-only: a flow mapping key may span lines, so the check is gated on block context.
|
||||
// An explicit key (`? ...`) carries no `:` directly after the scalar, so never reaches here.
|
||||
guard start.line == state.reader.mark.line || state.flowDepth > 0 else {
|
||||
throw PureYAML.Parsing.ParseError.multilineImplicitKey(line: start.line)
|
||||
}
|
||||
state.append(.mappingKey, mark: start, endMark: start)
|
||||
}
|
||||
state.append(.scalar(value: value, style: style), mark: start, endMark: state.reader.mark)
|
||||
}
|
||||
|
||||
func scanBlockScalarHeader(
|
||||
_ style: PureYAML.Parsing.BlockScalarStyle,
|
||||
state: inout State,
|
||||
) {
|
||||
let start = state.reader.mark
|
||||
var chomping = PureYAML.Parsing.BlockScalarChomping.clip
|
||||
var indentationIndicator: Int?
|
||||
state.reader.advance()
|
||||
while let character = state.reader.peek(), isBlockScalarHeaderCharacter(character) {
|
||||
if character == "-" {
|
||||
chomping = .strip
|
||||
} else if character == "+" {
|
||||
chomping = .keep
|
||||
} else if character.isNumber {
|
||||
indentationIndicator = Int(String(character))
|
||||
}
|
||||
state.reader.advance()
|
||||
}
|
||||
state.append(
|
||||
.blockScalarHeader(style: style, chomping: chomping),
|
||||
mark: start,
|
||||
endMark: state.reader.mark,
|
||||
)
|
||||
state.pendingBlockScalarIndentation = true
|
||||
state.pendingBlockScalarIndentationIndicator = indentationIndicator
|
||||
// A root block scalar (the document's own node) has no enclosing collection: it sits at the
|
||||
// base indentation with no mapping value, explicit key, block-entry implicit indent, or flow
|
||||
// context introducing it on this line. Only then is its parent the document (indentation -1),
|
||||
// letting its content begin at column 0. A sequence item or mapping value keeps its parent's
|
||||
// indent as the floor instead.
|
||||
state.pendingBlockScalarIsRootNode = state.indentation == [0]
|
||||
&& !state.sawMappingValueOnLine
|
||||
&& !state.pendingExplicitKey
|
||||
&& state.validImplicitIndentation.isEmpty
|
||||
&& state.flowDepth == 0
|
||||
// Whether this block scalar is an inline mapping value (`- key: |2`). Such a value's parent is
|
||||
// the mapping, which sits at an implicit indentation off the main stack, so an explicit
|
||||
// indicator is measured relative to it. A direct sequence item (`- |2`) has no mapping value
|
||||
// on its line and keeps the sequence's own indentation as the base.
|
||||
state.pendingBlockScalarInlineMappingValue = state.sawMappingValueOnLine && state.flowDepth == 0
|
||||
}
|
||||
|
||||
func scanBlockScalarContent(_ state: inout State) {
|
||||
let start = state.reader.mark
|
||||
let value = state.blockScalarContentPrefix + state.reader.consume { character in
|
||||
!isLineBreak(character)
|
||||
}
|
||||
state.blockScalarContentPrefix = ""
|
||||
state.append(.scalar(value: value, style: .plain), mark: start, endMark: state.reader.mark)
|
||||
}
|
||||
|
||||
func scanNamedToken(
|
||||
kind: (String) -> PureYAML.Parsing.TokenKind,
|
||||
missingNameError: (Int) -> PureYAML.Parsing.ParseError,
|
||||
canBeImplicitKey: Bool = false,
|
||||
state: inout State,
|
||||
) throws {
|
||||
let start = state.reader.mark
|
||||
state.reader.advance()
|
||||
let name = state.reader.consume { character in
|
||||
!isTokenTerminator(character)
|
||||
}
|
||||
guard !name.isEmpty else {
|
||||
throw missingNameError(start.line)
|
||||
}
|
||||
// An alias node may serve as an implicit mapping key (`*a : value`). As with a scalar key, a
|
||||
// `:` after optional spaces marks it as the key, so emit the key indicator before the alias.
|
||||
// An anchor is a node property, never a key, so it never asks for this. An alias name cannot
|
||||
// span lines, so the single-line implicit-key rule needs no separate check here.
|
||||
let isAliasImplicitKey = canBeImplicitKey && state.flowDepth == 0
|
||||
&& isAtMappingValueIndicatorAfterSpaces(state: state) && canSynthesizeImplicitBlockKey(state)
|
||||
if isAliasImplicitKey {
|
||||
state.append(.mappingKey, mark: start, endMark: start)
|
||||
}
|
||||
state.append(kind(name), mark: start, endMark: state.reader.mark)
|
||||
}
|
||||
|
||||
func scanTag(_ state: inout State) throws {
|
||||
let start = state.reader.mark
|
||||
state.reader.advance()
|
||||
var value = "!"
|
||||
if state.reader.peek() == "<" {
|
||||
value.append(state.reader.advance() ?? "<")
|
||||
var didClose = false
|
||||
while let character = state.reader.peek() {
|
||||
value.append(character)
|
||||
state.reader.advance()
|
||||
if character == ">" {
|
||||
didClose = true
|
||||
break
|
||||
}
|
||||
}
|
||||
guard didClose else {
|
||||
throw PureYAML.Parsing.ParseError.unterminatedTag(line: start.line)
|
||||
}
|
||||
} else {
|
||||
value += state.reader.consume { character in
|
||||
!isTokenTerminator(character)
|
||||
}
|
||||
value = expandTag(value, at: start, state: &state)
|
||||
}
|
||||
state.append(.tag(value), mark: start, endMark: state.reader.mark)
|
||||
}
|
||||
|
||||
func scanPlainScalar(_ state: inout State) {
|
||||
let start = state.reader.mark
|
||||
var value = ""
|
||||
while let character = state.reader.peek() {
|
||||
if shouldEndPlainScalar(character, state: state, accumulated: value) {
|
||||
break
|
||||
}
|
||||
value.append(character)
|
||||
state.reader.advance()
|
||||
}
|
||||
var trimmed = trimTrailingSpaces(value)
|
||||
let isMappingKey = appendMappingKeyIfNeeded(start: start, state: &state)
|
||||
if !isMappingKey {
|
||||
trimmed = scanPlainScalarContinuations(
|
||||
after: trimmed,
|
||||
firstLineIndent: start.column - 1,
|
||||
state: &state,
|
||||
)
|
||||
}
|
||||
state.append(.scalar(value: trimmed, style: .plain), mark: start, endMark: state.reader.mark)
|
||||
}
|
||||
}
|
||||
@ -1,112 +0,0 @@
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
struct State {
|
||||
var reader: PureYAML.Parsing.Reader
|
||||
var tokens: [PureYAML.Parsing.Token] = []
|
||||
var indentation: [Int] = [0]
|
||||
var isAtLineStart = true
|
||||
var hasDocumentContent = false
|
||||
var hasDocumentStartMarker = false
|
||||
var isDocumentClosed = false
|
||||
var hasYAMLDirective = false
|
||||
// The line of a directive awaiting its document. A directive must be followed by an explicit
|
||||
// `---`; this is set when one is scanned, cleared at a document start, and checked at a
|
||||
// document end and at stream end so a directive with no document is rejected.
|
||||
var pendingDirectiveLine: Int?
|
||||
var flowDepth = 0
|
||||
var flowContexts: [PureYAML.Parsing.ScannerFlowContext] = []
|
||||
var validImplicitIndentation: Set<Int> = []
|
||||
var pendingBlockScalarIndentation = false
|
||||
var pendingBlockScalarIndentationIndicator: Int?
|
||||
/// Whether the pending block scalar is the document's own root node (no enclosing collection),
|
||||
/// so its content may begin at column 0 (the document parent's indentation being -1).
|
||||
var pendingBlockScalarIsRootNode = false
|
||||
/// Whether the pending block scalar is an inline mapping value (`- key: |2`), whose parent
|
||||
/// mapping sits at an implicit indentation off the main stack, so an explicit indentation
|
||||
/// indicator is measured relative to that level rather than the stack's.
|
||||
var pendingBlockScalarInlineMappingValue = false
|
||||
/// The leading-whitespace characters of each blank line between a block scalar header and its
|
||||
/// first content line. Stored verbatim (not just counted) so once the content indentation is
|
||||
/// known, any surplus past it, notably a tab on an otherwise-blank line, survives as content.
|
||||
var pendingBlockScalarLeadingBlankWhitespace: [String] = []
|
||||
/// The widest indentation seen on a block scalar's leading blank lines (before its first
|
||||
/// content line). The spec forbids a leading empty line from being more indented than the
|
||||
/// content, so this is compared to the content indentation once that is known.
|
||||
var pendingBlockScalarMaxBlankIndent = 0
|
||||
/// The count of leading space characters (before any tab) from the most recent indentation
|
||||
/// scan. Indentation is measured in spaces, so this, not the tab-expanded width, is what the
|
||||
/// leading-blank comparison uses: a tab is not indentation.
|
||||
var lastLeadingSpaceCount = 0
|
||||
// The literal leading-whitespace characters from the most recent indentation scan. Block
|
||||
// scalar content keeps tabs beyond its (space-only) indentation verbatim, so the surplus is
|
||||
// reconstructed from these characters, not as spaces from the tab-expanded width.
|
||||
var lastLeadingWhitespace = ""
|
||||
var blockScalarIndentation: Int?
|
||||
var blockScalarContentPrefix = ""
|
||||
/// Whether a real token has been scanned on the current physical line. Reset when the scan
|
||||
/// loop consumes a line break and set as each token is scanned, so the scanner can tell a
|
||||
/// comment that opens a line (valid) from one glued to earlier content on it (invalid). A
|
||||
/// simple flag, not a token-line comparison, because a blank line inside a block scalar
|
||||
/// emits an empty scalar attributed to the next line, which would read as phantom content.
|
||||
var sawContentOnLine = false
|
||||
/// Like ``sawContentOnLine`` but not set by a node property (an anchor or tag). A `:` whose
|
||||
/// only preceding same-line content is a property (`&a :`, `!!str :`) is an empty implicit
|
||||
/// key carrying that property, so a null mapping key is synthesized for it exactly as for a
|
||||
/// bare `: value`. Reset at each line break.
|
||||
var sawNonPropertyContentOnLine = false
|
||||
// Whether a block-level mapping-value indicator (`:`) has been emitted on the current line.
|
||||
// Once one has, a later scalar followed by `:` is not a new implicit key: block style allows
|
||||
// only one mapping pair per line, so the document would be a forbidden inline nested mapping
|
||||
// (`a: b: c`). Reset at each line break. Flow `:` do not set it, since flow allows several
|
||||
// pairs on a line (`{a: b, c: d}`).
|
||||
var sawMappingValueOnLine = false
|
||||
/// Whether a flow collection has closed on the current line at the top level. A block
|
||||
/// sequence entry cannot follow a flow node on the same line (`{a: b}- x`), unlike a nested
|
||||
/// block entry (`- -`), which is not preceded by a flow close. Reset at each line break.
|
||||
var sawFlowCloseOnLine = false
|
||||
// Whether an explicit `?` mapping key is awaiting its `:` value indicator. A `:` that opens
|
||||
// a line is that key's value when one is pending, otherwise it is an implicit null key
|
||||
// (`: value`). Only the `?` indicator sets this; a synthesized implicit key does not.
|
||||
var pendingExplicitKey = false
|
||||
/// The tag handles in scope for the current document. A `%TAG` directive's handle applies
|
||||
/// only to the document it precedes (YAML 1.2), so handles are scoped per document, not
|
||||
/// accumulated across the stream: `pendingTagHandles` collects a document's directives and is
|
||||
/// merged onto the defaults at the document start, then both reset at the document boundary.
|
||||
static let defaultTagHandles = [
|
||||
"!": "!",
|
||||
"!!": "tag:yaml.org,2002:",
|
||||
]
|
||||
var tagHandles = Self.defaultTagHandles
|
||||
var pendingTagHandles: [String: String] = [:]
|
||||
/// Tag shorthands using a handle never declared in their document; collected, not rejected.
|
||||
var tagHandleIssues: [PureYAML.Parsing.TagHandleIssue] = []
|
||||
|
||||
mutating func append(_ kind: PureYAML.Parsing.TokenKind) {
|
||||
let mark = reader.mark
|
||||
tokens.append(PureYAML.Parsing.Token(kind: kind, mark: mark, endMark: mark))
|
||||
markDocumentContentIfNeeded(kind)
|
||||
}
|
||||
|
||||
mutating func append(
|
||||
_ kind: PureYAML.Parsing.TokenKind,
|
||||
mark: PureYAML.Parsing.Mark,
|
||||
endMark: PureYAML.Parsing.Mark,
|
||||
) {
|
||||
tokens.append(PureYAML.Parsing.Token(kind: kind, mark: mark, endMark: endMark))
|
||||
markDocumentContentIfNeeded(kind)
|
||||
}
|
||||
|
||||
mutating func markDocumentContentIfNeeded(_ kind: PureYAML.Parsing.TokenKind) {
|
||||
switch kind {
|
||||
case .comment, .dedent, .documentEnd, .documentStart, .indent, .streamEnd, .streamStart:
|
||||
return
|
||||
default:
|
||||
hasDocumentContent = true
|
||||
}
|
||||
}
|
||||
|
||||
var isExpectingFlowMappingKey: Bool {
|
||||
flowContexts.last?.kind == .mapping
|
||||
&& flowContexts.last?.isExpectingKey == true
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,45 +0,0 @@
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
func expandTag(
|
||||
_ tag: String,
|
||||
at start: PureYAML.Parsing.Mark,
|
||||
state: inout State,
|
||||
) -> String {
|
||||
// A named handle has the form `!name!suffix` with a non-empty name between two `!` (distinct
|
||||
// from the secondary handle `!!suffix`, whose name is empty, and a local tag `!suffix`, which
|
||||
// has no second `!`). Such a handle must be declared by a `%TAG` directive in this document;
|
||||
// an undeclared one is recorded as a diagnostic and the tag is left verbatim, so the parse
|
||||
// stays permissive while the issue can still be surfaced.
|
||||
if let named = namedTagHandle(in: tag) {
|
||||
if let prefix = state.tagHandles[named.handle] {
|
||||
return prefix + named.suffix
|
||||
}
|
||||
state.tagHandleIssues.append(.init(line: start.line, column: start.column, handle: named.handle))
|
||||
return tag
|
||||
}
|
||||
let handles = state.tagHandles.keys.sorted { first, second in
|
||||
first.count > second.count
|
||||
}
|
||||
for handle in handles where tag.hasPrefix(handle) {
|
||||
let suffix = tag.dropFirst(handle.count)
|
||||
return (state.tagHandles[handle] ?? handle) + suffix
|
||||
}
|
||||
return tag
|
||||
}
|
||||
|
||||
/// Splits a tag shorthand into its named handle and suffix when it has the `!name!suffix` form
|
||||
/// (a non-empty name delimited by two `!`). Returns nil for a local tag (`!suffix`), the
|
||||
/// secondary handle (`!!suffix`), or a verbatim tag.
|
||||
func namedTagHandle(in tag: String) -> (handle: String, suffix: String)? {
|
||||
guard tag.hasPrefix("!") else {
|
||||
return nil
|
||||
}
|
||||
let afterFirst = tag.index(after: tag.startIndex)
|
||||
guard let secondBang = tag[afterFirst...].firstIndex(of: "!"), secondBang > afterFirst else {
|
||||
return nil
|
||||
}
|
||||
return (
|
||||
handle: String(tag[...secondBang]),
|
||||
suffix: String(tag[tag.index(after: secondBang)...]),
|
||||
)
|
||||
}
|
||||
}
|
||||
@ -1,316 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
struct Scanner {
|
||||
init() {}
|
||||
|
||||
func scan(_ yaml: String) throws -> [Token] {
|
||||
let outcome = collect(yaml)
|
||||
if let error = outcome.error {
|
||||
throw error
|
||||
}
|
||||
return outcome.state.tokens
|
||||
}
|
||||
|
||||
/// The tag-handle issues found while scanning, for the diagnostic layer. A scan-aborting error
|
||||
/// still leaves the issues collected before it on the returned state, so they are reported; a
|
||||
/// clean scan returns the complete set.
|
||||
func tagHandleIssues(_ yaml: String) -> [PureYAML.Parsing.TagHandleIssue] {
|
||||
collect(yaml).state.tagHandleIssues
|
||||
}
|
||||
|
||||
/// Runs the scan and returns the resulting state with the error that aborted it, if any.
|
||||
/// Returning the partial state keeps the diagnostics gathered before an abort, which a `try?`
|
||||
/// over a throwing run would discard.
|
||||
private func collect(_ yaml: String) -> (state: State, error: (any Error)?) {
|
||||
var state = State(reader: Reader(yaml))
|
||||
do {
|
||||
try scanAll(&state)
|
||||
return (state, nil)
|
||||
} catch {
|
||||
return (state, error)
|
||||
}
|
||||
}
|
||||
|
||||
private func scanAll(_ state: inout State) throws {
|
||||
state.append(.streamStart)
|
||||
|
||||
while !state.reader.isAtEnd {
|
||||
if isLineBreak(state.reader.peek()) {
|
||||
state.reader.advance()
|
||||
captureBlockScalarBlankLine(&state)
|
||||
state.isAtLineStart = true
|
||||
state.sawContentOnLine = false
|
||||
state.sawNonPropertyContentOnLine = false
|
||||
state.sawMappingValueOnLine = false
|
||||
state.sawFlowCloseOnLine = false
|
||||
continue
|
||||
}
|
||||
|
||||
if state.isAtLineStart {
|
||||
try scanIndentation(&state)
|
||||
if state.reader.isAtEnd {
|
||||
break
|
||||
}
|
||||
if isLineBreak(state.reader.peek()) {
|
||||
continue
|
||||
}
|
||||
if try scanDirectiveOrDocumentMarker(&state) {
|
||||
continue
|
||||
}
|
||||
state.isAtLineStart = false
|
||||
}
|
||||
|
||||
try scanToken(&state)
|
||||
}
|
||||
|
||||
closeIndentation(&state)
|
||||
try enforceDirectiveHasDocument(state)
|
||||
state.append(.streamEnd)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
extension PureYAML.Parsing.Scanner {
|
||||
func scanToken(_ state: inout State) throws {
|
||||
let columnBeforeSeparation = state.reader.mark.column
|
||||
skipSeparation(&state)
|
||||
let consumedSeparation = state.reader.mark.column > columnBeforeSeparation
|
||||
guard let character = state.reader.peek(), !isLineBreak(character) else {
|
||||
return
|
||||
}
|
||||
|
||||
if state.blockScalarIndentation != nil {
|
||||
scanBlockScalarContent(&state)
|
||||
return
|
||||
}
|
||||
// A `#` begins a comment only at the start of a line's content or after whitespace. Glued to
|
||||
// earlier same-line content (e.g. `"x"#c`, `]#c`, `>#c`) it is not a plain-scalar character
|
||||
// either, so the document is malformed; a plain scalar's own `#` is consumed before reaching
|
||||
// here, so this rejects only the genuinely invalid case.
|
||||
if character == "#", !consumedSeparation, state.sawContentOnLine {
|
||||
throw PureYAML.Parsing.ParseError.commentNotPrecededByWhitespace(
|
||||
line: state.reader.mark.line,
|
||||
column: state.reader.mark.column,
|
||||
)
|
||||
}
|
||||
let hadContentOnLine = state.sawContentOnLine
|
||||
let hadNonPropertyContentOnLine = state.sawNonPropertyContentOnLine
|
||||
state.sawContentOnLine = true
|
||||
// A leading `&` or `!` begins a node property (anchor or tag), which is not itself a key or
|
||||
// value; every other token character is real content. A property-only line followed by `:`
|
||||
// is thus an empty implicit key (`&a :`), synthesized below like a bare `: value`.
|
||||
if character != "&", character != "!" {
|
||||
state.sawNonPropertyContentOnLine = true
|
||||
}
|
||||
// An implicit-key mapping value (`:` preceded by its key on the same line) means the line
|
||||
// now holds a pair, so a later scalar followed by `:` cannot start another: that would be a
|
||||
// forbidden inline nested block mapping (`a: b: c`). A `:` that opens the line is an explicit-
|
||||
// or null-key value indicator, whose value may legitimately be a compact mapping, so it does
|
||||
// not set the flag.
|
||||
if character == ":", hadContentOnLine, state.flowDepth == 0, isAtMappingValueIndicator(state: state) {
|
||||
state.sawMappingValueOnLine = true
|
||||
}
|
||||
synthesizeNullMappingKeyIfNeeded(character, hadNonPropertyContentOnLine: hadNonPropertyContentOnLine, state: &state)
|
||||
if try scanLineIndicatorToken(character, state: &state) {
|
||||
return
|
||||
}
|
||||
if scanFlowToken(character, state: &state) {
|
||||
return
|
||||
}
|
||||
if try scanQuotedOrBlockScalarToken(character, state: &state) {
|
||||
return
|
||||
}
|
||||
if try scanNodePropertyToken(character, state: &state) {
|
||||
return
|
||||
}
|
||||
// In a flow collection, a `-` that begins a plain scalar must be followed by a non-indicator
|
||||
// character. Immediately before a flow indicator (`[-]`, `[-, -]`) it is not a valid scalar.
|
||||
if state.flowDepth > 0, character == "-", let next = state.reader.peek(offset: 1), isFlowDelimiter(next) {
|
||||
throw PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: "a flow value",
|
||||
actual: "-",
|
||||
line: state.reader.mark.line,
|
||||
column: state.reader.mark.column,
|
||||
)
|
||||
}
|
||||
scanPlainScalar(&state)
|
||||
}
|
||||
|
||||
func scanLineIndicatorToken(
|
||||
_ character: Character,
|
||||
state: inout State,
|
||||
) throws -> Bool {
|
||||
switch character {
|
||||
case "#":
|
||||
scanComment(&state)
|
||||
return true
|
||||
case "-" where isBlockEntry(state.reader):
|
||||
// A block sequence cannot be an implicit key's value on the same line as its `:`
|
||||
// (`key: - a`); the value must begin on a following line. As with an inline nested
|
||||
// mapping, an explicit- or null-key `:` opens its line and does not set the flag, so a
|
||||
// compact sequence value after it stays valid.
|
||||
if state.sawMappingValueOnLine, state.flowDepth == 0 {
|
||||
throw PureYAML.Parsing.ParseError.blockSequenceAsInlineMappingValue(line: state.reader.mark.line)
|
||||
}
|
||||
// A block sequence entry cannot follow a flow collection that closed earlier on the
|
||||
// line (`{a: b}- x`); the entry would begin a sequence inline after a complete node.
|
||||
if state.sawFlowCloseOnLine {
|
||||
throw PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: "a line break or flow indicator",
|
||||
actual: "-",
|
||||
line: state.reader.mark.line,
|
||||
column: state.reader.mark.column,
|
||||
)
|
||||
}
|
||||
scanFixedToken(.blockEntry, state: &state)
|
||||
return true
|
||||
case "?" where isIndicatorBoundary(state.reader.peek(offset: 1)):
|
||||
scanFixedToken(.mappingKey, state: &state)
|
||||
return true
|
||||
case ":" where isAtMappingValueIndicator(state: state):
|
||||
scanFixedToken(.mappingValue, state: &state)
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func scanFlowToken(
|
||||
_ character: Character,
|
||||
state: inout State,
|
||||
) -> Bool {
|
||||
switch character {
|
||||
case "[":
|
||||
appendFlowImplicitKeyMarkerIfNeeded(state: &state)
|
||||
state.flowDepth += 1
|
||||
state.flowContexts.append(.init(kind: .sequence, isExpectingKey: false))
|
||||
scanFixedToken(.flowSequenceStart, state: &state)
|
||||
return true
|
||||
case "]":
|
||||
state.flowDepth = max(0, state.flowDepth - 1)
|
||||
scanFixedToken(.flowSequenceEnd, state: &state)
|
||||
_ = state.flowContexts.popLast()
|
||||
if state.flowDepth == 0 {
|
||||
state.sawFlowCloseOnLine = true
|
||||
}
|
||||
return true
|
||||
case "{":
|
||||
appendFlowImplicitKeyMarkerIfNeeded(state: &state)
|
||||
state.flowDepth += 1
|
||||
state.flowContexts.append(.init(kind: .mapping, isExpectingKey: true))
|
||||
scanFixedToken(.flowMappingStart, state: &state)
|
||||
return true
|
||||
case "}":
|
||||
state.flowDepth = max(0, state.flowDepth - 1)
|
||||
scanFixedToken(.flowMappingEnd, state: &state)
|
||||
_ = state.flowContexts.popLast()
|
||||
if state.flowDepth == 0 {
|
||||
state.sawFlowCloseOnLine = true
|
||||
}
|
||||
return true
|
||||
case ",":
|
||||
scanFixedToken(.flowEntry, state: &state)
|
||||
if state.flowContexts.last?.kind == .mapping {
|
||||
state.flowContexts[state.flowContexts.count - 1].isExpectingKey = true
|
||||
}
|
||||
if !state.flowContexts.isEmpty {
|
||||
state.flowContexts[state.flowContexts.count - 1].hasExplicitKey = false
|
||||
}
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func scanNodePropertyToken(
|
||||
_ character: Character,
|
||||
state: inout State,
|
||||
) throws -> Bool {
|
||||
switch character {
|
||||
case "&":
|
||||
try scanNamedToken(
|
||||
kind: PureYAML.Parsing.TokenKind.anchor,
|
||||
missingNameError: PureYAML.Parsing.ParseError.expectedAnchorName,
|
||||
state: &state,
|
||||
)
|
||||
return true
|
||||
case "*":
|
||||
try scanNamedToken(
|
||||
kind: PureYAML.Parsing.TokenKind.alias,
|
||||
missingNameError: PureYAML.Parsing.ParseError.expectedAliasName,
|
||||
canBeImplicitKey: true,
|
||||
state: &state,
|
||||
)
|
||||
return true
|
||||
case "!":
|
||||
try scanTag(&state)
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func scanQuotedOrBlockScalarToken(
|
||||
_ character: Character,
|
||||
state: inout State,
|
||||
) throws -> Bool {
|
||||
switch character {
|
||||
case "\"":
|
||||
try scanQuotedScalar(.doubleQuoted, state: &state)
|
||||
return true
|
||||
case "'":
|
||||
try scanQuotedScalar(.singleQuoted, state: &state)
|
||||
return true
|
||||
case "|":
|
||||
scanBlockScalarHeader(.literal, state: &state)
|
||||
return true
|
||||
case ">":
|
||||
scanBlockScalarHeader(.folded, state: &state)
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func skipSeparation(_ state: inout State) {
|
||||
// Mid-line separation may be spaces or tabs: a tab is separation white space in YAML and is
|
||||
// forbidden only as indentation, which the line-start indentation scan handles before this.
|
||||
while state.reader.peek() == " " || state.reader.peek() == "\t" {
|
||||
state.reader.advance()
|
||||
}
|
||||
}
|
||||
|
||||
func scanFixedToken(
|
||||
_ kind: PureYAML.Parsing.TokenKind,
|
||||
state: inout State,
|
||||
) {
|
||||
let start = state.reader.mark
|
||||
state.reader.advance()
|
||||
state.append(kind, mark: start, endMark: state.reader.mark)
|
||||
if case .blockEntry = kind {
|
||||
recordBlockEntryImplicitIndentation(start: start, state: &state)
|
||||
// A block entry opens a fresh node position, so a property-only key after it on the same
|
||||
// line (`- !!str : v`) is still an empty implicit key: the `-` is not content of the key.
|
||||
state.sawNonPropertyContentOnLine = false
|
||||
}
|
||||
if case .mappingValue = kind, state.flowContexts.last?.kind == .mapping {
|
||||
state.flowContexts[state.flowContexts.count - 1].isExpectingKey = false
|
||||
}
|
||||
// Only the explicit `?` indicator emits a mapping key through this path (an implicit key is
|
||||
// synthesized directly), so track a block-level one as pending until its `:` value indicator
|
||||
// is scanned. Flow `:`/`?` (inside a complex key, say) do not affect a pending block key.
|
||||
if state.flowDepth == 0, case .mappingKey = kind {
|
||||
state.pendingExplicitKey = true
|
||||
}
|
||||
if state.flowDepth == 0, case .mappingValue = kind {
|
||||
state.pendingExplicitKey = false
|
||||
}
|
||||
// In flow the same role is tracked on the current flow context, so it is per-level and a
|
||||
// nested collection's `:` does not affect an outer explicit key (`{? {a: b} : c}`).
|
||||
if state.flowDepth > 0, !state.flowContexts.isEmpty, case .mappingKey = kind {
|
||||
state.flowContexts[state.flowContexts.count - 1].hasExplicitKey = true
|
||||
}
|
||||
if state.flowDepth > 0, !state.flowContexts.isEmpty, case .mappingValue = kind {
|
||||
state.flowContexts[state.flowContexts.count - 1].hasExplicitKey = false
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,6 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
enum ScannerFlowContextKind {
|
||||
case sequence
|
||||
case mapping
|
||||
}
|
||||
}
|
||||
@ -1,11 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
/// A tag shorthand (`!handle!suffix`) whose named handle was never declared by a `%TAG`
|
||||
/// directive in the document where it is used. The scanner records these rather than rejecting,
|
||||
/// so a permissive parse still produces a value while the diagnostic layer can surface the issue.
|
||||
struct TagHandleIssue: Equatable {
|
||||
var line: Int
|
||||
var column: Int
|
||||
/// The undeclared handle, including its surrounding `!`, e.g. `!e!`.
|
||||
var handle: String
|
||||
}
|
||||
}
|
||||
@ -1,11 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
struct Token: Equatable, CustomStringConvertible {
|
||||
var kind: TokenKind
|
||||
var mark: Mark
|
||||
var endMark: Mark
|
||||
|
||||
var description: String {
|
||||
"\(kind) @\(mark)..\(endMark)"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,208 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
private final class TokenScannerStateBox {
|
||||
var state: Scanner.State
|
||||
|
||||
init(state: Scanner.State) {
|
||||
self.state = state
|
||||
}
|
||||
}
|
||||
|
||||
struct TokenCursor {
|
||||
private var tokens: [Token]
|
||||
private(set) var index: Int
|
||||
private var previousToken: Token?
|
||||
private var scanner: Scanner?
|
||||
private var scannerBox: TokenScannerStateBox?
|
||||
|
||||
init(tokens: [Token]) {
|
||||
self.tokens = tokens.filter { !$0.kind.isComment }
|
||||
index = 0
|
||||
previousToken = nil
|
||||
scanner = nil
|
||||
scannerBox = nil
|
||||
}
|
||||
|
||||
init(yaml: String, scanner: Scanner = .init()) {
|
||||
var state = Scanner.State(reader: Reader(yaml))
|
||||
state.append(.streamStart)
|
||||
tokens = state.tokens.filter { !$0.kind.isComment }
|
||||
state.tokens.removeAll(keepingCapacity: true)
|
||||
index = 0
|
||||
previousToken = nil
|
||||
self.scanner = scanner
|
||||
scannerBox = TokenScannerStateBox(state: state)
|
||||
}
|
||||
|
||||
var current: Token? {
|
||||
guard tokens.indices.contains(index) else {
|
||||
return nil
|
||||
}
|
||||
return tokens[index]
|
||||
}
|
||||
|
||||
var previous: Token? {
|
||||
previousToken
|
||||
}
|
||||
|
||||
mutating func peek(offset: Int = 1) throws -> Token? {
|
||||
try ensureAvailable(offset: offset)
|
||||
let targetIndex = index + offset
|
||||
guard tokens.indices.contains(targetIndex) else {
|
||||
return nil
|
||||
}
|
||||
return tokens[targetIndex]
|
||||
}
|
||||
|
||||
mutating func advance() throws -> Token? {
|
||||
try ensureAvailable(offset: 0)
|
||||
guard tokens.indices.contains(index) else {
|
||||
return nil
|
||||
}
|
||||
let token = tokens[index]
|
||||
previousToken = token
|
||||
index += 1
|
||||
compactConsumedTokens()
|
||||
try ensureAvailable(offset: 0)
|
||||
return token
|
||||
}
|
||||
|
||||
private mutating func ensureAvailable(offset: Int) throws {
|
||||
while tokens.count <= index + offset, scannerBox != nil {
|
||||
try scanMoreTokens()
|
||||
}
|
||||
}
|
||||
|
||||
private mutating func scanMoreTokens() throws {
|
||||
guard let scanner, let scannerBox else {
|
||||
return
|
||||
}
|
||||
|
||||
while scannerBox.state.tokens.isEmpty {
|
||||
guard !scannerBox.state.reader.isAtEnd else {
|
||||
scanner.closeIndentation(&scannerBox.state)
|
||||
try scanner.enforceDirectiveHasDocument(scannerBox.state)
|
||||
scannerBox.state.append(.streamEnd)
|
||||
drainScannedTokens(from: &scannerBox.state)
|
||||
self.scannerBox = nil
|
||||
return
|
||||
}
|
||||
|
||||
if scanner.isLineBreak(scannerBox.state.reader.peek()) {
|
||||
scannerBox.state.reader.advance()
|
||||
scanner.captureBlockScalarBlankLine(&scannerBox.state)
|
||||
scannerBox.state.isAtLineStart = true
|
||||
scannerBox.state.sawContentOnLine = false
|
||||
scannerBox.state.sawNonPropertyContentOnLine = false
|
||||
scannerBox.state.sawMappingValueOnLine = false
|
||||
scannerBox.state.sawFlowCloseOnLine = false
|
||||
continue
|
||||
}
|
||||
|
||||
if scannerBox.state.isAtLineStart {
|
||||
try scanner.scanIndentation(&scannerBox.state)
|
||||
if scannerBox.state.reader.isAtEnd {
|
||||
continue
|
||||
}
|
||||
if scanner.isLineBreak(scannerBox.state.reader.peek()) {
|
||||
continue
|
||||
}
|
||||
if try scanner.scanDirectiveOrDocumentMarker(&scannerBox.state) {
|
||||
if !scannerBox.state.tokens.isEmpty {
|
||||
break
|
||||
}
|
||||
continue
|
||||
}
|
||||
scannerBox.state.isAtLineStart = false
|
||||
}
|
||||
|
||||
try scanner.scanToken(&scannerBox.state)
|
||||
}
|
||||
|
||||
drainScannedTokens(from: &scannerBox.state)
|
||||
}
|
||||
|
||||
private mutating func drainScannedTokens(from state: inout Scanner.State) {
|
||||
tokens.append(contentsOf: state.tokens.filter { !$0.kind.isComment })
|
||||
state.tokens.removeAll(keepingCapacity: true)
|
||||
}
|
||||
|
||||
private mutating func compactConsumedTokens() {
|
||||
guard index > 4096 else {
|
||||
return
|
||||
}
|
||||
tokens.removeFirst(index)
|
||||
index = 0
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
extension PureYAML.Parsing.TokenKind {
|
||||
var isBlockEntry: Bool {
|
||||
if case .blockEntry = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isComment: Bool {
|
||||
if case .comment = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isDedent: Bool {
|
||||
if case .dedent = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isDocumentEnd: Bool {
|
||||
if case .documentEnd = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isDocumentStart: Bool {
|
||||
if case .documentStart = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isFlowTerminator: Bool {
|
||||
switch self {
|
||||
case .flowEntry, .flowMappingEnd, .flowSequenceEnd:
|
||||
true
|
||||
default:
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
var isIndent: Bool {
|
||||
if case .indent = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isMappingKey: Bool {
|
||||
if case .mappingKey = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isStreamEnd: Bool {
|
||||
if case .streamEnd = self {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
var isTerminator: Bool {
|
||||
isDedent || isDocumentEnd || isDocumentStart || isFlowTerminator || isStreamEnd
|
||||
}
|
||||
}
|
||||
@ -1,367 +0,0 @@
|
||||
extension PureYAML.Parsing.TokenEventParser {
|
||||
mutating func parseBlockSequence(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
allowMappingKeyTerminator: Bool = false,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
let start = try expect("block sequence entry") { $0.isBlockEntry }
|
||||
try enforceBlockSequenceNotInlineAfterProperty(properties, firstEntry: start)
|
||||
// A compact nested sequence (`- - x`) carries no indent/dedent tokens, only block entries at
|
||||
// their columns, so the sibling-entry loop must stay at this sequence's own column: a later
|
||||
// entry at a smaller column belongs to an enclosing sequence and ends this one.
|
||||
let sequenceColumn = start.mark.column
|
||||
let startMark = properties.mark ?? start.mark
|
||||
var endMark = start.endMark
|
||||
try emit(.sequenceStart(
|
||||
anchor: properties.anchor,
|
||||
tag: properties.tag,
|
||||
style: .block,
|
||||
mark: startMark,
|
||||
))
|
||||
|
||||
try parseBlockSequenceItem(
|
||||
after: start,
|
||||
endMark: &endMark,
|
||||
allowMappingKeyTerminator: allowMappingKeyTerminator,
|
||||
)
|
||||
try consumeSequenceItemBoundaryDedent(sequenceIndentWidth: start.mark.column - 1)
|
||||
try consumeSequenceItemBoundaryIndent(sequenceIndentWidth: start.mark.column - 1)
|
||||
while cursor.current?.kind.isBlockEntry == true, cursor.current?.mark.column == sequenceColumn {
|
||||
let entry = try expect("block sequence entry") { $0.isBlockEntry }
|
||||
try parseBlockSequenceItem(
|
||||
after: entry,
|
||||
endMark: &endMark,
|
||||
allowMappingKeyTerminator: allowMappingKeyTerminator,
|
||||
)
|
||||
try consumeSequenceItemBoundaryDedent(sequenceIndentWidth: entry.mark.column - 1)
|
||||
try consumeSequenceItemBoundaryIndent(sequenceIndentWidth: entry.mark.column - 1)
|
||||
}
|
||||
if let current = cursor.current, current.kind.isMappingKey, !allowMappingKeyTerminator {
|
||||
throw PureYAML.Parsing.ParseError.mixedCollectionStyles(line: current.mark.line)
|
||||
}
|
||||
|
||||
try emit(.sequenceEnd(mark: endMark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .sequence, endMark: endMark)
|
||||
}
|
||||
|
||||
/// A block collection must begin on the line after its node properties, so an anchor or tag on
|
||||
/// the same line as the sequence's first entry (`&a - x`) is malformed.
|
||||
func enforceBlockSequenceNotInlineAfterProperty(
|
||||
_ properties: PureYAML.Parsing.NodeProperties,
|
||||
firstEntry: PureYAML.Parsing.Token,
|
||||
) throws {
|
||||
guard properties.anchor != nil || properties.tag != nil,
|
||||
let propertyMark = properties.mark, propertyMark.line == firstEntry.mark.line
|
||||
else {
|
||||
return
|
||||
}
|
||||
throw PureYAML.Parsing.ParseError.inlineBlockSequenceAfterProperty(
|
||||
line: propertyMark.line, column: propertyMark.column,
|
||||
)
|
||||
}
|
||||
|
||||
mutating func consumeSequenceItemBoundaryDedent(sequenceIndentWidth: Int) throws {
|
||||
guard dedentWidth(cursor.current) == sequenceIndentWidth,
|
||||
try cursor.peek()?.kind.isBlockEntry == true
|
||||
else {
|
||||
return
|
||||
}
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
}
|
||||
|
||||
mutating func consumeSequenceItemBoundaryIndent(sequenceIndentWidth: Int) throws {
|
||||
guard indentWidth(cursor.current) == sequenceIndentWidth,
|
||||
try cursor.peek()?.kind.isBlockEntry == true
|
||||
else {
|
||||
return
|
||||
}
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
}
|
||||
|
||||
mutating func parseBlockSequenceItem(
|
||||
after entry: PureYAML.Parsing.Token,
|
||||
endMark: inout PureYAML.Parsing.Mark,
|
||||
allowMappingKeyTerminator: Bool,
|
||||
) throws {
|
||||
if hasNodeOnSameLine(after: entry) {
|
||||
let properties = try consumeProperties()
|
||||
if cursor.current?.kind.isIndent == true {
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
endMark = try parseNode(properties: properties).endMark
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
return
|
||||
}
|
||||
if cursor.current?.kind.isMappingKey == true {
|
||||
endMark = try parseBlockMapping(
|
||||
properties: properties,
|
||||
includeIndentedContinuation: true,
|
||||
minimumColumn: entry.mark.column + 1,
|
||||
).endMark
|
||||
} else if cursor.current?.kind.isBlockEntry == true {
|
||||
endMark = try parseBlockSequence(
|
||||
properties: properties,
|
||||
allowMappingKeyTerminator: allowMappingKeyTerminator,
|
||||
).endMark
|
||||
} else {
|
||||
endMark = try parseNode(properties: properties).endMark
|
||||
}
|
||||
} else if cursor.current?.kind.isIndent == true {
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
endMark = try parseNode().endMark
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
} else {
|
||||
let mark = entry.endMark
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: mark, endMark: mark))
|
||||
endMark = mark
|
||||
}
|
||||
}
|
||||
|
||||
mutating func parseBlockMapping(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
includeIndentedContinuation: Bool,
|
||||
minimumColumn: Int? = nil,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
let startMark = properties.mark ?? cursor.current?.mark ?? .start
|
||||
let tokenColumn = cursor.current?.mark.column ?? startMark.column
|
||||
let contentColumn = min(startMark.column, tokenColumn)
|
||||
let boundaryColumn = minimumColumn ?? contentColumn
|
||||
var endMark = startMark
|
||||
// A property written inline before the first key (`&a key: v`) binds to that key, not to the
|
||||
// mapping: the spec requires a block collection's own properties to be followed by a line
|
||||
// break (`s-l+block-collection`), so a property on the key's line is the key's. This matters
|
||||
// for anchors, which must register with the key (early enough for an alias inside a value to
|
||||
// resolve) rather than with the mapping, which registers only at its end. A property on its
|
||||
// own line above the first key still binds to the mapping.
|
||||
let firstKeyProperties: PureYAML.Parsing.NodeProperties
|
||||
let mappingProperties: PureYAML.Parsing.NodeProperties
|
||||
if properties != .none, properties.mark?.line == cursor.current?.mark.line {
|
||||
firstKeyProperties = properties
|
||||
mappingProperties = .none
|
||||
} else {
|
||||
firstKeyProperties = .none
|
||||
mappingProperties = properties
|
||||
}
|
||||
try emit(.mappingStart(
|
||||
anchor: mappingProperties.anchor,
|
||||
tag: mappingProperties.tag,
|
||||
style: .block,
|
||||
mark: startMark,
|
||||
))
|
||||
|
||||
try parseBlockMappingPairs(
|
||||
boundaryColumn: boundaryColumn,
|
||||
includeIndentedContinuation: includeIndentedContinuation,
|
||||
firstKeyProperties: firstKeyProperties,
|
||||
endMark: &endMark,
|
||||
)
|
||||
if let current = cursor.current, !canEndBlockMapping(at: current, minimumColumn: boundaryColumn) {
|
||||
throw PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: "mapping key",
|
||||
actual: current.kind.description,
|
||||
line: current.mark.line,
|
||||
column: current.mark.column,
|
||||
)
|
||||
}
|
||||
try emit(.mappingEnd(mark: endMark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .mapping, endMark: endMark)
|
||||
}
|
||||
|
||||
mutating func parseBlockMappingPairs(
|
||||
boundaryColumn: Int,
|
||||
includeIndentedContinuation: Bool,
|
||||
firstKeyProperties: PureYAML.Parsing.NodeProperties = .none,
|
||||
endMark: inout PureYAML.Parsing.Mark,
|
||||
) throws {
|
||||
var pendingKeyProperties = firstKeyProperties
|
||||
while canContinueBlockMapping(minimumColumn: boundaryColumn) {
|
||||
endMark = try parseMappingPairAndBoundaries(
|
||||
boundaryColumn: boundaryColumn,
|
||||
includeIndentedContinuation: includeIndentedContinuation,
|
||||
firstKeyProperties: pendingKeyProperties,
|
||||
).endMark
|
||||
pendingKeyProperties = .none
|
||||
}
|
||||
while includeIndentedContinuation, cursor.current?.kind.isIndent == true {
|
||||
endMark = try parseIndentedBlockMappingPairs(boundaryColumn: boundaryColumn).endMark
|
||||
}
|
||||
}
|
||||
|
||||
mutating func parseIndentedBlockMappingPairs(
|
||||
boundaryColumn: Int,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
guard cursor.current?.kind.isMappingKey == true else {
|
||||
throw unexpectedToken(expected: "mapping key")
|
||||
}
|
||||
var endMark = cursor.current?.mark ?? .start
|
||||
while canContinueBlockMapping(minimumColumn: boundaryColumn) {
|
||||
endMark = try parseMappingPairAndBoundaries(
|
||||
boundaryColumn: boundaryColumn,
|
||||
includeIndentedContinuation: true,
|
||||
).endMark
|
||||
}
|
||||
if !canEndBlockMapping(at: cursor.current, minimumColumn: boundaryColumn) {
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
}
|
||||
return PureYAML.Parsing.NodeResult(kind: .mapping, endMark: endMark)
|
||||
}
|
||||
|
||||
mutating func parseMappingPairAndBoundaries(
|
||||
boundaryColumn: Int,
|
||||
includeIndentedContinuation: Bool,
|
||||
firstKeyProperties: PureYAML.Parsing.NodeProperties = .none,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
let result = try parseMappingPair(firstKeyProperties: firstKeyProperties)
|
||||
try consumeMappingSiblingBoundaryDedents(minimumColumn: boundaryColumn)
|
||||
try consumeMappingSiblingBoundaryReindent(minimumColumn: boundaryColumn)
|
||||
if includeIndentedContinuation {
|
||||
try consumeDedents(atOrAbove: boundaryColumn)
|
||||
try consumeMappingSiblingBoundaryDedents(minimumColumn: boundaryColumn)
|
||||
try consumeMappingSiblingBoundaryReindent(minimumColumn: boundaryColumn)
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
mutating func parseBlockScalar(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
let header = try expect("block scalar header") { kind in
|
||||
if case .blockScalarHeader = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
var lines: [String] = []
|
||||
var endMark = header.endMark
|
||||
if cursor.current?.kind.isIndent == true {
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
while cursor.current != nil, cursor.current?.kind.isDedent != true {
|
||||
guard case let .scalar(value, style) = cursor.current?.kind else {
|
||||
throw unexpectedToken(expected: "block scalar content")
|
||||
}
|
||||
let token = try cursor.advance() ?? header
|
||||
try lines.append(decodeScalar(value, style: style, line: token.mark.line))
|
||||
endMark = token.endMark
|
||||
}
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
}
|
||||
|
||||
let value = blockScalarValue(lines: lines, header: header.kind)
|
||||
let mark = properties.mark ?? header.mark
|
||||
try emit(.scalar(
|
||||
value: value,
|
||||
anchor: properties.anchor,
|
||||
tag: properties.tag,
|
||||
style: scalarStyle(for: header.kind),
|
||||
mark: mark,
|
||||
endMark: endMark,
|
||||
))
|
||||
return PureYAML.Parsing.NodeResult(kind: .scalar, endMark: endMark)
|
||||
}
|
||||
|
||||
func scalarStyle(for kind: PureYAML.Parsing.TokenKind) -> PureYAML.Parsing.ScalarStyle {
|
||||
switch kind {
|
||||
case .blockScalarHeader(.folded, _):
|
||||
.folded
|
||||
case .blockScalarHeader(.literal, _):
|
||||
.literal
|
||||
default:
|
||||
.plain
|
||||
}
|
||||
}
|
||||
|
||||
func blockScalarValue(
|
||||
lines: [String],
|
||||
header: PureYAML.Parsing.TokenKind,
|
||||
) -> String {
|
||||
let (style, chomping): (PureYAML.Parsing.BlockScalarStyle, PureYAML.Parsing.BlockScalarChomping)
|
||||
switch header {
|
||||
case let .blockScalarHeader(headerStyle, headerChomping):
|
||||
style = headerStyle
|
||||
chomping = headerChomping
|
||||
default:
|
||||
style = .literal
|
||||
chomping = .clip
|
||||
}
|
||||
|
||||
// Each content line carries its own line break. The folded style folds a
|
||||
// single break between two non-empty lines into a space, while blank
|
||||
// lines stay as line breaks. Chomping then trims the trailing breaks.
|
||||
// For content with no blank lines (the only shape the scanner captures
|
||||
// today) this is identical to the previous join-and-append behavior.
|
||||
let body: String = switch style {
|
||||
case .literal:
|
||||
lines.map { $0 + "\n" }.joined()
|
||||
case .folded:
|
||||
foldedBlockBody(lines)
|
||||
}
|
||||
return chompedBlockScalar(body, chomping: chomping)
|
||||
}
|
||||
|
||||
func foldedBlockBody(_ lines: [String]) -> String {
|
||||
guard let firstContent = lines.firstIndex(where: { !$0.isEmpty }) else {
|
||||
// The block is entirely blank lines: one break each, no folding.
|
||||
return String(repeating: "\n", count: lines.count)
|
||||
}
|
||||
// Leading blank lines each contribute a line break before the content.
|
||||
var result = String(repeating: "\n", count: firstContent) + lines[firstContent]
|
||||
var index = firstContent + 1
|
||||
while index < lines.count {
|
||||
if lines[index].isEmpty {
|
||||
// Between two non-more-indented lines a blank run of N folds to N line breaks: the
|
||||
// first break is the absorbed fold, the rest are literal. Next to a more-indented line
|
||||
// nothing folds, so the absorbed break survives too and the run yields N+1 breaks.
|
||||
let beforeMoreIndented = isMoreIndentedBlockLine(lines[index - 1])
|
||||
var blankCount = 0
|
||||
while index < lines.count, lines[index].isEmpty {
|
||||
blankCount += 1
|
||||
index += 1
|
||||
}
|
||||
let afterMoreIndented = index < lines.count && isMoreIndentedBlockLine(lines[index])
|
||||
let breaks = beforeMoreIndented || afterMoreIndented ? blankCount + 1 : blankCount
|
||||
result += String(repeating: "\n", count: breaks)
|
||||
if index < lines.count {
|
||||
result += lines[index]
|
||||
index += 1
|
||||
}
|
||||
} else if isMoreIndentedBlockLine(lines[index]) || isMoreIndentedBlockLine(lines[index - 1]) {
|
||||
// A line break adjacent to a more-indented line is kept literal,
|
||||
// so the more-indented line and its breaks survive folding.
|
||||
result += "\n" + lines[index]
|
||||
index += 1
|
||||
} else {
|
||||
result += " " + lines[index]
|
||||
index += 1
|
||||
}
|
||||
}
|
||||
return result + "\n"
|
||||
}
|
||||
|
||||
func isMoreIndentedBlockLine(_ line: String) -> Bool {
|
||||
// In a block scalar the scanner strips content lines to the block's
|
||||
// indentation, so only a more-indented line retains leading whitespace.
|
||||
line.first == " " || line.first == "\t"
|
||||
}
|
||||
|
||||
func chompedBlockScalar(
|
||||
_ body: String,
|
||||
chomping: PureYAML.Parsing.BlockScalarChomping,
|
||||
) -> String {
|
||||
switch chomping {
|
||||
case .keep:
|
||||
return body
|
||||
case .strip:
|
||||
return blockScalarTrimmingTrailingNewlines(body)
|
||||
case .clip:
|
||||
let trimmed = blockScalarTrimmingTrailingNewlines(body)
|
||||
return trimmed.isEmpty ? "" : trimmed + "\n"
|
||||
}
|
||||
}
|
||||
|
||||
func blockScalarTrimmingTrailingNewlines(_ value: String) -> String {
|
||||
var result = value
|
||||
while result.hasSuffix("\n") {
|
||||
result.removeLast()
|
||||
}
|
||||
return result
|
||||
}
|
||||
}
|
||||
@ -1,26 +0,0 @@
|
||||
extension PureYAML.Parsing.TokenEventParser {
|
||||
/// The key node of an explicit (`?`) mapping entry. The key may be empty: `? : v`, a trailing
|
||||
/// `? }` / `? ]` / `? ,`, or `?` at stream end all denote a null key. With no node and no
|
||||
/// properties to anchor, emit a null scalar rather than failing to find a node.
|
||||
mutating func parseExplicitMappingKey(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
guard properties == .none, isAtEmptyExplicitKey else {
|
||||
return try parseNode(properties: properties)
|
||||
}
|
||||
let mark = cursor.current?.mark ?? cursor.previous?.endMark ?? .start
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: mark, endMark: mark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .scalar, endMark: mark)
|
||||
}
|
||||
|
||||
/// Whether an explicit key has no key node: the next token is the value indicator, a flow
|
||||
/// terminator (`}`, `]`, `,`), or the stream has ended.
|
||||
var isAtEmptyExplicitKey: Bool {
|
||||
switch cursor.current?.kind {
|
||||
case nil, .mappingValue, .flowMappingEnd, .flowSequenceEnd, .flowEntry:
|
||||
true
|
||||
default:
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,170 +0,0 @@
|
||||
extension PureYAML.Parsing.TokenEventParser {
|
||||
mutating func parseFlowSequence(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
let start = try expect("flow sequence start") { kind in
|
||||
if case .flowSequenceStart = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
let startMark = properties.mark ?? start.mark
|
||||
var endMark = start.endMark
|
||||
try emit(.sequenceStart(
|
||||
anchor: properties.anchor,
|
||||
tag: properties.tag,
|
||||
style: .flow,
|
||||
mark: startMark,
|
||||
))
|
||||
var hasEntry = false
|
||||
while cursor.current != nil {
|
||||
if case .flowSequenceEnd = cursor.current?.kind {
|
||||
let end = try expect("flow sequence end") { kind in
|
||||
if case .flowSequenceEnd = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
endMark = end.endMark
|
||||
try emit(.sequenceEnd(mark: endMark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .sequence, endMark: endMark)
|
||||
}
|
||||
if case .flowEntry = cursor.current?.kind {
|
||||
// A comma separates entries, so one must precede it: a leading or repeated comma
|
||||
// is an empty entry, which is not allowed. A single trailing comma before the
|
||||
// closing bracket is fine, since the close is handled above regardless of `hasEntry`.
|
||||
guard hasEntry else {
|
||||
throw unexpectedToken(expected: "flow sequence entry or flow sequence end")
|
||||
}
|
||||
hasEntry = false
|
||||
_ = try cursor.advance()
|
||||
continue
|
||||
}
|
||||
// A flow sequence element that begins with `:` is a single-pair mapping with an empty
|
||||
// key, e.g. `[ : value ]`; an element that begins with a node and is then followed by `:`
|
||||
// already becomes a mapping through the scalar key's implicit-key marker (`[ a: b ]`).
|
||||
if case .mappingValue = cursor.current?.kind {
|
||||
endMark = try parseEmptyKeyFlowMappingElement().endMark
|
||||
} else {
|
||||
endMark = try parseNode().endMark
|
||||
}
|
||||
hasEntry = true
|
||||
if cursor.current?.kind.isFlowTerminator == false {
|
||||
throw unexpectedToken(expected: "flow entry or flow sequence end")
|
||||
}
|
||||
}
|
||||
throw unexpectedToken(expected: "flow sequence end")
|
||||
}
|
||||
|
||||
/// Emits a single-pair flow mapping with an empty (null) key for a flow sequence element that
|
||||
/// begins with `:`. The value follows the `:` and may itself be empty before the next separator.
|
||||
mutating func parseEmptyKeyFlowMappingElement() throws -> PureYAML.Parsing.NodeResult {
|
||||
let mark = cursor.current?.mark ?? .start
|
||||
try emit(.mappingStart(anchor: nil, tag: nil, style: .flow, mark: mark))
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: mark, endMark: mark))
|
||||
let valueIndicator = try expect("mapping value") { kind in
|
||||
if case .mappingValue = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
let properties = try consumeProperties()
|
||||
let endMark: PureYAML.Parsing.Mark
|
||||
if properties == .none, cursor.current?.kind.isFlowTerminator != false {
|
||||
endMark = valueIndicator.endMark
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: endMark, endMark: endMark))
|
||||
} else {
|
||||
endMark = try parseNode(properties: properties).endMark
|
||||
}
|
||||
try emit(.mappingEnd(mark: endMark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .mapping, endMark: endMark)
|
||||
}
|
||||
|
||||
mutating func parseFlowMapping(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
let start = try expect("flow mapping start") { kind in
|
||||
if case .flowMappingStart = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
let startMark = properties.mark ?? start.mark
|
||||
var endMark = start.endMark
|
||||
try emit(.mappingStart(
|
||||
anchor: properties.anchor,
|
||||
tag: properties.tag,
|
||||
style: .flow,
|
||||
mark: startMark,
|
||||
))
|
||||
var hasEntry = false
|
||||
while cursor.current != nil {
|
||||
if case .flowMappingEnd = cursor.current?.kind {
|
||||
let end = try expect("flow mapping end") { kind in
|
||||
if case .flowMappingEnd = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
endMark = end.endMark
|
||||
try emit(.mappingEnd(mark: endMark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .mapping, endMark: endMark)
|
||||
}
|
||||
if case .flowEntry = cursor.current?.kind {
|
||||
// As in a flow sequence, a comma separates entries and so requires one before it;
|
||||
// a leading or repeated comma is an empty entry. A trailing comma is fine.
|
||||
guard hasEntry else {
|
||||
throw unexpectedToken(expected: "flow mapping entry or flow mapping end")
|
||||
}
|
||||
hasEntry = false
|
||||
_ = try cursor.advance()
|
||||
continue
|
||||
}
|
||||
endMark = try parseFlowMappingPair().endMark
|
||||
hasEntry = true
|
||||
if cursor.current?.kind.isFlowTerminator == false {
|
||||
throw unexpectedToken(expected: "flow entry or flow mapping end")
|
||||
}
|
||||
}
|
||||
throw unexpectedToken(expected: "flow mapping end")
|
||||
}
|
||||
|
||||
mutating func parseFlowMappingPair() throws -> PureYAML.Parsing.NodeResult {
|
||||
if cursor.current?.kind.isMappingKey == true {
|
||||
return try parseMappingPair(inFlow: true)
|
||||
}
|
||||
|
||||
// A flow mapping key may be empty (`{ : v }`) or carry only properties (`{ !!str : v }`):
|
||||
// both resolve to a null key, so emit one rather than requiring a node before the `:`. This is
|
||||
// valid only in flow mapping key position, so the empty-node allowance stays local here.
|
||||
let keyProperties = try consumeProperties()
|
||||
let keyEndMark: PureYAML.Parsing.Mark
|
||||
if case .mappingValue = cursor.current?.kind {
|
||||
let mark = keyProperties.mark ?? cursor.current?.mark ?? .start
|
||||
try emit(.scalar(value: "", anchor: keyProperties.anchor, tag: keyProperties.tag, style: .plain, mark: mark, endMark: mark))
|
||||
keyEndMark = mark
|
||||
} else {
|
||||
keyEndMark = try parseNode(properties: keyProperties).endMark
|
||||
}
|
||||
// A flow mapping entry may omit its `:` value indicator: `{ a }` and `{ a, b }` are sets whose
|
||||
// keys map to null. A value parses only when a `:` follows the key.
|
||||
guard case .mappingValue = cursor.current?.kind else {
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: keyEndMark, endMark: keyEndMark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .scalar, endMark: keyEndMark)
|
||||
}
|
||||
let valueIndicator = try expect("mapping value") { kind in
|
||||
if case .mappingValue = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
let properties = try consumeProperties()
|
||||
guard properties != .none || cursor.current?.kind.isFlowTerminator == false else {
|
||||
let mark = valueIndicator.endMark
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: mark, endMark: mark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .scalar, endMark: mark)
|
||||
}
|
||||
return try parseNode(properties: properties)
|
||||
}
|
||||
}
|
||||
@ -1,390 +0,0 @@
|
||||
extension PureYAML.Parsing.TokenEventParser {
|
||||
mutating func parseMappingPair(
|
||||
firstKeyProperties: PureYAML.Parsing.NodeProperties = .none,
|
||||
inFlow: Bool = false,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
// A subsequent pair's key may carry an anchor or tag (`&a key: …`). The first pair's inline
|
||||
// properties are consumed when the mapping is entered and passed back in here; consuming
|
||||
// again covers any on a later key, and is a no-op (`.none`) when there are none.
|
||||
var keyProperties = firstKeyProperties
|
||||
try keyProperties.mergeUnset(from: consumeProperties())
|
||||
_ = try expect("mapping key") { $0.isMappingKey }
|
||||
let key = try parseExplicitMappingKey(properties: keyProperties)
|
||||
// An explicit key (`? key`) with no following `:` has an implicit null value, so several
|
||||
// `? key` entries in a row (a set, for instance) each map to null rather than being an error.
|
||||
guard case .mappingValue = cursor.current?.kind else {
|
||||
return try emitNullMappingValue(at: key.endMark)
|
||||
}
|
||||
let valueIndicator = try expect("mapping value") { kind in
|
||||
if case .mappingValue = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
let properties = try consumeProperties()
|
||||
if inFlow {
|
||||
// In a flow mapping the value is a flow node and may sit on any line: flow is line
|
||||
// insensitive, so the block-only same-line and indentation rules below do not apply.
|
||||
// Only a flow terminator (`,`, `}`, `]`) or the stream end leaves the value empty (null),
|
||||
// mirroring the empty-value rule in `parseFlowMappingPair` for the non-`?` key path.
|
||||
guard properties != .none || cursor.current?.kind.isFlowTerminator == false else {
|
||||
return try emitNullMappingValue(at: valueIndicator.endMark)
|
||||
}
|
||||
return try parseNode(properties: properties)
|
||||
}
|
||||
try enforceValuePropertyIndented(properties, valueIndicator: valueIndicator)
|
||||
if cursor.current?.kind.isIndent == true {
|
||||
return try parseIndentedMappingValue(properties: properties)
|
||||
}
|
||||
if properties == .none, try isCommentInterruptedScalarValue(after: valueIndicator) {
|
||||
return try rejectCommentInterruptedScalar()
|
||||
}
|
||||
if cursor.current?.kind.isMappingKey == true, hasNodeOnSameLine(after: valueIndicator) {
|
||||
return try parseBlockMapping(
|
||||
properties: properties,
|
||||
includeIndentedContinuation: true,
|
||||
minimumColumn: cursor.current?.mark.column,
|
||||
)
|
||||
}
|
||||
// A value position holding only properties (`a: &anchor`) followed by a sibling mapping key
|
||||
// has an empty (null) node carrying those properties, not a nested mapping: the indented
|
||||
// nested case is consumed above by the indent branch, and an inline nested mapping is caught
|
||||
// by the mapping-key-with-same-line-node branch, so a bare mapping key here is a sibling.
|
||||
if properties != .none, cursor.current?.kind.isMappingKey == true {
|
||||
return try emitEmptyNodeOrThrow(properties: properties)
|
||||
}
|
||||
if cursor.current?.kind.isBlockEntry == true {
|
||||
// A block sequence value whose entries return to the mapping key's own indentation is a
|
||||
// zero-indent sequence, which a sibling mapping key terminates. This holds whether or not
|
||||
// the value carries properties (`seq: !!seq` then `- a`), so it is handled before the
|
||||
// generic property-bearing node below, which would not stop at the sibling key and would
|
||||
// misread it as a mixed collection.
|
||||
return try parseBlockSequence(
|
||||
properties: properties,
|
||||
allowMappingKeyTerminator: true,
|
||||
)
|
||||
}
|
||||
if properties != .none || hasNodeOnSameLine(after: valueIndicator) {
|
||||
return try parseNode(properties: properties)
|
||||
}
|
||||
|
||||
return try emitNullMappingValue(at: valueIndicator.endMark)
|
||||
}
|
||||
|
||||
/// Parses a mapping value that begins on an indented line below the `:`. A leading block entry
|
||||
/// is a block sequence; anything else is a node. The closing dedent is consumed unless the value
|
||||
/// ended on a sibling mapping key, which the surrounding mapping continues from.
|
||||
mutating func parseIndentedMappingValue(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
// A node property may sit alone on its own indented line above a block sequence whose entries
|
||||
// return to the mapping key's indentation (`seq:` then ` &anchor` then `- a` back at the key's
|
||||
// column). The grammar allows a block-in-block sequence at the parent indent n while the
|
||||
// property is at n+1, so the indent's matching dedent falls between the property and the
|
||||
// entries. Consume that dedent and bind the property to the sequence rather than emitting an
|
||||
// empty anchored value and orphaning the entries.
|
||||
if let kind = cursor.current?.kind, startsNodeProperties(kind) {
|
||||
var carried = properties
|
||||
try carried.mergeUnset(from: consumeProperties())
|
||||
if cursor.current?.kind.isDedent == true, try cursor.peek(offset: 1)?.kind.isBlockEntry == true {
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
return try parseBlockSequence(properties: carried, allowMappingKeyTerminator: true)
|
||||
}
|
||||
// The node carrying the properties may itself be indented further, when the properties
|
||||
// span their own lines above it (`key: &anchor` / ` !!map` / ` a: b`): descend into that
|
||||
// deeper indented value rather than expecting the node directly after the properties.
|
||||
let node: PureYAML.Parsing.NodeResult = if cursor.current?.kind.isIndent == true {
|
||||
try parseIndentedMappingValue(properties: carried)
|
||||
} else {
|
||||
try parseNode(properties: carried)
|
||||
}
|
||||
try consumeIndentedMappingValueClose()
|
||||
return node
|
||||
}
|
||||
let result: PureYAML.Parsing.NodeResult = if cursor.current?.kind.isBlockEntry == true {
|
||||
try parseBlockSequence(properties: properties, allowMappingKeyTerminator: true)
|
||||
} else {
|
||||
try parseNode(properties: properties)
|
||||
}
|
||||
try consumeIndentedMappingValueClose()
|
||||
return result
|
||||
}
|
||||
|
||||
/// Consume the dedent that closes an indented mapping value, unless the value ended on a sibling
|
||||
/// mapping key, which the surrounding mapping continues from at the same indentation.
|
||||
mutating func consumeIndentedMappingValueClose() throws {
|
||||
if cursor.current?.kind.isDedent == true {
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
} else if cursor.current?.kind.isMappingKey != true {
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
}
|
||||
}
|
||||
|
||||
/// A value's anchor or tag on a line below the `:` must be indented past the key. A properly
|
||||
/// indented property is preceded by an indent token and consumed inside the indented value, so a
|
||||
/// property consumed here on a line below the `:` sits at the key's level and is malformed. The
|
||||
/// last consumed property is `cursor.previous`; checking it catches both a single under-indented
|
||||
/// property (`seq:` then `&anchor`) and a second one after an inline first (`key: &x` then a
|
||||
/// `!!tag` in column 1).
|
||||
func enforceValuePropertyIndented(
|
||||
_ properties: PureYAML.Parsing.NodeProperties,
|
||||
valueIndicator: PureYAML.Parsing.Token,
|
||||
) throws {
|
||||
guard properties != .none, let lastProperty = cursor.previous,
|
||||
lastProperty.mark.line != valueIndicator.mark.line
|
||||
else {
|
||||
return
|
||||
}
|
||||
throw PureYAML.Parsing.ParseError.unexpectedIndentation(line: lastProperty.mark.line)
|
||||
}
|
||||
|
||||
/// Emit an empty (null) scalar as a mapping value and return its node result, used where a
|
||||
/// mapping entry has no value: an explicit key with no `:`, or a `key:` with nothing after it.
|
||||
mutating func emitNullMappingValue(at mark: PureYAML.Parsing.Mark) throws -> PureYAML.Parsing.NodeResult {
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: mark, endMark: mark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .scalar, endMark: mark)
|
||||
}
|
||||
|
||||
/// Rejects a plain scalar that the scanner split into a value-line scalar, an indent, and a
|
||||
/// further scalar. The scanner folds a genuine multi-line plain scalar into a single token, so
|
||||
/// this token shape arises only when a comment broke its continuation: a comment cannot
|
||||
/// interrupt a multi-line plain scalar, so the construct is malformed.
|
||||
mutating func rejectCommentInterruptedScalar() throws -> PureYAML.Parsing.NodeResult {
|
||||
let first = try expect("plain scalar") { kind in
|
||||
if case .scalar(_, .plain) = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
throw PureYAML.Parsing.ParseError.commentInMultilineScalar(line: first.mark.line)
|
||||
}
|
||||
|
||||
mutating func consumeProperties() throws -> PureYAML.Parsing.NodeProperties {
|
||||
var properties = PureYAML.Parsing.NodeProperties.none
|
||||
while let token = cursor.current {
|
||||
switch token.kind {
|
||||
case let .anchor(anchor):
|
||||
properties.anchor = anchor
|
||||
properties.mark = properties.mark ?? token.mark
|
||||
_ = try cursor.advance()
|
||||
case let .tag(tag):
|
||||
properties.tag = tag
|
||||
properties.mark = properties.mark ?? token.mark
|
||||
_ = try cursor.advance()
|
||||
default:
|
||||
return properties
|
||||
}
|
||||
}
|
||||
return properties
|
||||
}
|
||||
|
||||
func decodeScalar(
|
||||
_ value: String,
|
||||
style: PureYAML.Parsing.ScalarStyle,
|
||||
line: Int,
|
||||
) throws -> String {
|
||||
switch style {
|
||||
case .doubleQuoted:
|
||||
try scalarParser.parseDoubleQuoted("\"\(value)\"", line: line)
|
||||
case .folded, .literal:
|
||||
value
|
||||
case .plain:
|
||||
value
|
||||
case .singleQuoted:
|
||||
try scalarParser.parseSingleQuoted("'\(value)'", line: line)
|
||||
}
|
||||
}
|
||||
|
||||
func decodedScalar(from token: PureYAML.Parsing.Token) throws -> String {
|
||||
guard case let .scalar(value, style) = token.kind else {
|
||||
throw unexpectedToken(expected: "scalar")
|
||||
}
|
||||
return try decodeScalar(value, style: style, line: token.mark.line)
|
||||
}
|
||||
|
||||
func hasNodeOnSameLine(after token: PureYAML.Parsing.Token) -> Bool {
|
||||
guard let current = cursor.current, !current.kind.isTerminator else {
|
||||
return false
|
||||
}
|
||||
return current.mark.line == token.mark.line
|
||||
}
|
||||
|
||||
mutating func isCommentInterruptedScalarValue(after token: PureYAML.Parsing.Token) throws -> Bool {
|
||||
guard case .scalar(_, .plain) = cursor.current?.kind else {
|
||||
return false
|
||||
}
|
||||
guard cursor.current?.mark.line == token.mark.line else {
|
||||
return false
|
||||
}
|
||||
guard try cursor.peek()?.kind.isIndent == true else {
|
||||
return false
|
||||
}
|
||||
guard try isScalar(cursor.peek(offset: 2)) else {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
mutating func isCommentInterruptedScalar() throws -> Bool {
|
||||
guard case .scalar(_, .plain) = cursor.current?.kind else {
|
||||
return false
|
||||
}
|
||||
guard try cursor.peek()?.kind.isIndent == true else {
|
||||
return false
|
||||
}
|
||||
guard try isScalar(cursor.peek(offset: 2)) else {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func isScalar(_ token: PureYAML.Parsing.Token?) -> Bool {
|
||||
guard case .scalar = token?.kind else {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
mutating func consumeDedents(atOrAbove minimumColumn: Int) throws {
|
||||
while let width = dedentWidth(cursor.current), width >= minimumColumn {
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
}
|
||||
}
|
||||
|
||||
mutating func consumeMappingSiblingBoundaryDedents(minimumColumn: Int) throws {
|
||||
while try canConsumeMappingSiblingBoundaryDedent(minimumColumn: minimumColumn) {
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
}
|
||||
}
|
||||
|
||||
mutating func canConsumeMappingSiblingBoundaryDedent(minimumColumn: Int) throws -> Bool {
|
||||
guard let width = dedentWidth(cursor.current),
|
||||
let next = try cursor.peek(),
|
||||
next.kind.isMappingKey,
|
||||
next.mark.column >= minimumColumn
|
||||
else {
|
||||
return false
|
||||
}
|
||||
return width == next.mark.column - 1
|
||||
}
|
||||
|
||||
mutating func consumeMappingSiblingBoundaryReindent(minimumColumn: Int) throws {
|
||||
guard let dedent = dedentWidth(cursor.current),
|
||||
dedent < minimumColumn,
|
||||
try indentWidth(cursor.peek()) == minimumColumn,
|
||||
let mappingKey = try cursor.peek(offset: 2),
|
||||
mappingKey.kind.isMappingKey,
|
||||
mappingKey.mark.column == minimumColumn + 1
|
||||
else {
|
||||
return
|
||||
}
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
}
|
||||
|
||||
func dedentWidth(_ token: PureYAML.Parsing.Token?) -> Int? {
|
||||
guard case let .dedent(width) = token?.kind else {
|
||||
return nil
|
||||
}
|
||||
return width
|
||||
}
|
||||
|
||||
func indentWidth(_ token: PureYAML.Parsing.Token?) -> Int? {
|
||||
guard case let .indent(width) = token?.kind else {
|
||||
return nil
|
||||
}
|
||||
return width
|
||||
}
|
||||
|
||||
func canContinueBlockMapping(minimumColumn: Int?) -> Bool {
|
||||
guard let current = cursor.current, current.kind.isMappingKey || startsNodeProperties(current.kind) else {
|
||||
return false
|
||||
}
|
||||
guard let minimumColumn else {
|
||||
return true
|
||||
}
|
||||
return current.mark.column >= minimumColumn
|
||||
}
|
||||
|
||||
/// Whether a token kind begins a node's properties (an anchor or a tag), which a mapping key may
|
||||
/// carry, so the next pair's key is `&a key: …` rather than a bare key.
|
||||
func startsNodeProperties(_ kind: PureYAML.Parsing.TokenKind) -> Bool {
|
||||
switch kind {
|
||||
case .anchor, .tag:
|
||||
true
|
||||
default:
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
func canEndBlockMapping(
|
||||
at token: PureYAML.Parsing.Token?,
|
||||
minimumColumn: Int? = nil,
|
||||
) -> Bool {
|
||||
guard let token else {
|
||||
return true
|
||||
}
|
||||
return token.kind.isBlockEntry
|
||||
|| token.kind.isDedent
|
||||
|| token.kind.isDocumentEnd
|
||||
|| token.kind.isDocumentStart
|
||||
|| token.kind.isFlowTerminator
|
||||
|| isLowerColumnMappingKey(token, minimumColumn: minimumColumn)
|
||||
|| token.kind.isStreamEnd
|
||||
}
|
||||
|
||||
func isLowerColumnMappingKey(
|
||||
_ token: PureYAML.Parsing.Token?,
|
||||
minimumColumn: Int?,
|
||||
) -> Bool {
|
||||
guard let token, token.kind.isMappingKey, let minimumColumn else {
|
||||
return false
|
||||
}
|
||||
return token.mark.column < minimumColumn
|
||||
}
|
||||
|
||||
mutating func expect(
|
||||
_ expected: String,
|
||||
matching: (PureYAML.Parsing.TokenKind) -> Bool,
|
||||
) throws -> PureYAML.Parsing.Token {
|
||||
guard let token = cursor.current else {
|
||||
let mark = cursor.previous?.endMark ?? .start
|
||||
throw PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: expected,
|
||||
actual: "end of tokens",
|
||||
line: mark.line,
|
||||
column: mark.column,
|
||||
)
|
||||
}
|
||||
guard matching(token.kind) else {
|
||||
throw PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: expected,
|
||||
actual: token.kind.description,
|
||||
line: token.mark.line,
|
||||
column: token.mark.column,
|
||||
)
|
||||
}
|
||||
_ = try cursor.advance()
|
||||
return token
|
||||
}
|
||||
|
||||
func unexpectedToken(expected: String) -> PureYAML.Parsing.ParseError {
|
||||
guard let token = cursor.current else {
|
||||
let mark = cursor.previous?.endMark ?? .start
|
||||
return PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: expected,
|
||||
actual: "end of tokens",
|
||||
line: mark.line,
|
||||
column: mark.column,
|
||||
)
|
||||
}
|
||||
return PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: expected,
|
||||
actual: token.kind.description,
|
||||
line: token.mark.line,
|
||||
column: token.mark.column,
|
||||
)
|
||||
}
|
||||
}
|
||||
@ -1,305 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
struct TokenEventParser {
|
||||
var cursor: TokenCursor
|
||||
let scalarParser: Parser
|
||||
var events: [Event]?
|
||||
var eventSink: ((Event) throws -> Void)?
|
||||
|
||||
/// Current node recursion depth, tracked so deeply nested input fails
|
||||
/// with a diagnostic instead of overflowing the stack.
|
||||
var depth = 0
|
||||
|
||||
/// Maximum node nesting the parser will follow before failing with
|
||||
/// ``ParseError/nestingTooDeep(line:column:)``.
|
||||
///
|
||||
/// This bound is the single safety invariant for deeply nested input,
|
||||
/// and it is *necessary*, not merely convenient. A ``Model/Value`` is a
|
||||
/// recursive tree, so composing it, emitting it, comparing it, hashing
|
||||
/// it, and -- crucially -- deallocating it all recurse with the nesting
|
||||
/// depth. Swift tears down a recursive value type recursively, and that
|
||||
/// teardown cannot be made iterative cheaply, so the only way to keep
|
||||
/// every operation within the call stack is to bound the depth of the
|
||||
/// tree where it is built: here, while parsing.
|
||||
///
|
||||
/// Because every parsing entry point routes through ``parseNode``, this
|
||||
/// one bound caps the depth of any parsed tree, which transitively keeps
|
||||
/// the parser, the event composers, the emitter, equality, hashing, and
|
||||
/// deallocation all well inside their limits. Measured on the main
|
||||
/// thread's 8 MB stack: the parser overflows near depth 225 and the
|
||||
/// tree operations near depth 50,000, while real-world specs nest below
|
||||
/// ~40. So 64 accepts every realistic document yet rejects pathological
|
||||
/// input -- for example a 100 KB `[[[...]]]` -- cleanly and early.
|
||||
///
|
||||
/// The bound cannot simply be raised: above ~225 the recursive parser
|
||||
/// itself overflows before the guard fires (lifting it would require an
|
||||
/// iterative parser), and a tree past ~5,000 deep crashes on `deinit`
|
||||
/// regardless, so a higher limit would trade a parse-time failure for an
|
||||
/// un-fixable teardown crash. 64 is the largest value that is both safe
|
||||
/// here and sufficient for real input.
|
||||
///
|
||||
/// Caveat: a tree built *without* the parser -- for example by encoding
|
||||
/// a pathologically deep `Codable` value -- bypasses this bound, and
|
||||
/// operations on such a tree are O(depth) in stack like any recursive
|
||||
/// value type. Keeping those shallow is the caller's responsibility, the
|
||||
/// same way the standard library cannot prevent unbounded recursion in
|
||||
/// user code.
|
||||
static let maxNestingDepth = 64
|
||||
|
||||
init(
|
||||
tokens: [Token],
|
||||
scalarParser: Parser,
|
||||
) {
|
||||
cursor = TokenCursor(tokens: tokens)
|
||||
self.scalarParser = scalarParser
|
||||
events = []
|
||||
eventSink = nil
|
||||
}
|
||||
|
||||
mutating func parse() throws -> [Event] {
|
||||
try parseEvents()
|
||||
return events ?? []
|
||||
}
|
||||
|
||||
init(
|
||||
tokens: [Token],
|
||||
scalarParser: Parser,
|
||||
eventSink: @escaping (Event) throws -> Void,
|
||||
) {
|
||||
cursor = TokenCursor(tokens: tokens)
|
||||
self.scalarParser = scalarParser
|
||||
events = nil
|
||||
self.eventSink = eventSink
|
||||
}
|
||||
|
||||
init(
|
||||
yaml: String,
|
||||
scalarParser: Parser,
|
||||
eventSink: @escaping (Event) throws -> Void,
|
||||
) {
|
||||
cursor = TokenCursor(yaml: yaml)
|
||||
self.scalarParser = scalarParser
|
||||
events = nil
|
||||
self.eventSink = eventSink
|
||||
}
|
||||
|
||||
mutating func parseStreaming() throws {
|
||||
try parseEvents()
|
||||
}
|
||||
|
||||
mutating func parseEvents() throws {
|
||||
let streamStart = try expect("stream start") { kind in
|
||||
if case .streamStart = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
try emit(.streamStart(mark: streamStart.mark))
|
||||
// A stream whose only content is comments or whitespace has zero documents
|
||||
// (YAML 1.2, matching libyaml and the test suite), so no document events are
|
||||
// emitted. The value-level API maps an empty document stream to a single null
|
||||
// value separately.
|
||||
while cursor.current?.kind.isStreamEnd != true {
|
||||
try parseDocument()
|
||||
}
|
||||
let streamEnd = try expect("stream end") { kind in
|
||||
if case .streamEnd = kind {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
try emit(.streamEnd(mark: streamEnd.mark))
|
||||
}
|
||||
|
||||
mutating func emit(_ event: Event) throws {
|
||||
if let eventSink {
|
||||
try eventSink(event)
|
||||
} else {
|
||||
events?.append(event)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
extension PureYAML.Parsing.TokenEventParser {
|
||||
mutating func parseDocument() throws {
|
||||
let startMark = try parseDocumentStartMark()
|
||||
try emit(.documentStart(mark: startMark))
|
||||
|
||||
if cursor.current?.kind.isDocumentEnd == true {
|
||||
let end = try expect("document end") { $0.isDocumentEnd }
|
||||
try appendEmptyDocument(at: end.mark)
|
||||
try emit(.documentEnd(mark: end.endMark))
|
||||
return
|
||||
}
|
||||
|
||||
if cursor.current?.kind.isDocumentStart == true || cursor.current?.kind.isStreamEnd == true {
|
||||
try appendEmptyDocument(at: cursor.current?.mark ?? startMark)
|
||||
try emit(.documentEnd(mark: cursor.current?.mark ?? startMark))
|
||||
return
|
||||
}
|
||||
|
||||
var endMark = try parseDocumentRootNode().endMark
|
||||
while cursor.current?.kind.isDedent == true {
|
||||
let dedent = try expect("dedent") { $0.isDedent }
|
||||
endMark = dedent.endMark
|
||||
}
|
||||
if cursor.current?.kind.isDocumentEnd == true {
|
||||
// A `...` ends this document. Content after it begins a new bare document, parsed by the
|
||||
// outer stream loop, so emit the end and return rather than expecting a boundary here.
|
||||
let end = try expect("document end") { $0.isDocumentEnd }
|
||||
try emit(.documentEnd(mark: end.endMark))
|
||||
return
|
||||
}
|
||||
if let current = cursor.current, !canEndDocument(at: current) {
|
||||
throw PureYAML.Parsing.ParseError.unexpectedToken(
|
||||
expected: "document boundary or stream end",
|
||||
actual: current.kind.description,
|
||||
line: current.mark.line,
|
||||
column: current.mark.column,
|
||||
)
|
||||
}
|
||||
try emit(.documentEnd(mark: endMark))
|
||||
}
|
||||
|
||||
mutating func parseDocumentRootNode() throws -> PureYAML.Parsing.NodeResult {
|
||||
guard cursor.current?.kind.isIndent == true else {
|
||||
return try parseNode()
|
||||
}
|
||||
_ = try expect("indent") { $0.isIndent }
|
||||
let result = try parseNode()
|
||||
if cursor.current?.kind.isDedent == true {
|
||||
_ = try expect("dedent") { $0.isDedent }
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
mutating func parseDocumentStartMark() throws -> PureYAML.Parsing.Mark {
|
||||
guard cursor.current?.kind.isDocumentStart == true else {
|
||||
return cursor.current?.mark ?? cursor.previous?.endMark ?? .start
|
||||
}
|
||||
let start = try cursor.advance()
|
||||
return start?.mark ?? .start
|
||||
}
|
||||
|
||||
mutating func appendEmptyDocument(at mark: PureYAML.Parsing.Mark) throws {
|
||||
try emit(.scalar(value: "", anchor: nil, tag: nil, style: .plain, mark: mark, endMark: mark))
|
||||
}
|
||||
|
||||
func canEndDocument(at token: PureYAML.Parsing.Token) -> Bool {
|
||||
token.kind.isDocumentStart || token.kind.isStreamEnd
|
||||
}
|
||||
|
||||
mutating func parseNode() throws -> PureYAML.Parsing.NodeResult {
|
||||
let properties = try consumeProperties()
|
||||
return try parseNode(properties: properties)
|
||||
}
|
||||
|
||||
/// Reject documents nested past ``maxNestingDepth`` with a diagnostic so a
|
||||
/// pathologically nested input fails cleanly instead of overflowing the
|
||||
/// recursive-descent stack.
|
||||
func enforceNestingLimit(properties: PureYAML.Parsing.NodeProperties) throws {
|
||||
guard depth > Self.maxNestingDepth else {
|
||||
return
|
||||
}
|
||||
let mark = properties.mark ?? cursor.current?.mark ?? cursor.previous?.endMark ?? .start
|
||||
throw PureYAML.Parsing.ParseError.nestingTooDeep(line: mark.line, column: mark.column)
|
||||
}
|
||||
|
||||
/// Handles a node position with no node token. An anchor or tag with nothing after it
|
||||
/// (`- !!str`, `&a` alone) applies to an empty node, so emit a null scalar carrying the
|
||||
/// properties; with no properties, a missing node is an error.
|
||||
mutating func emitEmptyNodeOrThrow(
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
let mark = properties.mark ?? cursor.current?.mark ?? cursor.previous?.endMark ?? .start
|
||||
guard properties.anchor != nil || properties.tag != nil else {
|
||||
throw PureYAML.Parsing.ParseError.expectedNode(line: mark.line, column: mark.column)
|
||||
}
|
||||
try emit(.scalar(value: "", anchor: properties.anchor, tag: properties.tag, style: .plain, mark: mark, endMark: mark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .scalar, endMark: mark)
|
||||
}
|
||||
|
||||
/// An alias is a bare reference, so it cannot carry an anchor or tag of its own (`&b *a`).
|
||||
mutating func parseAlias(
|
||||
anchor: String,
|
||||
token: PureYAML.Parsing.Token,
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
guard properties.anchor == nil, properties.tag == nil else {
|
||||
let mark = properties.mark ?? token.mark
|
||||
throw PureYAML.Parsing.ParseError.aliasWithProperties(line: mark.line, column: mark.column)
|
||||
}
|
||||
_ = try cursor.advance()
|
||||
try emit(.alias(anchor: anchor, mark: token.mark))
|
||||
return PureYAML.Parsing.NodeResult(kind: .alias, endMark: token.endMark)
|
||||
}
|
||||
|
||||
mutating func parseNode(
|
||||
properties initialProperties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
depth += 1
|
||||
defer { depth -= 1 }
|
||||
try enforceNestingLimit(properties: initialProperties)
|
||||
var properties = initialProperties
|
||||
try properties.mergeUnset(from: consumeProperties())
|
||||
|
||||
guard let token = cursor.current, !token.kind.isTerminator else {
|
||||
return try emitEmptyNodeOrThrow(properties: properties)
|
||||
}
|
||||
|
||||
switch token.kind {
|
||||
case let .alias(anchor):
|
||||
return try parseAlias(anchor: anchor, token: token, properties: properties)
|
||||
case .blockEntry:
|
||||
return try parseBlockSequence(properties: properties)
|
||||
case .blockScalarHeader:
|
||||
return try parseBlockScalar(properties: properties)
|
||||
case .flowMappingStart:
|
||||
return try parseFlowMapping(properties: properties)
|
||||
case .flowSequenceStart:
|
||||
return try parseFlowSequence(properties: properties)
|
||||
case .mappingKey:
|
||||
return try parseBlockMapping(
|
||||
properties: properties,
|
||||
includeIndentedContinuation: false,
|
||||
)
|
||||
case let .scalar(value, style):
|
||||
return try parseScalarNode(value: value, style: style, token: token, properties: properties)
|
||||
case .mappingValue:
|
||||
// An explicit key that is only properties before its `:` (`? &e` then `: v`) is an empty
|
||||
// key carrying those properties. Emit the empty scalar without consuming the `:`, so the
|
||||
// mapping machinery reads it as the value indicator. With no properties this is a misplaced
|
||||
// `:`, which `emitEmptyNodeOrThrow` keeps as an error. (The implicit form `&e : v` never
|
||||
// reaches here: the scanner synthesizes its empty key as an explicit scalar token.)
|
||||
return try emitEmptyNodeOrThrow(properties: properties)
|
||||
default:
|
||||
throw PureYAML.Parsing.ParseError.expectedNode(line: token.mark.line, column: token.mark.column)
|
||||
}
|
||||
}
|
||||
|
||||
/// Emits a scalar node from a scalar token, applying any node properties. A plain scalar that the
|
||||
/// scanner split around a comment (the comment-interrupted shape) is rejected rather than emitted.
|
||||
mutating func parseScalarNode(
|
||||
value: String,
|
||||
style: PureYAML.Parsing.ScalarStyle,
|
||||
token: PureYAML.Parsing.Token,
|
||||
properties: PureYAML.Parsing.NodeProperties,
|
||||
) throws -> PureYAML.Parsing.NodeResult {
|
||||
if try isCommentInterruptedScalar() {
|
||||
return try rejectCommentInterruptedScalar()
|
||||
}
|
||||
_ = try cursor.advance()
|
||||
let decoded = try decodeScalar(value, style: style, line: token.mark.line)
|
||||
let mark = properties.mark ?? token.mark
|
||||
try emit(.scalar(
|
||||
value: decoded,
|
||||
anchor: properties.anchor,
|
||||
tag: properties.tag,
|
||||
style: style,
|
||||
mark: mark,
|
||||
endMark: token.endMark,
|
||||
))
|
||||
return PureYAML.Parsing.NodeResult(kind: .scalar, endMark: token.endMark)
|
||||
}
|
||||
}
|
||||
@ -1,96 +0,0 @@
|
||||
extension PureYAML.Parsing {
|
||||
enum TokenKind: Equatable {
|
||||
case streamStart
|
||||
case streamEnd
|
||||
case documentStart
|
||||
case documentEnd
|
||||
case indent(width: Int)
|
||||
case dedent(width: Int)
|
||||
case blockEntry
|
||||
case mappingKey
|
||||
case mappingValue
|
||||
case flowSequenceStart
|
||||
case flowSequenceEnd
|
||||
case flowMappingStart
|
||||
case flowMappingEnd
|
||||
case flowEntry
|
||||
case scalar(value: String, style: ScalarStyle)
|
||||
case blockScalarHeader(style: BlockScalarStyle, chomping: BlockScalarChomping)
|
||||
case anchor(String)
|
||||
case alias(String)
|
||||
case tag(String)
|
||||
case comment(String)
|
||||
}
|
||||
}
|
||||
|
||||
extension PureYAML.Parsing.TokenKind: CustomStringConvertible {
|
||||
var description: String {
|
||||
switch self {
|
||||
case .streamStart:
|
||||
"streamStart"
|
||||
case .streamEnd:
|
||||
"streamEnd"
|
||||
case .documentStart:
|
||||
"documentStart"
|
||||
case .documentEnd:
|
||||
"documentEnd"
|
||||
case let .indent(width):
|
||||
"indent width=\(width)"
|
||||
case let .dedent(width):
|
||||
"dedent width=\(width)"
|
||||
case .blockEntry:
|
||||
"blockEntry"
|
||||
case .mappingKey:
|
||||
"mappingKey"
|
||||
case .mappingValue:
|
||||
"mappingValue"
|
||||
case .flowSequenceStart:
|
||||
"flowSequenceStart"
|
||||
case .flowSequenceEnd:
|
||||
"flowSequenceEnd"
|
||||
case .flowMappingStart:
|
||||
"flowMappingStart"
|
||||
case .flowMappingEnd:
|
||||
"flowMappingEnd"
|
||||
case .flowEntry:
|
||||
"flowEntry"
|
||||
case let .scalar(value, style):
|
||||
"scalar value=\"\(escape(value))\" style=\(style)"
|
||||
case let .blockScalarHeader(style, chomping):
|
||||
if chomping == .clip {
|
||||
"blockScalarHeader style=\(style)"
|
||||
} else {
|
||||
"blockScalarHeader style=\(style) chomping=\(chomping)"
|
||||
}
|
||||
case let .anchor(value):
|
||||
"anchor name=\(value)"
|
||||
case let .alias(value):
|
||||
"alias name=\(value)"
|
||||
case let .tag(value):
|
||||
"tag value=\(value)"
|
||||
case let .comment(value):
|
||||
"comment value=\"\(escape(value))\""
|
||||
}
|
||||
}
|
||||
|
||||
private func escape(_ value: String) -> String {
|
||||
var output = ""
|
||||
for character in value {
|
||||
switch character {
|
||||
case "\\":
|
||||
output += "\\\\"
|
||||
case "\"":
|
||||
output += "\\\""
|
||||
case "\n":
|
||||
output += "\\n"
|
||||
case "\r":
|
||||
output += "\\r"
|
||||
case "\t":
|
||||
output += "\\t"
|
||||
default:
|
||||
output.append(character)
|
||||
}
|
||||
}
|
||||
return output
|
||||
}
|
||||
}
|
||||
44
Sources/PureYAMLJSON/JSON.Check.Builtins.swift
Normal file
44
Sources/PureYAMLJSON/JSON.Check.Builtins.swift
Normal file
@ -0,0 +1,44 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.Validation.BuiltinCheck {
|
||||
/// Reports any mapping key that is not a string. JSON object names must be
|
||||
/// strings, so a YAML complex key (a sequence or mapping used as a key) has no
|
||||
/// JSON form. PureYAML core already ships ``mappingKeysAreUnique``; this check and
|
||||
/// ``numbersAreFinite`` complete the JSON-representability set.
|
||||
static var mappingKeysAreStrings: PureYAML.Validation.Check<PureYAML.Model.Mapping, PureYAML.Model.Value> {
|
||||
PureYAML.Validation.Check(description: "Mapping keys are strings") { context in
|
||||
context.subject.pairs.compactMap { pair in
|
||||
guard pair.keyNode.stringValue == nil else {
|
||||
return nil
|
||||
}
|
||||
return PureYAML.Validation.Issue(
|
||||
severity: .error,
|
||||
reason: "Mapping key '\(pair.keyNode.description)' is not a string",
|
||||
path: context.path.appending(.complexKey(pair.keyNode)),
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Reports any non-finite number (NaN or infinity). JSON has no literal for
|
||||
/// these, so a value tree carrying one is not representable as JSON.
|
||||
static var numbersAreFinite: PureYAML.Validation.Check<PureYAML.Model.Value, PureYAML.Model.Value> {
|
||||
PureYAML.Validation.Check(description: "Numbers are finite") { context in
|
||||
guard case let .double(number) = context.subject, !number.isFinite else {
|
||||
return []
|
||||
}
|
||||
return [
|
||||
PureYAML.Validation.Issue(
|
||||
severity: .error,
|
||||
reason: "Number '\(number)' is not finite",
|
||||
path: context.path,
|
||||
),
|
||||
]
|
||||
} when: { context in
|
||||
if case .double = context.subject {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
13
Sources/PureYAMLJSON/JSON.Convert.Failure.swift
Normal file
13
Sources/PureYAMLJSON/JSON.Convert.Failure.swift
Normal file
@ -0,0 +1,13 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.JSON.Convert {
|
||||
/// A reason a YAML <-> JSON conversion did not complete.
|
||||
enum Failure: Error, Equatable, Sendable {
|
||||
/// The source text did not parse.
|
||||
case sourceInvalid(String)
|
||||
/// The value cannot be represented in the target format.
|
||||
case notRepresentable([PureYAML.Validation.Issue])
|
||||
/// The emitted output did not parse back to the source value.
|
||||
case roundTripMismatch(PureYAML.JSON.Format)
|
||||
}
|
||||
}
|
||||
64
Sources/PureYAMLJSON/JSON.Convert.swift
Normal file
64
Sources/PureYAMLJSON/JSON.Convert.swift
Normal file
@ -0,0 +1,64 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.JSON {
|
||||
/// Standalone YAML <-> JSON conversion. It depends only on the parsing/emitting
|
||||
/// engine and the JSON writer; it knows nothing about any editor or UI, so it is
|
||||
/// usable on its own (a CLI, a server, a build step, a test).
|
||||
///
|
||||
/// A conversion succeeds only when, in order:
|
||||
/// 1. the source parses,
|
||||
/// 2. the value is representable in the target format, and
|
||||
/// 3. the emitted output parses back to a value identical to the source
|
||||
/// (a lossless round-trip check that also guards against emitter defects).
|
||||
///
|
||||
/// JSON is a subset of YAML 1.2, so the same engine parses both the source and
|
||||
/// the re-parsed output; only "to JSON" can be rejected for representability (a
|
||||
/// value may use a non-string key or a non-finite number). "To YAML" still
|
||||
/// rejects duplicate keys, which are legal in JSON but not in YAML.
|
||||
enum Convert {
|
||||
/// Convert source text to the target format, validated end to end.
|
||||
public static func convert(_ source: String, to target: Format, pretty: Bool = true) throws -> String {
|
||||
let value = try parse(source)
|
||||
let output = try emit(value, as: target, pretty: pretty)
|
||||
guard try parse(output) == value else {
|
||||
throw Failure.roundTripMismatch(target)
|
||||
}
|
||||
return output
|
||||
}
|
||||
|
||||
/// Emit a value tree in the target format. The value is validated for
|
||||
/// representability first; the round-trip check is not run here (use
|
||||
/// ``convert(_:to:pretty:)`` for the fully validated path).
|
||||
public static func emit(_ value: PureYAML.Model.Value, as target: Format, pretty: Bool = true) throws -> String {
|
||||
let issues = validator(for: target).collect(value).issues
|
||||
guard issues.isEmpty else {
|
||||
throw Failure.notRepresentable(issues)
|
||||
}
|
||||
switch target {
|
||||
case .json:
|
||||
return try PureYAML.JSON.Writer.string(from: value, options: .init(pretty: pretty, indent: 2))
|
||||
case .yaml:
|
||||
return PureYAML.dump(value, options: .init(scalarStyle: .plainWhenSafe, collectionStyle: .block))
|
||||
}
|
||||
}
|
||||
|
||||
/// The validator for a target format. JSON additionally requires string keys
|
||||
/// and finite numbers; YAML allows complex keys but still forbids duplicates.
|
||||
public static func validator(for target: Format) -> PureYAML.Validation.Validator {
|
||||
switch target {
|
||||
case .json:
|
||||
PureYAML.JSON.Validator.representability
|
||||
case .yaml:
|
||||
PureYAML.Validation.Validator()
|
||||
}
|
||||
}
|
||||
|
||||
private static func parse(_ text: String) throws -> PureYAML.Model.Value {
|
||||
do {
|
||||
return try PureYAML.parse(text)
|
||||
} catch {
|
||||
throw Failure.sourceInvalid(String(describing: error))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
11
Sources/PureYAMLJSON/JSON.Format.swift
Normal file
11
Sources/PureYAMLJSON/JSON.Format.swift
Normal file
@ -0,0 +1,11 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.JSON {
|
||||
/// A serialization format the converter can emit. The source is always parsed by
|
||||
/// PureYAML (JSON is a subset of YAML, so one parser reads both); only the target
|
||||
/// needs to be named.
|
||||
enum Format: String, CaseIterable, Sendable, Equatable {
|
||||
case yaml
|
||||
case json
|
||||
}
|
||||
}
|
||||
35
Sources/PureYAMLJSON/JSON.Validator.swift
Normal file
35
Sources/PureYAMLJSON/JSON.Validator.swift
Normal file
@ -0,0 +1,35 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.JSON {
|
||||
/// JSON-representability checks built on PureYAML's own validation framework.
|
||||
/// The parser already guarantees the input is well-formed YAML (and so covers
|
||||
/// JSON syntax); this layer reports the value-level reasons a tree cannot be
|
||||
/// emitted as JSON: a non-string mapping key, a non-finite number, or (for a
|
||||
/// deterministic single-meaning document) a duplicate key.
|
||||
///
|
||||
/// JSON the format tolerates duplicate object names, but a tree that carries two
|
||||
/// pairs with the same key emits an ambiguous document, so the representability
|
||||
/// validator rejects duplicates for determinism. Drop ``unique`` from the rule
|
||||
/// set if you want the permissive reading.
|
||||
enum Validator {
|
||||
/// The full JSON-representability rule set: unique keys, string keys, finite
|
||||
/// numbers. Suitable to pass straight to `PureYAML.JSON.Convert`.
|
||||
public static var representability: PureYAML.Validation.Validator {
|
||||
PureYAML.Validation.Validator(checks: [
|
||||
PureYAML.Validation.AnyCheck(PureYAML.Validation.BuiltinCheck.mappingKeysAreUnique),
|
||||
PureYAML.Validation.AnyCheck(PureYAML.Validation.BuiltinCheck.mappingKeysAreStrings),
|
||||
PureYAML.Validation.AnyCheck(PureYAML.Validation.BuiltinCheck.numbersAreFinite),
|
||||
])
|
||||
}
|
||||
|
||||
/// Every reason the value cannot be emitted as strict JSON, in document order.
|
||||
public static func issues(for value: PureYAML.Model.Value) -> [PureYAML.Validation.Issue] {
|
||||
representability.collect(value).issues
|
||||
}
|
||||
|
||||
/// Whether the value can be emitted as strict JSON.
|
||||
public static func isRepresentable(_ value: PureYAML.Model.Value) -> Bool {
|
||||
issues(for: value).isEmpty
|
||||
}
|
||||
}
|
||||
}
|
||||
11
Sources/PureYAMLJSON/JSON.Writer.Failure.swift
Normal file
11
Sources/PureYAMLJSON/JSON.Writer.Failure.swift
Normal file
@ -0,0 +1,11 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.JSON.Writer {
|
||||
/// A reason a value tree cannot be written as strict JSON.
|
||||
enum Failure: Error, Sendable, Equatable {
|
||||
/// A YAML complex (sequence/mapping) key has no JSON object-name form.
|
||||
case nonStringKey(String)
|
||||
/// JSON cannot represent NaN or infinity.
|
||||
case nonFiniteNumber(Double)
|
||||
}
|
||||
}
|
||||
19
Sources/PureYAMLJSON/JSON.Writer.Options.swift
Normal file
19
Sources/PureYAMLJSON/JSON.Writer.Options.swift
Normal file
@ -0,0 +1,19 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.JSON.Writer {
|
||||
/// Formatting options for the JSON writer.
|
||||
struct Options: Sendable, Equatable {
|
||||
/// Indent with newlines (`true`) or emit a single compact line (`false`).
|
||||
public var pretty: Bool
|
||||
/// Spaces per indentation level when `pretty`.
|
||||
public var indent: Int
|
||||
|
||||
public init(pretty: Bool = true, indent: Int = 2) {
|
||||
self.pretty = pretty
|
||||
self.indent = indent
|
||||
}
|
||||
|
||||
public static let pretty = Options(pretty: true, indent: 2)
|
||||
public static let compact = Options(pretty: false, indent: 0)
|
||||
}
|
||||
}
|
||||
129
Sources/PureYAMLJSON/JSON.Writer.swift
Normal file
129
Sources/PureYAMLJSON/JSON.Writer.swift
Normal file
@ -0,0 +1,129 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML.JSON {
|
||||
/// A pure-Swift JSON serializer for the YAML/JSON value tree. Converting a
|
||||
/// parsed document to strict JSON is the core "emit as JSON" operation; since
|
||||
/// JSON is a subset of YAML 1.2, any value PureYAML parses can be emitted as
|
||||
/// JSON unless it uses a YAML-only feature with no JSON representation (a
|
||||
/// non-string mapping key, or a non-finite number).
|
||||
enum Writer {
|
||||
/// Serialize a value tree to a JSON string.
|
||||
public static func string(from value: PureYAML.Model.Value, options: Options = .pretty) throws -> String {
|
||||
var out = ""
|
||||
try write(value, into: &out, depth: 0, options: options)
|
||||
return out
|
||||
}
|
||||
|
||||
private static func write(
|
||||
_ value: PureYAML.Model.Value,
|
||||
into out: inout String,
|
||||
depth: Int,
|
||||
options: Options,
|
||||
) throws {
|
||||
switch value {
|
||||
case .null:
|
||||
out += "null"
|
||||
case let .bool(flag):
|
||||
out += flag ? "true" : "false"
|
||||
case let .int(number):
|
||||
out += String(number)
|
||||
case let .double(number):
|
||||
guard number.isFinite else { throw Failure.nonFiniteNumber(number) }
|
||||
out += String(number)
|
||||
case let .string(text):
|
||||
out += escaped(text)
|
||||
case let .sequence(items):
|
||||
try writeArray(items, into: &out, depth: depth, options: options)
|
||||
case let .mapping(mapping):
|
||||
try writeObject(mapping, into: &out, depth: depth, options: options)
|
||||
}
|
||||
}
|
||||
|
||||
private static func writeArray(
|
||||
_ items: [PureYAML.Model.Value],
|
||||
into out: inout String,
|
||||
depth: Int,
|
||||
options: Options,
|
||||
) throws {
|
||||
guard !items.isEmpty else {
|
||||
out += "[]"
|
||||
return
|
||||
}
|
||||
out += "["
|
||||
for (index, item) in items.enumerated() {
|
||||
out += index == 0 ? "" : ","
|
||||
newline(&out, depth: depth + 1, options: options)
|
||||
try write(item, into: &out, depth: depth + 1, options: options)
|
||||
}
|
||||
newline(&out, depth: depth, options: options)
|
||||
out += "]"
|
||||
}
|
||||
|
||||
private static func writeObject(
|
||||
_ mapping: PureYAML.Model.Mapping,
|
||||
into out: inout String,
|
||||
depth: Int,
|
||||
options: Options,
|
||||
) throws {
|
||||
guard !mapping.pairs.isEmpty else {
|
||||
out += "{}"
|
||||
return
|
||||
}
|
||||
out += "{"
|
||||
for (index, pair) in mapping.pairs.enumerated() {
|
||||
guard let key = pair.keyNode.stringValue else {
|
||||
throw Failure.nonStringKey(pair.keyNode.description)
|
||||
}
|
||||
out += index == 0 ? "" : ","
|
||||
newline(&out, depth: depth + 1, options: options)
|
||||
out += escaped(key)
|
||||
out += options.pretty ? ": " : ":"
|
||||
try write(pair.value, into: &out, depth: depth + 1, options: options)
|
||||
}
|
||||
newline(&out, depth: depth, options: options)
|
||||
out += "}"
|
||||
}
|
||||
|
||||
private static func newline(_ out: inout String, depth: Int, options: Options) {
|
||||
guard options.pretty else { return }
|
||||
out += "\n"
|
||||
out += String(repeating: " ", count: depth * options.indent)
|
||||
}
|
||||
|
||||
/// Quote and escape a string per RFC 8259: `"` and `\` are escaped, the
|
||||
/// named short escapes are used where they apply, and any remaining control
|
||||
/// character below U+0020 becomes a `\u00XX` sequence.
|
||||
private static func escaped(_ text: String) -> String {
|
||||
var out = "\""
|
||||
for scalar in text.unicodeScalars {
|
||||
switch scalar {
|
||||
case "\"": out += "\\\""
|
||||
case "\\": out += "\\\\"
|
||||
case "\u{08}": out += "\\b"
|
||||
case "\u{0C}": out += "\\f"
|
||||
case "\n": out += "\\n"
|
||||
case "\r": out += "\\r"
|
||||
case "\t": out += "\\t"
|
||||
default:
|
||||
if scalar.value < 0x20 {
|
||||
out += "\\u" + hex4(scalar.value)
|
||||
} else {
|
||||
out.unicodeScalars.append(scalar)
|
||||
}
|
||||
}
|
||||
}
|
||||
out += "\""
|
||||
return out
|
||||
}
|
||||
|
||||
private static func hex4(_ value: UInt32) -> String {
|
||||
let digits = "0123456789abcdef"
|
||||
var result = ""
|
||||
for shift in stride(from: 12, through: 0, by: -4) {
|
||||
let index = Int((value >> UInt32(shift)) & 0xF)
|
||||
result.append(digits[digits.index(digits.startIndex, offsetBy: index)])
|
||||
}
|
||||
return result
|
||||
}
|
||||
}
|
||||
}
|
||||
14
Sources/PureYAMLJSON/JSON.swift
Normal file
14
Sources/PureYAMLJSON/JSON.swift
Normal file
@ -0,0 +1,14 @@
|
||||
import PureYAML
|
||||
|
||||
public extension PureYAML {
|
||||
/// Strict-JSON support layered on the shared YAML value tree: a pure-Swift
|
||||
/// RFC-8259 emitter, JSON-representability rules expressed in PureYAML's own
|
||||
/// validation framework, and a standalone YAML <-> JSON converter.
|
||||
///
|
||||
/// JSON is a subset of YAML 1.2, so the parser PureYAML already ships reads both
|
||||
/// JSON and YAML sources into the same ``Model/Value`` tree. This module adds the
|
||||
/// missing half (emitting that tree as JSON) plus the value-level checks that
|
||||
/// decide whether a tree is representable as JSON at all. Anchor file; concrete
|
||||
/// types live in the sibling `JSON.*` files.
|
||||
enum JSON {}
|
||||
}
|
||||
@ -35,7 +35,7 @@ extension PureYAML.Tagged {
|
||||
}
|
||||
return false
|
||||
}
|
||||
if case let .documentStart(mark)? = current {
|
||||
if case let .documentStart(_, mark)? = current {
|
||||
throw PureYAML.Parsing.ParseError.unsupportedMultiDocumentStream(line: mark.line)
|
||||
}
|
||||
_ = try expect("stream end") { event in
|
||||
|
||||
@ -213,10 +213,7 @@ public extension PureYAML {
|
||||
reason: streamIssue.issue.reason,
|
||||
)
|
||||
}
|
||||
// The parse is permissive: a tag whose handle was never declared by a `%TAG` directive
|
||||
// in its document still yields a value, but it is a genuine error, so surface it here.
|
||||
let tagHandleDiagnostics = tagHandleDiagnostics(in: yaml, file: file)
|
||||
return Validation.Report(tagHandleDiagnostics + diagnostics)
|
||||
return Validation.Report(diagnostics)
|
||||
} catch let error as PureYAML.Parsing.ParseError {
|
||||
return Validation.Report([
|
||||
Validation.Diagnostic(
|
||||
@ -241,26 +238,6 @@ public extension PureYAML {
|
||||
}
|
||||
}
|
||||
|
||||
/// Diagnostics for tag shorthands whose named handle was never declared by a `%TAG` directive
|
||||
/// in the document that uses it. The scanner finds these with full lexical context (it knows a
|
||||
/// tag token from a `!` inside a scalar or comment), so the check is exact, not a heuristic.
|
||||
static func tagHandleDiagnostics(
|
||||
in yaml: String,
|
||||
file: String?,
|
||||
) -> [Validation.Diagnostic] {
|
||||
Parsing.Scanner().tagHandleIssues(yaml).map { issue in
|
||||
Validation.Diagnostic(
|
||||
kind: .parse,
|
||||
code: "undefinedTagHandle",
|
||||
severity: .error,
|
||||
file: file,
|
||||
line: issue.line,
|
||||
column: issue.column,
|
||||
reason: "tag handle '\(issue.handle)' is not declared by a %TAG directive in this document",
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// Validates many YAML inputs without throwing or stopping at the first failure.
|
||||
static func validationReports(
|
||||
_ sources: [Validation.Source],
|
||||
|
||||
@ -2,105 +2,120 @@ import Foundation
|
||||
@testable import PureYAML
|
||||
import Testing
|
||||
|
||||
/// Canonical event-stream conformance gate: the strongest oracle for the parser. It serializes
|
||||
/// PureYAML's parse events into the official YAML test-suite "event" (yeast) format and compares them
|
||||
/// exactly to each case's `test.event`. Unlike the JSON-projection oracle, this checks anchors, tags,
|
||||
/// scalar styles, flow vs block, and aliases, for every valid case including those without an `in.json`.
|
||||
///
|
||||
/// Opt-in like the other suite gates, since the corpus is fetched, not vendored:
|
||||
/// bash scripts/fetch-yaml-test-suite.sh
|
||||
/// PUREYAML_RUN_YAML_TEST_SUITE=1 swift test --filter CanonicalEventConformanceTests
|
||||
///
|
||||
/// `knownEventGaps` is the authoritative gap set: every valid case whose event stream PureYAML does not
|
||||
/// yet emit exactly. It is a gate, not a measurement: a fix must remove an id (the test fails until it
|
||||
/// is removed), and a regression adds one (the test fails). No case is ever silently skipped.
|
||||
///
|
||||
/// One documented normalization. `Parsing.Event` does not carry the explicit document-marker flag, so
|
||||
/// `+DOC`/`-DOC` lines are compared without their ` ---`/` ...` suffix. The marker distinction is the
|
||||
/// single property this gate does not yet check; enriching the event API to carry the flag removes the
|
||||
/// normalization and this caveat with it.
|
||||
/// Authoritative conformance gate: serialize PureYAML's parse events into the
|
||||
/// official YAML test suite "event" (yeast) format and compare EXACTLY to each
|
||||
/// case's vendored `test.event`. Unlike the JSON-projection oracle, this checks
|
||||
/// anchors, tags, scalar styles, flow vs block, and explicit document markers,
|
||||
/// for every valid case (including those with no `in.json`). No normalization.
|
||||
@Suite("Canonical Event Conformance")
|
||||
struct CanonicalEventConformanceTests {
|
||||
private struct Case {
|
||||
let id: String
|
||||
let input: String
|
||||
let expectedEvents: String
|
||||
}
|
||||
|
||||
/// The authoritative event-level frontier: valid cases that parse+value-match but whose canonical
|
||||
/// event stream PureYAML does not yet emit exactly. Each is a precise, recorded gap (first-line
|
||||
/// event diff available via the triage below), not a silent skip. Closing one removes its id; a
|
||||
/// regression adds one. Audited 2026-06-29: every entry is a real parser deviation (e.g. 4ABK
|
||||
/// truncates `http://foo.com` at the `:`; FH7J/PW8X bind a property to the collection instead of
|
||||
/// the empty scalar), not a serializer artifact.
|
||||
static let knownEventGaps: Set<String> = [
|
||||
"4ABK", "4FJ6", "6BFJ", "6CK3", "7BMT", "7FWL", "87E4", "8UDB", "9KAX", "9MMW",
|
||||
"CN3R", "CT4Q", "DC7X", "FH7J", "HWV9", "L9U5", "LP6E", "LQZ7", "M5C3", "M5DY",
|
||||
"PW8X", "QF4Y", "QT73", "RZP5", "U3XV", "UGM3", "UT92", "V9D5", "XW4D", "ZF4X", "ZK9H",
|
||||
]
|
||||
/// Valid suite cases whose event stream PureYAML does NOT yet emit exactly.
|
||||
/// This is the authoritative gap set: a fix removes an id (and the test fails
|
||||
/// until it is removed), a regression adds one (and the test fails).
|
||||
///
|
||||
/// The ported libyaml engine now emits the exact canonical event stream for
|
||||
/// every valid case in the vendored YAML test suite, so this set is empty.
|
||||
/// Each entry that used to live here was a libyaml deviation from YAML 1.2 that
|
||||
/// the port has since surpassed; adding any id back means a regression.
|
||||
static let knownEventGaps: Set<String> = []
|
||||
|
||||
@Test("Every valid case emits canonical events, except the recorded gaps")
|
||||
func test_canonicalEventConformance() throws {
|
||||
guard ProcessInfo.processInfo.environment["PUREYAML_RUN_YAML_TEST_SUITE"] == "1" else {
|
||||
return
|
||||
}
|
||||
let cases = try #require(loadValidCases(), "fetch the suite first: bash scripts/fetch-yaml-test-suite.sh")
|
||||
var observedGaps: Set<String> = []
|
||||
var compared = 0
|
||||
for testCase in cases {
|
||||
for testCase in CanonicalEventCorpus.validCases() {
|
||||
guard let yaml = testCase.input, let expected = testCase.expectedEvents else { continue }
|
||||
let events: [PureYAML.Parsing.Event]
|
||||
do {
|
||||
events = try PureYAML.Parsing.Parser().parseEvents(testCase.input)
|
||||
events = try PureYAML.Parsing.Parser().parseEvents(yaml)
|
||||
} catch {
|
||||
observedGaps.insert(testCase.id)
|
||||
continue
|
||||
}
|
||||
compared += 1
|
||||
let actual = Self.normalizeMarkers(events.map(Self.serialize).joined(separator: "\n"))
|
||||
if actual != Self.normalizeMarkers(Self.trimmingTrailingNewlines(testCase.expectedEvents)) {
|
||||
let actual = events.map(CanonicalEventCorpus.serialize).joined(separator: "\n")
|
||||
if actual != expected.trimmingTrailingNewline {
|
||||
observedGaps.insert(testCase.id)
|
||||
}
|
||||
}
|
||||
let regressions = observedGaps.subtracting(Self.knownEventGaps).sorted()
|
||||
let fixed = Self.knownEventGaps.subtracting(observedGaps).sorted()
|
||||
report("""
|
||||
Canonical event conformance: \(cases.count - observedGaps.count)/\(cases.count) valid cases emit exact canonical events
|
||||
recorded gaps: \(observedGaps.count) (\(compared) parsed, \(cases.count - compared) threw)
|
||||
""")
|
||||
#expect(regressions.isEmpty, "Canonical-event REGRESSIONS (now wrong, were correct): \(regressions.joined(separator: " "))")
|
||||
#expect(fixed.isEmpty, "Canonical-event cases now FIXED; remove from knownEventGaps: \(fixed.joined(separator: " "))")
|
||||
#expect(compared > 100, "non-vacuity: only \(compared) cases compared; the corpus did not load")
|
||||
}
|
||||
|
||||
private func loadValidCases() -> [Case]? {
|
||||
let manager = FileManager.default
|
||||
let root = ProcessInfo.processInfo.environment["PUREYAML_YAML_TEST_SUITE_DIR"]
|
||||
?? manager.currentDirectoryPath + "/.build/yaml-test-suite"
|
||||
guard let names = try? manager.contentsOfDirectory(atPath: root) else {
|
||||
return nil
|
||||
@Test("differential: dump PureYAML yeast events for files under a directory")
|
||||
func test_dumpYeast() throws {
|
||||
guard let dir = ProcessInfo.processInfo.environment["PUREYAML_YEAST_DIR"] else { return }
|
||||
let fileManager = FileManager.default
|
||||
guard let walker = fileManager.enumerator(atPath: dir) else { return }
|
||||
for case let path as String in walker where path.hasSuffix(".yaml") || path.hasSuffix(".yml") {
|
||||
let full = dir + "/" + path
|
||||
guard let yaml = try? String(contentsOfFile: full, encoding: .utf8) else { continue }
|
||||
print("@@@FILE@@@ \(path)")
|
||||
do {
|
||||
let events = try PureYAML.Parsing.Parser().parseEvents(yaml)
|
||||
print(events.map(CanonicalEventCorpus.serialize).joined(separator: "\n"))
|
||||
} catch {
|
||||
print("@@@ERROR@@@ \(error)")
|
||||
}
|
||||
print("@@@END@@@")
|
||||
}
|
||||
var cases: [Case] = []
|
||||
for name in names.sorted() {
|
||||
let dir = root + "/" + name
|
||||
guard !manager.fileExists(atPath: dir + "/error"),
|
||||
let input = try? String(contentsOfFile: dir + "/in.yaml", encoding: .utf8),
|
||||
let events = try? String(contentsOfFile: dir + "/test.event", encoding: .utf8)
|
||||
else {
|
||||
}
|
||||
|
||||
@Test("triage: dump first-line diff for each recorded gap")
|
||||
func test_triage() throws {
|
||||
guard ProcessInfo.processInfo.environment["PUREYAML_TRIAGE_EVENTS"] == "1" else { return }
|
||||
for id in Self.knownEventGaps.sorted() {
|
||||
let testCase = CanonicalEventCorpus.Case(id: id)
|
||||
guard let yaml = testCase.input, let expectedRaw = testCase.expectedEvents else { continue }
|
||||
let expected = expectedRaw.trimmingTrailingNewline.split(separator: "\n").map(String.init)
|
||||
let events: [PureYAML.Parsing.Event]
|
||||
do {
|
||||
events = try PureYAML.Parsing.Parser().parseEvents(yaml)
|
||||
} catch {
|
||||
print("\(id) THROWS: \(error)")
|
||||
continue
|
||||
}
|
||||
cases.append(Case(id: name, input: input, expectedEvents: events))
|
||||
let actual = events.map(CanonicalEventCorpus.serialize)
|
||||
var line = 0
|
||||
while line < min(expected.count, actual.count), expected[line] == actual[line] {
|
||||
line += 1
|
||||
}
|
||||
let exp = line < expected.count ? expected[line] : "(end)"
|
||||
let act = line < actual.count ? actual[line] : "(end)"
|
||||
print("\(id) @\(line): EXP[\(exp)] ACT[\(act)]")
|
||||
}
|
||||
return cases
|
||||
}
|
||||
}
|
||||
|
||||
enum CanonicalEventCorpus {
|
||||
struct Case { let id: String }
|
||||
|
||||
static func validCases() -> [Case] {
|
||||
// Every vendored case that is NOT an error case (no `error` marker).
|
||||
guard let base = Bundle.module.url(forResource: "yaml-test-suite", withExtension: nil, subdirectory: "Fixtures") else { return [] }
|
||||
let fileManager = FileManager.default
|
||||
guard let walker = fileManager.enumerator(at: base, includingPropertiesForKeys: nil) else { return [] }
|
||||
var out: [Case] = []
|
||||
for case let url as URL in walker where url.lastPathComponent == "in.yaml" {
|
||||
let dir = url.deletingLastPathComponent()
|
||||
let id = dir.path.replacingOccurrences(of: base.path + "/", with: "")
|
||||
if !fileManager.fileExists(atPath: dir.appendingPathComponent("error").path) {
|
||||
out.append(Case(id: id))
|
||||
}
|
||||
}
|
||||
return out.sorted { $0.id < $1.id }
|
||||
}
|
||||
|
||||
// MARK: - Yeast serialization
|
||||
static func url(_ id: String, _ name: String, _ ext: String?) -> URL? {
|
||||
Bundle.module.url(forResource: name, withExtension: ext, subdirectory: "Fixtures/yaml-test-suite/" + id)
|
||||
}
|
||||
|
||||
static func serialize(_ event: PureYAML.Parsing.Event) -> String {
|
||||
switch event {
|
||||
case .streamStart: "+STR"
|
||||
case .streamEnd: "-STR"
|
||||
case .documentStart: "+DOC"
|
||||
case .documentEnd: "-DOC"
|
||||
case let .documentStart(explicit, _): explicit ? "+DOC ---" : "+DOC"
|
||||
case let .documentEnd(explicit, _): explicit ? "-DOC ..." : "-DOC"
|
||||
case let .sequenceStart(anchor, tag, style, _): "+SEQ" + (style == .flow ? " []" : "") + props(anchor, tag)
|
||||
case .sequenceEnd: "-SEQ"
|
||||
case let .mappingStart(anchor, tag, style, _): "+MAP" + (style == .flow ? " {}" : "") + props(anchor, tag)
|
||||
@ -142,29 +157,26 @@ struct CanonicalEventConformanceTests {
|
||||
}
|
||||
return out
|
||||
}
|
||||
}
|
||||
|
||||
/// Drops the explicit `---`/`...` suffix from document-boundary lines, since the event API does not
|
||||
/// yet record whether a document marker was explicit. Applied to both sides for symmetry.
|
||||
static func normalizeMarkers(_ events: String) -> String {
|
||||
events
|
||||
.split(separator: "\n", omittingEmptySubsequences: false)
|
||||
.map { line in
|
||||
if line == "+DOC ---" { return "+DOC" }
|
||||
if line == "-DOC ..." { return "-DOC" }
|
||||
return String(line)
|
||||
}
|
||||
.joined(separator: "\n")
|
||||
extension CanonicalEventCorpus.Case {
|
||||
var input: String? {
|
||||
guard let url = CanonicalEventCorpus.url(id, "in", "yaml") else { return nil }
|
||||
return try? String(contentsOf: url, encoding: .utf8)
|
||||
}
|
||||
|
||||
static func trimmingTrailingNewlines(_ value: String) -> String {
|
||||
var copy = value
|
||||
var expectedEvents: String? {
|
||||
guard let url = CanonicalEventCorpus.url(id, "test", "event") else { return nil }
|
||||
return try? String(contentsOf: url, encoding: .utf8)
|
||||
}
|
||||
}
|
||||
|
||||
private extension String {
|
||||
var trimmingTrailingNewline: String {
|
||||
var copy = self
|
||||
while copy.hasSuffix("\n") {
|
||||
copy.removeLast()
|
||||
}
|
||||
return copy
|
||||
}
|
||||
|
||||
private func report(_ text: String) {
|
||||
FileHandle.standardError.write(Data("\n\(text)\n".utf8))
|
||||
}
|
||||
}
|
||||
|
||||
@ -214,8 +214,12 @@ enum CollectionCompatibilityFixtures {
|
||||
- one
|
||||
key: value
|
||||
""",
|
||||
expected: .mixedCollectionStyles(line: 2),
|
||||
expectedDescription: "mapping and sequence entries are mixed at line 2",
|
||||
expected: .malformed(
|
||||
reason: "while parsing a block collection, did not find expected '-' indicator",
|
||||
line: 1,
|
||||
column: 1,
|
||||
),
|
||||
expectedDescription: "while parsing a block collection, did not find expected '-' indicator at line 1, column 1",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@ -136,17 +136,17 @@ struct ComplexMappingKeyParsingTests {
|
||||
#expect(service?.rootMapping?["[\"a\", \"b\"]"] == nil)
|
||||
}
|
||||
|
||||
@Test("A complex key with no value is null; a bare sequence after it is extra content")
|
||||
func test_complexKeyWithoutValueIsNullThenRejectsExtraContent() {
|
||||
// `? [a, b]` with no `:` is a valid null-valued entry, so the following block sequence is a
|
||||
// second root node, rejected as extra content rather than as a missing mapping value.
|
||||
@Test("Rejects a sequence dangling after a complex key mapping")
|
||||
func test_rejectsSequenceDanglingAfterComplexKeyMapping() {
|
||||
// `? [a, b]` is a valid explicit key with an implicit empty (null) value,
|
||||
// so the mapping closes cleanly. The same-column `- invalid` then dangles
|
||||
// outside any node and is rejected at the document boundary.
|
||||
expectParseError("""
|
||||
? [a, b]
|
||||
- invalid
|
||||
""", .unexpectedToken(
|
||||
expected: "document boundary or stream end",
|
||||
actual: "blockEntry",
|
||||
line: 2,
|
||||
""", .malformed(
|
||||
reason: "while parsing a block mapping, did not find expected key",
|
||||
line: 1,
|
||||
column: 1,
|
||||
))
|
||||
}
|
||||
|
||||
@ -217,6 +217,9 @@ struct DiagnosticValidationReportTests {
|
||||
column: 12,
|
||||
reason: "missing space after ':' in mapping entry",
|
||||
))
|
||||
// The ported engine treats the colon-less, space-broken lines as one long
|
||||
// plain scalar and reaches the genuinely unterminated quote on line 11, so
|
||||
// the trailing parse diagnostic is the unterminated-quoted-string failure.
|
||||
#expect(report.diagnostics.last == .init(
|
||||
kind: .parse,
|
||||
code: "unterminatedQuotedString",
|
||||
|
||||
@ -275,3 +275,21 @@ struct DumperTests {
|
||||
#expect(dumper.dump(value) == PureYAML.dump(value))
|
||||
}
|
||||
}
|
||||
|
||||
@Suite("Dumper Line Break Escaping")
|
||||
struct DumperLineBreakEscapingTests {
|
||||
@Test("Escapes CR and LF in a CRLF pair so the value round-trips")
|
||||
func test_escapesCarriageReturnLineFeedPair() throws {
|
||||
// A CR+LF pair is a single Swift Character (extended grapheme cluster), so a
|
||||
// Character-level escape switch would emit it unescaped, and the folding
|
||||
// parser would not read it back. Each line break must escape independently.
|
||||
let value = PureYAML.Model.Value.string("a\r\nb\r\n\r\nc")
|
||||
let dumped = PureYAML.dump(value)
|
||||
|
||||
#expect(dumped == "\"a\\r\\nb\\r\\n\\r\\nc\"\n")
|
||||
#expect(!dumped.unicodeScalars.contains("\r"))
|
||||
|
||||
let reparsed = try PureYAML.parse(dumped)
|
||||
#expect(reparsed == value)
|
||||
}
|
||||
}
|
||||
|
||||
@ -43,18 +43,6 @@ struct EventStreamingParserTests {
|
||||
"error: Duplicate mapping key 'id' at $.flow.id",
|
||||
])
|
||||
}
|
||||
|
||||
@Test(
|
||||
"Lazy token cursor matches full scanner tokens",
|
||||
arguments: EventStreamingFixtures.singleDocuments + EventStreamingFixtures.streams,
|
||||
)
|
||||
func test_lazyTokenCursorMatchesFullScannerTokens(testCase: EventStreamingFixtures.DocumentCase) throws {
|
||||
let full = try PureYAML.Parsing.Scanner().scan(testCase.yaml)
|
||||
.filter { !$0.kind.isComment }
|
||||
let lazy = try collectLazyTokens(testCase.yaml)
|
||||
|
||||
#expect(lazy == full)
|
||||
}
|
||||
}
|
||||
|
||||
enum EventStreamingFixtures {
|
||||
@ -155,13 +143,3 @@ private func composeStreamWithFullEvents(_ yaml: String) throws -> [PureYAML.Str
|
||||
var composer = PureYAML.Parsing.EventComposer(events: events, scalarParser: parser)
|
||||
return try composer.composeStream()
|
||||
}
|
||||
|
||||
private func collectLazyTokens(_ yaml: String) throws -> [PureYAML.Parsing.Token] {
|
||||
var cursor = PureYAML.Parsing.TokenCursor(yaml: yaml)
|
||||
var tokens: [PureYAML.Parsing.Token] = []
|
||||
while let token = cursor.current {
|
||||
tokens.append(token)
|
||||
_ = try cursor.advance()
|
||||
}
|
||||
return tokens
|
||||
}
|
||||
|
||||
@ -6,11 +6,14 @@ import Testing
|
||||
/// line ending at `:` is instead a new mapping key, so the scalar stops before it, unchanged.
|
||||
@Suite("Explicit key multiline fold")
|
||||
struct ExplicitKeyMultilineFoldTests {
|
||||
@Test("An explicit key folds a continuation line that ends at the value colon")
|
||||
func test_continuationEndingAtColon() throws {
|
||||
#expect(try PureYAML.parse("? foo\n bar : baz\n") == .mapping(.init([
|
||||
.init(key: "foo bar", value: .string("baz")),
|
||||
])))
|
||||
@Test("An explicit key continuation ending at a value colon is rejected")
|
||||
func test_continuationEndingAtColon() {
|
||||
// libyaml rejects a `: value` on a key continuation line ("mapping values are not
|
||||
// allowed here"): the value indicator must sit at the key's own level, on a later
|
||||
// line, as the next test proves. The ported engine agrees with the reference.
|
||||
#expect(throws: PureYAML.Parsing.ParseError.self) {
|
||||
try PureYAML.parse("? foo\n bar : baz\n")
|
||||
}
|
||||
}
|
||||
|
||||
@Test("An explicit key with the colon on its own line still folds")
|
||||
|
||||
3231
Tests/Fixtures/yaml-test-suite-manifest.yaml
Normal file
3231
Tests/Fixtures/yaml-test-suite-manifest.yaml
Normal file
File diff suppressed because it is too large
Load Diff
12
Tests/Fixtures/yaml-test-suite/229Q/in.json
Normal file
12
Tests/Fixtures/yaml-test-suite/229Q/in.json
Normal file
@ -0,0 +1,12 @@
|
||||
[
|
||||
{
|
||||
"name": "Mark McGwire",
|
||||
"hr": 65,
|
||||
"avg": 0.278
|
||||
},
|
||||
{
|
||||
"name": "Sammy Sosa",
|
||||
"hr": 63,
|
||||
"avg": 0.288
|
||||
}
|
||||
]
|
||||
8
Tests/Fixtures/yaml-test-suite/229Q/in.yaml
Normal file
8
Tests/Fixtures/yaml-test-suite/229Q/in.yaml
Normal file
@ -0,0 +1,8 @@
|
||||
-
|
||||
name: Mark McGwire
|
||||
hr: 65
|
||||
avg: 0.278
|
||||
-
|
||||
name: Sammy Sosa
|
||||
hr: 63
|
||||
avg: 0.288
|
||||
22
Tests/Fixtures/yaml-test-suite/229Q/test.event
Normal file
22
Tests/Fixtures/yaml-test-suite/229Q/test.event
Normal file
@ -0,0 +1,22 @@
|
||||
+STR
|
||||
+DOC
|
||||
+SEQ
|
||||
+MAP
|
||||
=VAL :name
|
||||
=VAL :Mark McGwire
|
||||
=VAL :hr
|
||||
=VAL :65
|
||||
=VAL :avg
|
||||
=VAL :0.278
|
||||
-MAP
|
||||
+MAP
|
||||
=VAL :name
|
||||
=VAL :Sammy Sosa
|
||||
=VAL :hr
|
||||
=VAL :63
|
||||
=VAL :avg
|
||||
=VAL :0.288
|
||||
-MAP
|
||||
-SEQ
|
||||
-DOC
|
||||
-STR
|
||||
0
Tests/Fixtures/yaml-test-suite/236B/error
Normal file
0
Tests/Fixtures/yaml-test-suite/236B/error
Normal file
3
Tests/Fixtures/yaml-test-suite/236B/in.yaml
Normal file
3
Tests/Fixtures/yaml-test-suite/236B/in.yaml
Normal file
@ -0,0 +1,3 @@
|
||||
foo:
|
||||
bar
|
||||
invalid
|
||||
5
Tests/Fixtures/yaml-test-suite/236B/test.event
Normal file
5
Tests/Fixtures/yaml-test-suite/236B/test.event
Normal file
@ -0,0 +1,5 @@
|
||||
+STR
|
||||
+DOC
|
||||
+MAP
|
||||
=VAL :foo
|
||||
=VAL :bar
|
||||
18
Tests/Fixtures/yaml-test-suite/26DV/in.json
Normal file
18
Tests/Fixtures/yaml-test-suite/26DV/in.json
Normal file
@ -0,0 +1,18 @@
|
||||
{
|
||||
"top1": {
|
||||
"key1": "scalar1"
|
||||
},
|
||||
"top2": {
|
||||
"key2": "scalar2"
|
||||
},
|
||||
"top3": {
|
||||
"scalar1": "scalar3"
|
||||
},
|
||||
"top4": {
|
||||
"scalar2": "scalar4"
|
||||
},
|
||||
"top5": "scalar5",
|
||||
"top6": {
|
||||
"key6": "scalar6"
|
||||
}
|
||||
}
|
||||
12
Tests/Fixtures/yaml-test-suite/26DV/in.yaml
Normal file
12
Tests/Fixtures/yaml-test-suite/26DV/in.yaml
Normal file
@ -0,0 +1,12 @@
|
||||
"top1" :
|
||||
"key1" : &alias1 scalar1
|
||||
'top2' :
|
||||
'key2' : &alias2 scalar2
|
||||
top3: &node3
|
||||
*alias1 : scalar3
|
||||
top4:
|
||||
*alias2 : scalar4
|
||||
top5 :
|
||||
scalar5
|
||||
top6:
|
||||
&anchor6 'key6' : scalar6
|
||||
33
Tests/Fixtures/yaml-test-suite/26DV/test.event
Normal file
33
Tests/Fixtures/yaml-test-suite/26DV/test.event
Normal file
@ -0,0 +1,33 @@
|
||||
+STR
|
||||
+DOC
|
||||
+MAP
|
||||
=VAL "top1
|
||||
+MAP
|
||||
=VAL "key1
|
||||
=VAL &alias1 :scalar1
|
||||
-MAP
|
||||
=VAL 'top2
|
||||
+MAP
|
||||
=VAL 'key2
|
||||
=VAL &alias2 :scalar2
|
||||
-MAP
|
||||
=VAL :top3
|
||||
+MAP &node3
|
||||
=ALI *alias1
|
||||
=VAL :scalar3
|
||||
-MAP
|
||||
=VAL :top4
|
||||
+MAP
|
||||
=ALI *alias2
|
||||
=VAL :scalar4
|
||||
-MAP
|
||||
=VAL :top5
|
||||
=VAL :scalar5
|
||||
=VAL :top6
|
||||
+MAP
|
||||
=VAL &anchor6 'key6
|
||||
=VAL :scalar6
|
||||
-MAP
|
||||
-MAP
|
||||
-DOC
|
||||
-STR
|
||||
1
Tests/Fixtures/yaml-test-suite/27NA/in.json
Normal file
1
Tests/Fixtures/yaml-test-suite/27NA/in.json
Normal file
@ -0,0 +1 @@
|
||||
"text"
|
||||
2
Tests/Fixtures/yaml-test-suite/27NA/in.yaml
Normal file
2
Tests/Fixtures/yaml-test-suite/27NA/in.yaml
Normal file
@ -0,0 +1,2 @@
|
||||
%YAML 1.2
|
||||
--- text
|
||||
5
Tests/Fixtures/yaml-test-suite/27NA/test.event
Normal file
5
Tests/Fixtures/yaml-test-suite/27NA/test.event
Normal file
@ -0,0 +1,5 @@
|
||||
+STR
|
||||
+DOC ---
|
||||
=VAL :text
|
||||
-DOC
|
||||
-STR
|
||||
6
Tests/Fixtures/yaml-test-suite/2AUY/in.json
Normal file
6
Tests/Fixtures/yaml-test-suite/2AUY/in.json
Normal file
@ -0,0 +1,6 @@
|
||||
[
|
||||
"a",
|
||||
"b",
|
||||
42,
|
||||
"d"
|
||||
]
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user