Parser throughput toward libyaml parity #5

New Issue

2026-07-09T21:03:52Z

Mihaela commented

2026-07-09 21:03:52 +00:00

(Migrated from codeberg.org)

Tracks closing the parse-throughput gap between PureYAML and Yams (libyaml-backed).

Last measured (PureYAMLYamsComparison, run 2026-06-05, PureYAML pinned to b47ca4fc31, 111-file real-world corpus):

PureYAML: 7054.098ms total
Yams: 4688.357ms total
PureYAML roughly 1.5x slower, 0 parser disagreements, both at Yams parse-success parity (110/111 each, both fail only the intentional damaged fixture)

That run predates the 0.2.0 libyaml-port engine merge (2026-07-05), which replaced the hand-rolled scanner with a line-for-line Swift port of libyaml's scanner. Current throughput relative to Yams is unmeasured.

Remaining work:

Re-run swift run compare-yaml from PureYAMLYamsComparison against current main to get a post-port baseline.
Profile the ported engine and close the gap, or document why the remaining delta is an acceptable tradeoff for a dependency-free, cross-platform parser.

Reconstructed from PureYAML's README roadmap, which referenced this as issue #48 before the GitHub to Codeberg migration reset issue numbering on this repo.

Part of #8.

Tracks closing the parse-throughput gap between PureYAML and Yams (libyaml-backed). Last measured (PureYAMLYamsComparison, run 2026-06-05, PureYAML pinned to b47ca4fc310cf44df0d0430cd9b7ea6ff59e8060, 111-file real-world corpus): - PureYAML: 7054.098ms total - Yams: 4688.357ms total - PureYAML roughly 1.5x slower, 0 parser disagreements, both at Yams parse-success parity (110/111 each, both fail only the intentional damaged fixture) That run predates the 0.2.0 libyaml-port engine merge (2026-07-05), which replaced the hand-rolled scanner with a line-for-line Swift port of libyaml's scanner. Current throughput relative to Yams is unmeasured. Remaining work: - Re-run `swift run compare-yaml` from PureYAMLYamsComparison against current main to get a post-port baseline. - Profile the ported engine and close the gap, or document why the remaining delta is an acceptable tradeoff for a dependency-free, cross-platform parser. Reconstructed from PureYAML's README roadmap, which referenced this as issue #48 before the GitHub to Codeberg migration reset issue numbering on this repo. Part of #8.

Mihaela commented

2026-07-10 00:07:06 +00:00

(Migrated from codeberg.org)

Closed out at scale (uncommitted work pending review in PureYAMLYamsComparison; will link the commit once pushed).

The private OpenAPICorpus bundle referenced in the original benchmark was unavailable on this machine, so I fetched 35 of the largest real-world OpenAPI specs directly from APIs.guru (CC0-1.0, pinned commit f04b8d0), giving a 221.6MB / 150-file / 5.2M-line corpus, 2.3x the original 95MB benchmark. The sample itself is not checked in, only the report and its provenance, matching how the original private bundle was handled.

Result (PureYAML 0.2.0 vs Yams):

PureYAML: 148 parsed, 2 failed. Yams: 146 parsed, 4 failed.
2 disagreements, both in PureYAML's favor (Yams rejects, PureYAML accepts; never the reverse). One is a tab inside a block scalar's established indentation (the same over-rejection class already fixed against the official test suite); the other is a raw/mis-encoded control character where correctness direction versus the spec has not been separately verified, so it's reported as a difference, not claimed as a win.
Release-mode parse time: PureYAML 11.933s vs Yams 12.420s, ~4% faster.

PureYAML is not just at parse-success parity with Yams at this scale, it's measurably faster and strictly more permissive (a superset of what Yams accepts) on this corpus. See PureYAMLYamsComparison's README and Reports/macos-2026-07-10-apisguru-sample/ once pushed.

Closed out at scale (uncommitted work pending review in PureYAMLYamsComparison; will link the commit once pushed). The private OpenAPICorpus bundle referenced in the original benchmark was unavailable on this machine, so I fetched 35 of the largest real-world OpenAPI specs directly from [APIs.guru](https://github.com/APIs-guru/openapi-directory) (CC0-1.0, pinned commit `f04b8d0`), giving a 221.6MB / 150-file / 5.2M-line corpus, 2.3x the original 95MB benchmark. The sample itself is not checked in, only the report and its provenance, matching how the original private bundle was handled. Result (PureYAML 0.2.0 vs Yams): - PureYAML: 148 parsed, 2 failed. Yams: 146 parsed, 4 failed. - 2 disagreements, both in PureYAML's favor (Yams rejects, PureYAML accepts; never the reverse). One is a tab inside a block scalar's established indentation (the same over-rejection class already fixed against the official test suite); the other is a raw/mis-encoded control character where correctness direction versus the spec has not been separately verified, so it's reported as a difference, not claimed as a win. - Release-mode parse time: PureYAML 11.933s vs Yams 12.420s, ~4% faster. PureYAML is not just at parse-success parity with Yams at this scale, it's measurably faster and strictly more permissive (a superset of what Yams accepts) on this corpus. See PureYAMLYamsComparison's README and `Reports/macos-2026-07-10-apisguru-sample/` once pushed.

Sign in to join this conversation.

No Label

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: NodetteHQ/PureYAML#5