Bad-YAML mutation suite #6

New Issue

2026-07-09T21:03:52Z

Mihaela commented

2026-07-09 21:03:52 +00:00

(Migrated from codeberg.org)

Tracks broadening PureYAML's malformed/damaged-input test fixtures beyond the single intentionally damaged production-style fixture currently used in PureYAMLYamsComparison.

Current state: both PureYAML and Yams correctly reject that one damaged fixture (0 disagreements). PureYAML's diagnostic-first APIs (parseValidated, parseValidatedStream) need broader coverage of realistic corruption patterns to validate diagnostic quality, not just pass/fail.

Remaining work:

Generate a systematic mutation suite (byte-level truncation, encoding corruption, mixed tab/space indentation, unterminated quotes/blocks) from the existing real-world corpus.
Assert diagnostic quality (correct location, useful message) for each mutation class, not just that parsing fails.

Reconstructed from PureYAML's README roadmap, which referenced this as issue #51 before the GitHub to Codeberg migration reset issue numbering on this repo.

Part of #8.

Tracks broadening PureYAML's malformed/damaged-input test fixtures beyond the single intentionally damaged production-style fixture currently used in PureYAMLYamsComparison. Current state: both PureYAML and Yams correctly reject that one damaged fixture (0 disagreements). PureYAML's diagnostic-first APIs (`parseValidated`, `parseValidatedStream`) need broader coverage of realistic corruption patterns to validate diagnostic quality, not just pass/fail. Remaining work: - Generate a systematic mutation suite (byte-level truncation, encoding corruption, mixed tab/space indentation, unterminated quotes/blocks) from the existing real-world corpus. - Assert diagnostic quality (correct location, useful message) for each mutation class, not just that parsing fails. Reconstructed from PureYAML's README roadmap, which referenced this as issue #51 before the GitHub to Codeberg migration reset issue numbering on this repo. Part of #8.

Mihaela commented

2026-07-09 23:07:44 +00:00

(Migrated from codeberg.org)

Broadened the real-seed mutation property suite with the mutation classes this issue named that weren't yet covered:

invalidControlCharacter (a raw control byte injected into a line)
mixedTabSpaceIndentation (a tab after leading spaces, distinct from the existing pure-tab case)
truncatedFlowCollection (an unterminated [a, b, c appended after the real seed sample, exercising the deep parser's malformed error rather than the preflight scanner)

Also added the diagnostic-quality assertions this issue asked for, not just pass/fail: every matching diagnostic's line must fall within the mutated document, and its column, when present, must be 1-based (some ParseError cases, like .tabIndentation, are line-only by design, so column is checked when present rather than required).

Commit: NodetteHQ/PureYAML@18fdf9b, pushed to main.

Broadened the real-seed mutation property suite with the mutation classes this issue named that weren't yet covered: - `invalidControlCharacter` (a raw control byte injected into a line) - `mixedTabSpaceIndentation` (a tab after leading spaces, distinct from the existing pure-tab case) - `truncatedFlowCollection` (an unterminated `[a, b, c` appended after the real seed sample, exercising the deep parser's `malformed` error rather than the preflight scanner) Also added the diagnostic-quality assertions this issue asked for, not just pass/fail: every matching diagnostic's `line` must fall within the mutated document, and its `column`, when present, must be 1-based (some `ParseError` cases, like `.tabIndentation`, are line-only by design, so column is checked when present rather than required). Commit: NodetteHQ/PureYAML@18fdf9b, pushed to main.

Sign in to join this conversation.

No Label

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: NodetteHQ/PureYAML#6