diff --git a/CHANGELOG.md b/CHANGELOG.md index a7eb45b..836486f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,20 +12,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add `Tests/StreamingScannerPerformanceTests.swift` (issue #4), measuring `scanStep()`'s overhead over `scanAll()` on the largest real-world fixture (github-rest-api.yaml, 9.5MB) across a spread of chunk sizes, gated behind - `PUREYAML_STREAMING_PERF_MEASURE=1`. Measured, not guessed: small chunk sizes - (16-256 scalars) show a real, chunk-size-dependent rescan cost (up to ~6x at - 16), consistent with the checkpoint-and-retry design re-scanning a fetch - attempt once per chunk it spans. But a control run with `isFinal: true` from - construction, where `ranDry` can never be set and no rollback ever happens, - still showed ~1.5x overhead over `scanAll()`, proving a separate, - chunk-size-independent cost dominates at any realistic chunk size (4KB+): the - checkpoint was being copied on every fetch even when it could never be used. - Fixed by skipping the checkpoint entirely when `reader.isFinal` is true. - Re-measured: the isolated control improved from 1.49x to 1.38x, a real but - modest gain; the large-chunk-size ratios (4096+) were unchanged within - measurement noise (~1.5x). The remaining floor's cause is not identified. - Closing it needs a real profiler, not further guessing, so it is left open on - issue #4 rather than patched further on speculation. + `PUREYAML_STREAMING_PERF_MEASURE=1`. First pass ran in debug mode (release + test builds hit an unrelated `PureYAMLJSON` test-target issue) and showed a + ~1.5x floor even in a control with `isFinal: true` from construction, where + `ranDry` can never be set and no rollback ever happens; fixed by skipping the + checkpoint entirely in that case, which only closed part of the gap in debug + numbers (1.49x to 1.38x). + `sample`(1) profiling of that debug build pointed at `mutating throws` + methods on multi-field structs (`consumePlainContent`, called from every + plain scalar), which is a known trigger for defensive retain/copy traffic + under `-Onone` that the optimizer removes in release. Confirmed by working + around the release-test-build issue with `-Xswiftc -enable-testing`: in + release, the `isFinal: true` control measured **1.04x**, essentially batch + parity, meaning the checkpoint-skip fix was actually a full fix for that + case, not the small, unexplained one it looked like in debug mode. + What remains, correctly explained rather than guessed at: a chunked run's + `isFinal` only becomes true once, on the last chunk, so for any input that + arrives incrementally, the checkpoint stays live for virtually the entire + scan no matter how large the chunks are. Release-mode chunked ratios: 11.70x + at 16-scalar chunks down to 1.75x at 1,000,000-scalar chunks (10 chunks + total for this file), never reaching the 1.04x no-chunking floor. Closing + that gap needs the deferred persisted mid-token suspension, which removes + the need for a checkpoint at all rather than making it cheaper; the + checkpoint's cost cannot be optimized away further without it, since + `isFinal: true` is the only condition under which skipping it is provably + safe (a single scalar can span past any finite "probably buffered enough" + margin). - Broaden the real-seed mutation property suite (issue #6) with three mutation classes named in that issue but not yet covered: `invalidControlCharacter` diff --git a/Tests/StreamingScannerPerformanceTests.swift b/Tests/StreamingScannerPerformanceTests.swift index 49ab77a..7815e39 100644 --- a/Tests/StreamingScannerPerformanceTests.swift +++ b/Tests/StreamingScannerPerformanceTests.swift @@ -12,6 +12,17 @@ import Testing /// practice before persisted mid-token suspension (a materially larger, riskier /// change to the scalar-scanning loops) is worth building. Gated: release-mode /// timing on a 9.5MB fixture is not a default `swift test` concern. +/// +/// Run this in release mode; debug-mode numbers are not representative (Swift's +/// `-Onone` inserts defensive retain/copy traffic around `mutating throws` +/// methods on multi-field structs that the optimizer removes in release). This +/// package's own test target needs `-enable-testing` explicitly in release, +/// working around an unrelated `PureYAMLJSON` test-target issue: +/// +/// ```sh +/// PUREYAML_STREAMING_PERF_MEASURE=1 swift test -c release -Xswiftc -enable-testing \ +/// --filter StreamingScannerPerformanceTests +/// ``` @Suite("Streaming Scanner Performance") struct StreamingScannerPerformanceTests { @Test("measure scanStep() overhead over scanAll() at a spread of chunk sizes")