diff --git a/CHANGELOG.md b/CHANGELOG.md index 836486f..cf26f05 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,18 +26,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 release, the `isFinal: true` control measured **1.04x**, essentially batch parity, meaning the checkpoint-skip fix was actually a full fix for that case, not the small, unexplained one it looked like in debug mode. - What remains, correctly explained rather than guessed at: a chunked run's - `isFinal` only becomes true once, on the last chunk, so for any input that - arrives incrementally, the checkpoint stays live for virtually the entire - scan no matter how large the chunks are. Release-mode chunked ratios: 11.70x - at 16-scalar chunks down to 1.75x at 1,000,000-scalar chunks (10 chunks - total for this file), never reaching the 1.04x no-chunking floor. Closing - that gap needs the deferred persisted mid-token suspension, which removes - the need for a checkpoint at all rather than making it cheaper; the - checkpoint's cost cannot be optimized away further without it, since - `isFinal: true` is the only condition under which skipping it is provably - safe (a single scalar can span past any finite "probably buffered enough" - margin). + What remains splits into two distinct costs, decomposed by one more + measurement rather than assumed: feeding the whole 9.5MB file as a single + chunk (`isFinal` stays false until the true end, so at most one token in the + entire file could ever actually be rescanned) still measured 1.75x, the same + as 1,000,000-scalar chunks (1.74x, ~10 chunks) and 65536-scalar chunks + (1.74x). Real rescanning is not the floor's cause, since there is + essentially no rescanning to have caused it here. The floor is the cost of + copying the whole `Scanner` on every `fetchNextToken()` call whenever + `isFinal` is false, whether or not that copy is ever used to roll back; + since a chunked run's `isFinal` only becomes true once, on the last chunk, + that copy is taken for virtually every token in the file. Only *shrinking* + chunk sizes below roughly 4096 scalars adds a second, genuinely + rescan-driven cost on top of that floor (11.54x at 16 scalars). + This corrects the previous entry's conclusion: the deferred persisted + mid-token suspension would eliminate the rescan-driven tail at small chunk + sizes, but would not touch the 1.75x floor, since that floor is not caused + by rescanning. Closing the floor needs a different fix, not yet designed: + something that avoids copying the whole scanner on every fetch while + `isFinal` is false, without weakening the correctness guarantee that only + `isFinal: true` currently provides (every other field the checkpoint covers + can be mutated before a dry point is reached within a single fetch, so no + smaller subset of fields is obviously safe to skip either). - Broaden the real-seed mutation property suite (issue #6) with three mutation classes named in that issue but not yet covered: `invalidControlCharacter` diff --git a/Tests/StreamingScannerPerformanceTests.swift b/Tests/StreamingScannerPerformanceTests.swift index 7815e39..99aa232 100644 --- a/Tests/StreamingScannerPerformanceTests.swift +++ b/Tests/StreamingScannerPerformanceTests.swift @@ -4,14 +4,15 @@ import Testing /// Measures the resumable scanner's checkpoint-and-retry cost (issue #4) against /// the batch scanner, across a spread of chunk sizes, over the largest real-world -/// fixture. `scanStep()`'s first increment restores the whole scanner and retries -/// a fetch from its start whenever it runs dry, so a token that spans many small -/// chunks is rescanned once per chunk it spans: correct (proven by -/// `StreamingScannerDifferentialTests`) but worst-case quadratic in the number of -/// chunks a single token spans. This measures whether that cost matters in -/// practice before persisted mid-token suspension (a materially larger, riskier -/// change to the scalar-scanning loops) is worth building. Gated: release-mode -/// timing on a 9.5MB fixture is not a default `swift test` concern. +/// fixture, plus a `singleChunk` control that isolates the floor from real +/// rescanning (see CHANGELOG.md). The overhead splits into two independent +/// costs: a fixed floor (~1.75x release-mode, present even with a single chunk, +/// caused by copying the whole `Scanner` on every fetch whenever `isFinal` is +/// false) and a rescan-driven tail that only appears below roughly 4096-scalar +/// chunks. Persisted mid-token suspension, the deferred next increment, would +/// remove the rescan tail but not the floor; a fix for the floor is not yet +/// designed. Gated: release-mode timing on a 9.5MB fixture is not a default +/// `swift test` concern. /// /// Run this in release mode; debug-mode numbers are not representative (Swift's /// `-Onone` inserts defensive retain/copy traffic around `mutating throws` @@ -70,6 +71,18 @@ struct StreamingScannerPerformanceTests { let ratio = duration / batchDuration print("STREAMING-PERF chunkSize=\(chunkSize) time=\(duration) ratio=\(String(format: "%.2f", ratio))x") } + + // Distinguishes two candidate explanations for the chunked-mode floor: (a) + // real rescanning at chunk boundaries, which should vanish as chunk count + // drops toward 1, or (b) a fixed per-fetchNextToken() cost paid on every + // token regardless of how many chunks exist, which would not. A single + // chunk covering the whole file means `isFinal` is false (so the checkpoint + // is taken every fetch) but `ranDry` can only ever fire once, at the true + // end: the only possible rescan is the very last token. + let wholeFileAsOneChunk = yaml.unicodeScalars.count + 1 + let singleChunkDuration = try streamingDuration(yaml: yaml, chunkSize: wholeFileAsOneChunk, clock: clock) + let singleChunkRatio = singleChunkDuration / batchDuration + print("STREAMING-PERF singleChunk(wholeFile) time=\(singleChunkDuration) ratio=\(String(format: "%.2f", singleChunkRatio))x") } }