Procházet zdrojové kódy

Lower the decompression limit to 256 MB

bajdev chose 256 MB over the plan's 512 MB: the limit covers only the
unpacked project file, not the WAVs, and 256 MB holds about half a
million recorded events against 3 MB for a blank project. This halves
the worst-case memory. Updates the plan's Decisions table and PR 2
tasks, the test and the PR record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude před 14 hodinami
rodič
revize
ce1fd0a3ff

+ 3 - 3
docs/review-fixes-plan.md

@@ -8,7 +8,7 @@ These were settled with the user on 2026-11-12. Agents follow them and do not re
 
 | Topic | Decision |
 | --- | --- |
-| Decompression limit (S2) | 512 MB of decompressed project data. |
+| Decompression limit (S2) | 256 MB of decompressed project data (lowered from 512 MB on 2026-11-12, during PR 2: a blank project is 3 MB and 256 MB holds about half a million recorded events). |
 | Bound paths (S4) | Approved: `LoadPath` and `AssignSamplePath` stop being bound; export methods stop taking a path. |
 | Pad labels (M6) | Always the "A1" format (bank letter, pad number without a leading zero). |
 | Releases | 0.5.2 after PRs 1-3, 0.5.3 after PRs 5-8. PRs in between do not change the version. |
@@ -119,8 +119,8 @@ Each PR starts from `main` after the previous one is merged. The order matters i
 
 **Tasks**
 
-- [ ] Add `const MaxDecompressed = 512 << 20` in `internal/xpj` with a comment giving the reason.
-- [ ] In `Read`, read through `io.LimitReader(zr, MaxDecompressed+1)` and return `xpj: project is larger than 512 MB when decompressed` past the limit.
+- [ ] Add `const MaxDecompressed = 256 << 20` in `internal/xpj` with a comment giving the reason.
+- [ ] In `Read`, read through `io.LimitReader(zr, MaxDecompressed+1)` and return `xpj: project is larger than 256 MB when decompressed` past the limit.
 
 **Done when**
 

+ 3 - 2
internal/xpj/file.go

@@ -72,8 +72,9 @@ func (f *File) Encode() ([]byte, error) {
 // MaxDecompressed is the largest project Read accepts, in decompressed
 // bytes. A small file can decompress to gigabytes (a 1 MB file padded with
 // spaces used 3.1 GB), so Read stops there instead of running out of
-// memory. Device projects are about 3 MB.
-const MaxDecompressed = 512 << 20
+// memory. Device projects are about 3 MB; 256 MB holds about half a
+// million recorded events.
+const MaxDecompressed = 256 << 20
 
 // Read decompresses and decodes a project file.
 func Read(r io.Reader) (*File, error) {

+ 2 - 2
internal/xpj/xpj_test.go

@@ -414,11 +414,11 @@ func gzipPadded(t *testing.T, size int) []byte {
 
 func TestDecompressionLimit(t *testing.T) {
 	if testing.Short() {
-		t.Skip("decompresses 1 GB")
+		t.Skip("decompresses 512 MB")
 	}
 	// One byte over: refused.
 	_, err := Read(bytes.NewReader(gzipPadded(t, MaxDecompressed+1)))
-	want := "xpj: project is larger than 512 MB when decompressed"
+	want := "xpj: project is larger than 256 MB when decompressed"
 	if err == nil || err.Error() != want {
 		t.Fatalf("over the limit: got %v, want %q", err, want)
 	}

+ 15 - 11
pull-requests/limit-decompression/cap-decompressed-project-size.md

@@ -6,25 +6,29 @@ PR 2 of the review fixes plan (`docs/review-fixes-plan.md`). Part of 0.5.2; the
 
 ## What changed
 
-**Projects over 512 MB decompressed are refused (S2).** `xpj.Read` used to read the whole gzip stream into memory, however large it was. In the review, a 1 MB file padded with spaces used 3.1 GB and took 10 seconds. `Read` now reads through `io.LimitReader` and stops one byte past the new `xpj.MaxDecompressed` (512 MB, as decided). Past that, it returns `xpj: project is larger than 512 MB when decompressed`. A comment on the constant gives the reason. Device projects are about 3 MB decompressed, so real projects are far below the limit.
+**Projects over 256 MB decompressed are refused (S2).** `xpj.Read` used to read the whole gzip stream into memory, however large it was. In the review, a 1 MB file padded with spaces used 3.1 GB and took 10 seconds. `Read` now reads through `io.LimitReader` and stops one byte past the new `xpj.MaxDecompressed`, 256 MB. Past that, it returns `xpj: project is larger than 256 MB when decompressed`. A comment on the constant gives the reason.
+
+**The limit is 256 MB, not the plan's 512 MB** (bajdev, 2026-11-12, during this PR). The limit counts only the project file once it is unpacked: the pads, the pool's list of file names and the sequences with their events. WAV files are separate files in the data folder and are not read through `xpj`.
+- A blank project is about 3 MB, almost all of it the 128 empty pad definitions.
+- Each recorded event takes about 480 bytes, and an empty sequence about 12 KB.
+- So 256 MB holds about half a million events, around 4,000 for each of 128 sequences. DemoFile has 225 events in total.
+
+The Decisions table and PR 2's tasks in `docs/review-fixes-plan.md` now say 256 MB.
 
 **Test.** `TestDecompressionLimit` builds its gzip streams in memory with `gzip.NewWriter`; nothing new is committed:
 - A minimal project padded with spaces to one byte over the limit is refused with that exact error.
 - The same project padded to exactly the limit still decodes.
 - Against the old `Read`, the over-limit case loads without error, so the test fails.
-- It takes about 11 seconds, and `go test -short` skips it.
+- It takes about 3 seconds, and `go test -short` skips it.
 
 ## Notes
 
-**Memory is still high near the limit.** The limit stops the read early, but memory still grows well past 512 MB:
-- **Over the limit:** a file padded to 3.1 GB (6 MB compressed) is now refused after 1.3 seconds, at a peak of about 1.3 GB. The buffer `io.ReadAll` grows can reach roughly 2.5 times what it holds before the check runs.
-- **Exactly at the limit:** a file that passes the check still costs about 2.3 GB while it decodes, because the JSON decoder keeps its own copy of the padding.
-
-This is what the 512 MB decision implies, and it no longer grows without bound. If you want it lower, there are two options for a later PR:
-- Lower the limit. Device projects are about 3 MB, so 64 MB would still leave plenty of room.
-- Read into a buffer sized up front, which saves the growth but not the decoder's copy.
+**Memory near the limit, as measured on the server:**
+- A file padded to 3.1 GB (6 MB compressed) is now refused after 0.3 seconds, at a peak of about 600 MB. The buffer `io.ReadAll` grows can pass the limit before the check runs.
+- A file exactly at the limit still loads, at a peak of about 925 MB, because the JSON decoder keeps its own copy of the padding.
+- At 512 MB, these were about 1.3 GB and 2.3 GB.
 
-**The test needs memory.** `TestDecompressionLimit` peaks at about 2.3 GB while it decodes the at-the-limit file. The server ran it with no trouble alongside the other tests. Use `go test -short ./...` to skip it.
+**The test needs memory:** about 860 MB while it runs. `go test -short ./...` skips it.
 
 ## Testing
 
@@ -37,6 +41,6 @@ This is what the 512 MB decision implies, and it no longer grows without bound.
 
 **On the laptop (PowerShell, in the repo):**
 1. `git fetch; git checkout limit-decompression`
-2. `go test -run TestDecompressionLimit -v ./internal/xpj`: `PASS`, in about 10–20 seconds. It uses about 2.3 GB of memory.
+2. `go test -run TestDecompressionLimit -v ./internal/xpj`: `PASS`, in a few seconds. It uses about 900 MB of memory.
 3. `go test ./...`: all `ok`.
 4. `wails3 dev`: load `testdata\device\DemoFile.xpj` and `testdata\accepted\MpckitT1.xpj`. Both open as before.