From e2157e9226b155a8bd16517b6ab019df5ccb070d Mon Sep 17 00:00:00 2001 From: DATA Date: Sat, 25 Jul 2026 15:32:51 +0200 Subject: [PATCH] Ship reasoning-budget 8192 as the compromise Third data point: 8192 also gives the correct answer, at 9041 tokens between 4096's 5325 and 16384's 17417. All three runs hit the ceiling exactly -- the model never terminates thinking on its own, it fills whatever budget it gets. Only the content changes: derivation at 4096, second-guessing at 8192, a flat repetition loop at 16384. On this problem the budget is a pure latency dial with no measured quality effect. 8192 leaves room to finish a derivation without paying for the spin. Co-Authored-By: Claude Opus 5 --- README.md | 22 ++++++++++++++-------- docker-compose.yml | 9 +++++---- 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 091d25e..b55da9b 100644 --- a/README.md +++ b/README.md @@ -228,17 +228,23 @@ force), one run per setting at `temperature 0.3`: | budget | thinking | total tokens | wall clock | answer | |---|---|---|---|---| | 4096 | 4095, cut mid-derivation | 5325 | ~2 min | **correct** | +| **8192 (shipped)** | 8191, cut mid-self-doubt | 9041 | ~3 min | **correct** | | 16384 | 16383, cut mid-**repetition-loop** | 17417 | ~6 min | **correct** | -At 4096 the thinking is guillotined mid-derivation (`...Since $x^2 \equiv`) and the -model recovers, rebuilding the parity argument cleanly in the visible answer. At -16384 it never converges — it spends the extra ~12k tokens looping on -`Maybe it's $a^2 + b^2 = c^2 + 7$?` over and over, then answers correctly anyway. +**The model never stops thinking on its own — it exhausts whatever budget it is +given.** All three runs hit the ceiling exactly, and all three produced the same +correct answer. What changes is only what the thinking is spent on: -So the extra budget cost **3.3x the tokens and wall clock for an identical answer**, -and the degenerate loop is exactly the rambling the brake exists to cut off. The -intuition that "it was truncated, so it needs more room" did not survive measurement: -this model does not use more thinking budget productively, it fills it. +- 4096 — guillotined mid-derivation (`...Since $x^2 \equiv`); the model recovers and + rebuilds the parity argument cleanly in the visible answer +- 8192 — already past the proof and into second-guessing (`Let's check if I missed + any "positive integers" definition`) +- 16384 — a flat repetition loop, `Maybe it's $a^2 + b^2 = c^2 + 7$?` over and over + +So on this problem the budget is a **pure latency dial with no measured quality +effect**. The intuition that "it was truncated, so it needs more room" did not +survive measurement. 8192 is shipped as a compromise: enough to finish a derivation, +short of the point where it just spins. Caveat: one problem, one sample per setting. Treat the direction as indicative, not as a tuned value. Raising the budget does not weaken runaway protection either way, diff --git a/docker-compose.yml b/docker-compose.yml index 907e2ac..862d8ca 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -58,11 +58,12 @@ services: - --predict - "32768" # Hard server-side ceiling on the thinking block — a client CANNOT raise it - # per request (verified). Note the README measurement: on the test problem - # 4096 and 16384 produced the SAME correct answer, but 16384 burned 3.3x the - # tokens looping. 4096 is defensible; revisit if real workloads disagree. + # per request (verified). Compromise value: 4096 and 16384 gave the SAME + # correct answer on the test problem, with 16384 burning 3.3x the tokens in + # a repetition loop. 8192 leaves room to finish a derivation without paying + # for that. See README for the measurements. - --reasoning-budget - - "16384" + - "8192" - --slot-save-path - /slots - --jinja