From 9daabf32235cae64813a16dd661eb77cde25a14d Mon Sep 17 00:00:00 2001 From: DATA Date: Sat, 25 Jul 2026 15:22:48 +0200 Subject: [PATCH] Set reasoning-budget 16384; measurement shows it buys nothing here Deployed as requested, then verified. On the same problem 4096 and 16384 give the same correct answer, but 16384 uses 17417 tokens instead of 5325 (~6 min vs ~2 min) and spends the extra budget in a repetition loop ("Maybe it's a^2+b^2=c^2+7?" repeated) rather than converging. The premise for raising it -- thinking was truncated, so it needs room -- did not survive measurement. Documented; the value is left at 16384 as requested rather than reverted unilaterally. Co-Authored-By: Claude Opus 5 --- README.md | 31 ++++++++++++++++++++++--------- docker-compose.yml | 6 +++++- 2 files changed, 27 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 35c39da..091d25e 100644 --- a/README.md +++ b/README.md @@ -220,16 +220,29 @@ A client **cannot** raise it. Where `--predict` was a default anyone could overr this one is a hard server-side ceiling, so changing it means editing the compose file and restarting. -It does bite: on a moderately hard number-theory problem the thinking was guillotined -mid-derivation (`...Since $x^2 \equiv`) at exactly the budget. But the model recovered -— it produced the correct answer (empty solution set, verified by brute force) and -rebuilt the parity argument cleanly in the visible response. So 4096 truncates -reasoning without necessarily damaging the result. +### More budget did not buy better answers -Raising it does not weaken the runaway protection, since total generation is already -bounded by `--predict` / the client's `max_tokens`. For heavier math or agent -workloads, 16384 is the obvious next step; there is no measured evidence here that -4096 is actively breaking anything. +Same number-theory problem (correct answer: empty solution set, verified by brute +force), one run per setting at `temperature 0.3`: + +| budget | thinking | total tokens | wall clock | answer | +|---|---|---|---|---| +| 4096 | 4095, cut mid-derivation | 5325 | ~2 min | **correct** | +| 16384 | 16383, cut mid-**repetition-loop** | 17417 | ~6 min | **correct** | + +At 4096 the thinking is guillotined mid-derivation (`...Since $x^2 \equiv`) and the +model recovers, rebuilding the parity argument cleanly in the visible answer. At +16384 it never converges — it spends the extra ~12k tokens looping on +`Maybe it's $a^2 + b^2 = c^2 + 7$?` over and over, then answers correctly anyway. + +So the extra budget cost **3.3x the tokens and wall clock for an identical answer**, +and the degenerate loop is exactly the rambling the brake exists to cut off. The +intuition that "it was truncated, so it needs more room" did not survive measurement: +this model does not use more thinking budget productively, it fills it. + +Caveat: one problem, one sample per setting. Treat the direction as indicative, not +as a tuned value. Raising the budget does not weaken runaway protection either way, +since total generation is bounded by `--predict` / the client's `max_tokens`. ## Deploy diff --git a/docker-compose.yml b/docker-compose.yml index 1ef3758..907e2ac 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -57,8 +57,12 @@ services: # generate until the 128k window is full: ~50 min blocking the single slot. - --predict - "32768" + # Hard server-side ceiling on the thinking block — a client CANNOT raise it + # per request (verified). Note the README measurement: on the test problem + # 4096 and 16384 produced the SAME correct answer, but 16384 burned 3.3x the + # tokens looping. 4096 is defensible; revisit if real workloads disagree. - --reasoning-budget - - "4096" + - "16384" - --slot-save-path - /slots - --jinja