DATA/llama_qwen3.6_A3B
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Set reasoning-budget 16384; measurement shows it buys nothing here

Browse Source 
Deployed as requested, then verified. On the same problem 4096 and 16384
give the same correct answer, but 16384 uses 17417 tokens instead of 5325
(~6 min vs ~2 min) and spends the extra budget in a repetition loop
("Maybe it's a^2+b^2=c^2+7?" repeated) rather than converging.

The premise for raising it -- thinking was truncated, so it needs room --
did not survive measurement. Documented; the value is left at 16384 as
requested rather than reverted unilaterally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
DATA
2026-07-25 15:22:48 +02:00
co-authored by Claude Opus 5
parent 09230bc499
commit 9daabf3223
2 changed files with 27 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+22 -9
View File
@@ -220,16 +220,29 @@ A client **cannot** raise it. Where `--predict` was a default anyone could overr
this one is a hard server-side ceiling, so changing it means editing the compose file
and restarting.
It does bite: on a moderately hard number-theory problem the thinking was guillotined
mid-derivation (`...Since $x^2 \equiv`) at exactly the budget. But the model recovered
— it produced the correct answer (empty solution set, verified by brute force) and
rebuilt the parity argument cleanly in the visible response. So 4096 truncates
reasoning without necessarily damaging the result.
### More budget did not buy better answers
Raising it does not weaken the runaway protection, since total generation is already
bounded by `--predict` / the client's `max_tokens`. For heavier math or agent
workloads, 16384 is the obvious next step; there is no measured evidence here that
4096 is actively breaking anything.
Same number-theory problem (correct answer: empty solution set, verified by brute
force), one run per setting at `temperature 0.3`:
| budget | thinking | total tokens | wall clock | answer |
|---|---|---|---|---|
| 4096 | 4095, cut mid-derivation | 5325 | ~2 min | **correct** |
| 16384 | 16383, cut mid-**repetition-loop** | 17417 | ~6 min | **correct** |
At 4096 the thinking is guillotined mid-derivation (`...Since $x^2 \equiv`) and the
model recovers, rebuilding the parity argument cleanly in the visible answer. At
16384 it never converges — it spends the extra ~12k tokens looping on
`Maybe it's $a^2 + b^2 = c^2 + 7$?` over and over, then answers correctly anyway.
So the extra budget cost **3.3x the tokens and wall clock for an identical answer**,
and the degenerate loop is exactly the rambling the brake exists to cut off. The
intuition that "it was truncated, so it needs more room" did not survive measurement:
this model does not use more thinking budget productively, it fills it.
Caveat: one problem, one sample per setting. Treat the direction as indicative, not
as a tuned value. Raising the budget does not weaken runaway protection either way,
since total generation is bounded by `--predict` / the client's `max_tokens`.
## Deploy