Ship reasoning-budget 8192 as the compromise
Third data point: 8192 also gives the correct answer, at 9041 tokens between 4096's 5325 and 16384's 17417. All three runs hit the ceiling exactly -- the model never terminates thinking on its own, it fills whatever budget it gets. Only the content changes: derivation at 4096, second-guessing at 8192, a flat repetition loop at 16384. On this problem the budget is a pure latency dial with no measured quality effect. 8192 leaves room to finish a derivation without paying for the spin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+5
-4
@@ -58,11 +58,12 @@ services:
|
||||
- --predict
|
||||
- "32768"
|
||||
# Hard server-side ceiling on the thinking block — a client CANNOT raise it
|
||||
# per request (verified). Note the README measurement: on the test problem
|
||||
# 4096 and 16384 produced the SAME correct answer, but 16384 burned 3.3x the
|
||||
# tokens looping. 4096 is defensible; revisit if real workloads disagree.
|
||||
# per request (verified). Compromise value: 4096 and 16384 gave the SAME
|
||||
# correct answer on the test problem, with 16384 burning 3.3x the tokens in
|
||||
# a repetition loop. 8192 leaves room to finish a derivation without paying
|
||||
# for that. See README for the measurements.
|
||||
- --reasoning-budget
|
||||
- "16384"
|
||||
- "8192"
|
||||
- --slot-save-path
|
||||
- /slots
|
||||
- --jinja
|
||||
|
||||
Reference in New Issue
Block a user