ATHENA-TIR

Ask one model the same maths problem sixty-four times, in eight different mathematical voices, and make every answer prove itself by executing. Then pool what survives by how sure the model sounded.

Token-aware heuristic ensembler with tool-integrated reasoning, built for AIMO-3. Ported from the notebook and checked against it on every build.

The pipeline

prompts

sympy, number theory, combinatorics, enumeration, equations, direct, recursion, analysis

strategies

temperature 0.7 for breadth, 0.3 for precision

samples each

vLLM's n parameter, batched

executions

every one runs; failures filter themselves out

Every generation is Python, and every program is run. Wrong code raises or prints nothing and removes itself — so there is no regex guessing about what the answer was.

What the tokens say

vLLM returns the top-5 logprobs per token. Two things are read off them: $-\log P(t)$ for how unexpected a token was, and how much of the probability mass the chosen token took.

The model is sure at every step. All three confidence views agree.

token 0last 25% — where the answer usually is

0.960

mean confidence

0.960

geometric confidence

0.980

critical-token confidence

0.9680

evidence strength

E \;=\; \bigl(0.25\,\bar{c} \;+\; 0.35\,c_{\text{geo}} \;+\; 0.40\,c_{\text{crit}}\bigr)\,\bigl(1 - \mathrm{clip}(\sigma^2_{s}/10,\,0,\,0.3)\bigr)

The heaviest weight is on the closing tokens, not the average. A solution that rambles then commits scores above one that is smooth throughout and wavers at the end — compare the last two presets.

Pooling the survivors

Five runs from five different prompts all reach 42.

Sharpnessβ = 0.150

the notebook default — very sharp

Answer	Runs	Prompts	Mean evidence	Diversity ×	P(answer)
42MAP	5	5	0.800	1.693	1.0000

Nothing to arbitrate — one answer, maximum diversity, posterior near certainty.

Two properties worth knowing

The prior does nothing

Every answer receives the same pseudocount $\alpha/|A|$ , so it is a constant factor that cancels when the posterior is normalised.

Verified: priorStrength 1.5 and 99 give identical posteriors to 1e-12. Harmless, but the knob is inert.

Geometric mean is log pooling

$c_{\text{geo}} = \exp\!\bigl(\tfrac{1}{n}\textstyle\sum_i \log c_i\bigr)$ is a weighted geometric mean over tokens, which is logarithmic opinion pooling.

One badly unsure token therefore drags the whole solution down, where an arithmetic mean would let confident neighbours hide it. That veto is the point, not a side effect.

Qwen 3 30B A3B Instruct via vLLM, top-5 logprobs, temperatures 0.7 and 0.3, β = 0.15. Ported from athena.ipynb.