Reasoning & Test-Time Compute

For years, a model answered every question in a single breath — the first words came out immediately, whether the question was “what’s 2+2?” or a gnarly logic puzzle. The defining shift of 2025–26 is the reasoning model: one that stops and thinks before it speaks, working through a hard problem step by step in private before writing its final answer. This lesson is about why letting a model deliberate — spending more compute at answer time — often beats reaching for a bigger model.

Fast answers vs slow thinking

☺ Like you’re 10: Your brain has two speeds. One is the instant “7×8 is… 56!” you just know. The other is the slow “let me work it out on paper” you use for a puzzle. Both are you — but the slow one is for the hard stuff, and it takes a bit longer.

Psychologists have a famous way to describe how people think, popularized by Daniel Kahneman: System 1 is fast, automatic, and intuitive — the answer just appears. System 2 is slow, effortful, and deliberate — you consciously work through steps. You use System 1 to read a stop sign and System 2 to plan a road trip. The analogy maps almost perfectly onto the two kinds of language model you’ll meet today.

A standard model is pure System 1. When a prompt arrives, it produces its answer in one continuous pass, predicting each word of the response directly (the mechanics are in How models work). It doesn’t pause to check itself or explore alternatives — it commits to a direction from the first token and rolls forward. For most everyday requests that’s exactly right: fast, cheap, and perfectly correct.

A reasoning model adds a System 2 gear. Before it writes the answer you see, it first works through the problem — laying out the steps, trying an approach, noticing a mistake, backtracking, checking the result — and only then composes a clean final reply. It’s the same underlying next-token machinery; the difference is that the model spends a chunk of that machinery thinking before it starts answering.

What a reasoning model does

☺ Like you’re 10: Before Sol tells you the answer, he scribbles his working-out on a private notepad — “try this… no wait… okay this works.” You don’t see the messy notepad; you just get the neat final answer he wrote after thinking it through.

Under the hood, a reasoning model does something surprisingly down-to-earth: it thinks out loud to itself. Before producing the final answer, it generates a long internal stretch of text — a chain-of-thought — where it reasons through the problem in words. These are called reasoning tokens (or “thinking tokens”). The model literally writes out its scratch work: restating the problem, breaking it into parts, trying a step, catching an error, and converging on an answer.

This internal monologue is usually hidden or summarized. You typically don’t see the raw chain-of-thought verbatim — the interface shows a tidy summary of the thinking (or a collapsible “thinking…” panel) and then the final response. The reasoning is real and is what the model actually computed with; it’s just not the headline output.

Why does writing scratch work help at all? Because each token a model produces is another chance to compute. When the model has already written “the two trains are 300 km apart and closing at 150 km/h combined,” those words become part of what it reads next — so the next step builds on solid ground instead of trying to leap to the answer in one bound. Reasoning gives the model room to work, and room to catch its own mistakes before they reach you.

Test-time compute: pay with thinking, not size

☺ Like you’re 10: You can get better at a puzzle two ways: grow a bigger brain (hard and slow), or just spend more time thinking about this puzzle (easy — do it right now). Reasoning models take the second path: give the same brain more time on the tough question.

Here’s the idea that makes reasoning models matter. There are two very different moments where you can spend compute on a model:

• Training time. The expensive, one-off process of building the model — more data, more parameters, more GPUs — that bakes capability into the weights. Historically, “make the model smarter” meant “train a bigger one.” This is slow, costly, and done ahead of time.
• Test time (also called inference time). The moment the model actually answers your question. Every time you hit send, the model runs. Test-time compute is how much work it does right then to produce this one answer.

The breakthrough insight of this era: spending more compute at test time can beat using a bigger model. Instead of pouring everything into training a larger model, you let a capable model think longer on the hard question in front of it — generating more reasoning tokens, exploring more of the problem — and on genuinely difficult tasks (competition math, hard logic, multi-step planning) that extra thinking often lifts accuracy more than a bigger single-pass model would. This is sometimes called scaling inference, as opposed to only scaling training.

Neither replaces the other — you still need a well-trained model — but the second lever is new, cheap to pull, and per-question. That’s the whole reason reasoning models exist: they turn “think harder” from a training-time megaproject into a dial you can turn on any single request.

Thinking budgets & reasoning effort

☺ Like you’re 10: It’s like telling Sol how long he’s allowed to think. “Just a few seconds” for an easy one, “take all the time you need” for a stumper. More time usually means a better answer — but you wait longer for it.

If thinking is a thing you spend, then naturally you get to control how much. Reasoning models let you set a thinking budget or a reasoning effort level — a knob that says how hard the model should work before it answers. Typically this looks like a low / medium / high effort setting, or a cap on the number of reasoning tokens the model may spend.

The knob is a straight trade: more effort buys accuracy but costs latency and money. Turn it up for a hard proof or a delicate refactor; turn it down (or off, using a standard model) for a quick rewrite. It lets one model serve both a snappy autocomplete and a careful multi-step analysis, depending on how you set the dial.

This capability exists across the major providers, under different names but the same core idea. Anthropic’s Claude offers an extended thinking mode with a configurable thinking budget; OpenAI’s reasoning models (the o-series family) expose a reasoning effort setting; Google’s Gemini has a thinking capability with an adjustable budget. The vocabulary differs, but in every case you’re doing the same thing: deciding how much test-time compute this request deserves. Treat “how hard should it think?” as a first-class design choice, just like picking the model itself (see The AI landscape for choosing models by capability and cost).

When to reach for it (and when not)

☺ Like you’re 10: Slow, careful thinking is great for a hard puzzle — but silly if someone just asks your name. You don’t furrow your brow for two minutes to say “I’m fine, thanks.” Save the deep thinking for the hard stuff.

Reasoning is powerful, but it’s not free and it’s not always better. The skill is knowing which tasks deserve deliberation. As a rule: reach for a reasoning model when the problem has multiple steps that build on each other, a right answer that can be reasoned toward, or a plan that must hold together. Stick with a standard model when the task is a lookup, a reformatting, or a quick exchange where thinking adds only delay.

Notice the pattern: reasoning pays off exactly where a wrong early move poisons everything after it, and it’s wasteful where the answer is a single look-up or a shape change. This is also why agentic systems lean on reasoning — an agent that plans a sequence of tool calls benefits enormously from thinking the plan through before acting (see Agentic AI and Building agents).

You already did this by hand

☺ Like you’re 10: Grown-ups used to tell the model “show your working, step by step” to make it think harder. Reasoning models just do that on their own now — the trick got built into the toy.

If “think through it step by step” sounds familiar, it should — you met it in Prompting. For years the way to coax better reasoning out of a standard model was a prompt trick called chain-of-thought prompting: simply adding “let’s think step by step” or “show your reasoning” to the request. That instruction nudged the model into writing out intermediate steps instead of jumping to an answer — and on hard problems, accuracy jumped, because you’d manually given it room to work.

# The old manual way — coax reasoning with a prompt:
"A bat and a ball cost $1.10. The bat costs $1 more than the ball.
 How much is the ball?  Let's think step by step."

# The reasoning-model way — it deliberates on its own, no trick needed:
"A bat and a ball cost $1.10. The bat costs $1 more than the ball.
 How much is the ball?"
#  → model privately reasons through it, then answers "5 cents"

Reasoning models bake that trick into the model itself. Instead of you remembering to ask for step-by-step thinking, the model is trained to do it automatically — and to do it better, with backtracking and self-checking that a one-line prompt could never fully induce. Chain-of-thought went from a prompting technique you applied to a built-in behavior you dial up. It’s a lovely example of a pattern in this field: today’s clever prompt is tomorrow’s native capability.

Costs & caveats

☺ Like you’re 10: Slow thinking isn’t magic. It’s slower, it costs more, sometimes Sol overthinks a simple thing, and every now and then he thinks really hard and is still wrong — very confidently. So Timmy checks the answer no matter how long it took.

Reasoning is a genuine capability leap, but it comes with strings attached. Keep four caveats firmly in mind:

• Slower and pricier. All those reasoning tokens take time to generate and cost money to produce — even though you often never see them, you pay for them, and you wait for them. A reasoning answer can take many times longer and cost several times more than a standard one. For latency-sensitive or high-volume paths, that adds up fast.
• The thinking is hidden or summarized. You usually can’t inspect the raw chain-of-thought verbatim — providers show a summary or a condensed view. So you can’t always fully audit how it reached an answer, and a plausible-looking summary isn’t proof the underlying reasoning was sound.
• More thinking isn’t always better. There’s such a thing as overthinking — a model can talk itself out of a correct answer, spiral on a simple task, or burn a huge budget with no accuracy gain. Past a point, extra reasoning stops helping and just costs you. Bigger budgets have diminishing (sometimes negative) returns.
• It can still be confidently wrong. This is the big one. A long, careful-looking chain-of-thought does not guarantee a correct answer. Reasoning models still hallucinate, still make arithmetic slips, still reach fluent, well-argued, wrong conclusions — and the polished deliberation can make the error more convincing, not less.

That last point is where 🐢 Timmy the Turtle plants his flag. A model that “thinks” feels more trustworthy — the visible deliberation reads like diligence — but feeling is not verifying. Treat a reasoning model’s answer exactly like any other model output: a claim to check, not a fact to accept. For anything that matters — a number, a plan you’ll act on, a diagnosis, a legal or financial figure — verify it against reality, the same way you would a fast answer. Longer thinking earns a bit more benefit of the doubt, never a free pass. (For grounding answers in real sources, see Retrieval & RAG; for evaluating and monitoring these systems in production, see Production & Ops.)