PathweysAI Engineer Track · Working With Llms
Completion vs Chat vs Reasoning Models
Completion vs Chat vs Reasoning Models
Models are offered through a few distinct modes, and picking the wrong one wastes money or breaks expectations. Plain completion mode simply continues a block of text; chat mode structures a conversation into roles across multiple turns; and reasoning models go a step further, spending extra hidden effort working through a problem step by step before answering, at meaningfully higher cost and slower response time.
Why this matters
That extra reasoning effort is often the difference between a wrong and a right answer on hard, multi-step problems, but it is pure waste on simple, high-volume tasks. Routing every request through the most expensive, slowest mode by default is a common and genuinely costly mistake.
Completion vs chat vs reasoning models
Completion mode is the most primitive: the model is given raw text and continues it, with no concept of roles or turns, which makes it best suited for plain text continuation rather than interactive use. Chat mode is message-based, built around system, user, and assistant roles, and the model is trained specifically to treat the input as a conversation and respond in turn, which is why it is the default mode behind almost every interactive product. Reasoning models are trained to produce a long internal chain of thought before committing to an answer, which measurably improves performance on math, logic, and multi-step problems at the cost of noticeably higher latency and token spend.
Conflating these three modes is a common source of real bugs, not just a conceptual nitpick, because the right choice depends entirely on the shape of the task in front of you rather than on habit.
The model is not one single thing with one single behavior. Pick the mode that matches the task, and do not pay reasoning-model prices for problems a chat model would have solved just as well.
When to use each
The choice between modes is a cost, quality, and latency tradeoff, not a matter of which one is generally "better." Reasoning models earn their cost on multi-step math, code that genuinely has to be correct, planning, and any task where a wrong first step cascades into everything after it; they think before answering, so both latency and token cost go up accordingly. Chat models are the right default for the large majority of work: summarization, extraction, drafting, classification, and tool use, where the answer is mostly a matter of following instructions rather than working through long internal reasoning. Completion mode stays niche, useful mainly for fill-in-the-middle code completion inside an editor or other raw-continuation use cases rather than interactive responses.
A useful heuristic is to ask whether a human would need to pause and work something out on paper before answering. If so, reach for a reasoning model. If a human could answer instantly just from reading the text in front of them, a chat model is not just sufficient, it is cheaper and faster for the exact same quality of answer.
Reasoning is a feature you pay for on a per-task basis, not a universal upgrade to apply everywhere. Route requests by how difficult the task actually is, not by default.
Worked example
Sending twenty tasks of varying difficulty, from trivial classification up through multi-step math, through both a fast chat model and a reasoning model typically produces a clear pattern: accuracy is roughly tied on the easy tasks while the reasoning model costs several times more, and the gap in accuracy only opens up in the reasoning model's favor once the task genuinely requires multiple dependent steps. That crossover point, not a blanket preference for one mode over the other, is what a sensible routing rule should be built around.
Practice
Take twenty tasks that vary in difficulty, from trivial classification through multi-step math to open-ended research questions. Run each one through both a fast chat model and a reasoning model, and build a table of tokens, latency, and accuracy per task class. Finish with a written decision rule for which classes of question actually justify paying for the reasoning model.
Interview checks
- Why would a reasoning model ever perform worse than a cheaper chat model on the same task?
- Is it ever correct to use 'think step by step' phrasing with a reasoning model?
- How should token cost factor into choosing between these modes for a high-volume endpoint?
Common failure modes
- Routing every request through a reasoning model by default, incurring ten times or more the cost and several seconds of added latency for tasks that did not need it.
- Prompting a reasoning model with explicit 'think step by step' instructions, which is redundant at best since the model already reasons internally, and can actively interfere at worst.
- Not capping the model's thinking budget, so a single hard question can silently burn an enormous number of thinking tokens.
- Confusing hidden thinking tokens, which are typically billed but not shown, with the visible output tokens the user actually sees.
Resources
- OpenAI o1 System Card — Primary documentation of the o1 reasoning paradigm, its capabilities, and its safety evaluations.
- Anthropic: Extended Thinking — The official reference for Claude's reasoning mode, thinking budgets, and how thinking tokens are billed.
Expert signals
Knowing that reasoning tokens are usually hidden and billed separately from visible output, that reasoning models generally want simple, direct instructions rather than chain-of-thought scaffolding, and being able to cite roughly where the latency and cost crossover sits relative to a larger, faster chat model is what separates informed routing from a blanket policy of always using the strongest available model.
Confidence
medium