Are AI Models Smarter or Just Searching More? 2026

The uncomfortable question behind every model launch: how much of the improvement is a smarter brain, and how much is a better search box bolted to the same brain?

Here is a fact that should unsettle anyone who reads model leaderboards: on the BrowseComp-Plus benchmark, you can hold the model completely fixed at GPT-5, change nothing but the retriever it searches with, and watch accuracy jump from 55.9% to 70.1%. The weights did not move. The training did not change. A different search component, feeding the same model, produced a swing larger than the gap most labs celebrate between one flagship and the next.

That single experiment is the whole debate in miniature. When a new model "scores higher," we instinctively credit the model. But a modern benchmark score is not a property of a model. It is the output of a system: the frozen weights, plus how long the model is allowed to think, plus which tools it can call, plus the scaffold that orchestrates those calls. Change any layer and the number moves. The question this guide takes seriously is whether the visible gains of 2025 and 2026 are mostly the model getting smarter, or mostly the search and tool layer getting better and getting used at far larger scale.

The short version: the hunch is more right than the marketing admits, and more wrong than a cynic would like. A large and growing share of headline improvement is search, tools, and inference-time compute, not a smarter base network. But "search" turns out to mean two very different things, and on a specific family of search-proof benchmarks, raw reasoning genuinely improved too. This guide separates the layers, quantifies each with primary sources, and gives builders a framework for where the value actually lives.

This is not a consensus piece. It reasons from first principles about what a benchmark number physically measures, then layers on data from lab model cards, arXiv papers, independent benchmark hosts, and the search economy's own financials. We cover what "better" actually measures, the two meanings of search, where the hunch is strongest, where it breaks, the real engine of the 2026 leap, why leaderboards cannot answer the question cleanly, and what all of it means if you are building on top of these models.

Contents

1. What "Better" Actually Measures
2. Two Kinds of Search: The Bitter Lesson Hiding in Plain Sight
3. The Web Search Tool Became the Default (and a Metered Product)
4. Where the Hunch Is Strongest: Tools Move the Number More Than Weights
5. Where the Hunch Breaks: Gains Search Cannot Explain
6. The Real Engine of 2026: Test-Time Compute
7. The Attribution Muddle: Why a Leaderboard Cannot Tell You
8. Read the Footnotes: How Labs Actually Report Scores
9. The Economics: Value Is Moving to the System
10. What This Means If You Are Building
11. The Verdict: A Refined Answer to the Hunch

1. What "Better" Actually Measures

Start with the structural question, not the surface one. The surface question is "did the model get smarter?" The structural question is "what does a benchmark score physically measure in 2026?" and the honest answer is that it no longer measures a model at all. It measures a pipeline. When Anthropic reports a coding score, it says so directly: the evaluation "doesn't just evaluate the AI model in isolation, but rather an entire 'agent' system," and performance "can vary significantly based on this scaffolding, even when using the same underlying AI model" - Anthropic Engineering. Read that twice. The company shipping the model is telling you the number is a property of the system, not the weights.

This matters because the intuitive mental model, that a bigger score means a smarter brain, was roughly true in the GPT-3 era and is roughly false now. In 2020 a model answered from its weights in a single forward pass, so a higher score really did mean a better network. In 2026 the same prompt triggers a chain of internal reasoning, a series of tool calls, a web search or three, code execution, and a scaffold that retries and re-ranks. The frozen weights are one input among several, and often not the one that moved. Untangling "the model improved" from "the system around it improved" is now the central measurement problem in AI, and almost no public leaderboard does it for you. Our own breakdown of how these tests are constructed, in The Best AI Agent Evals and Benchmarks, shows how few of them isolate the variable a reader assumes they are reading.

The consequence is that two different questions get collapsed into one word, "better." One question is closed-book: what can the model do from its weights alone, with no tools, no browsing, no extra thinking budget? The other is open-book: what can the whole system accomplish when it can search, execute code, and think for as long as it wants? Both are legitimate, and both improved. But they improved for different reasons, at different layers, and conflating them is how a genuine advance in tool orchestration gets sold as a genuine advance in intelligence. The rest of this guide keeps the two apart on purpose, because the founder's hunch, that search is doing more of the work than we admit, is precisely a claim about which layer moved.

A concrete example makes the split tangible. Ask a 2026 assistant for the current chief executive of a mid-sized public company and, closed-book, it may answer from stale training data and get it wrong; open-book, it searches, reads the company's own page, and answers correctly. The words on screen look equally confident either way, but only one of them reflects the model knowing anything. Now ask it to prove an unfamiliar geometry lemma. No search can help, and the answer reflects reasoning or nothing. Most real prompts sit between these poles and draw on both channels at once, which is exactly why a single "better" score blurs them. A benchmark that mixes searchable and unsearchable questions, as most do, produces a number that no amount of staring can decompose into "the model knew more" versus "the system fetched more." The only way to separate them is to design tests that deliberately allow or forbid each channel, which is what the search-proof benchmarks in Section 5 do, and it is the reason they are the most informative evidence in the whole debate.

The following diagram makes the layering explicit. Almost everything a reader calls "the model" in casual conversation is actually the outer ring, the system, and the actual weights are a smaller part of the machine than the branding implies.

2. Two Kinds of Search: The Bitter Lesson Hiding in Plain Sight

The word "search" is doing enormous and unnoticed work in the founder's hunch, and pulling it apart is the single most clarifying move in this entire analysis. There are two completely different things people mean when they say a model "searches more," and they live at different layers of the machine. The first is external search: the model calls a web search tool or a retrieval index, reads what comes back, and grounds its answer in fetched documents. The second is internal search: the model spends more of its own inference compute exploring possibilities, sampling many candidate solutions, and self-correcting before it answers. Both are search. Only one of them involves the internet.

This distinction is not a modern invention. It sits at the heart of the most cited essay in the field, Rich Sutton's "The Bitter Lesson," which argues that "the biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin" - Rich Sutton. Sutton names the two methods that scale: "The two methods that seem to scale arbitrarily in this way are search and learning." In Sutton's usage, search is not a web query. It is any procedure that spends compute at decision time to explore a space of possibilities, the way a chess engine or AlphaGo searches game trees. When people say the 2025 reasoning models "think," what they are describing is exactly this internal, compute-driven search, wrapped in natural language.

So the founder's hunch splits cleanly into two testable claims, and they have opposite answers. The narrow claim, that gains come from the external web-search tool, is partly true for factual tasks and mostly false for hard reasoning tasks. The broad claim, that gains come from "search" in Sutton's sense of spending more inference compute rather than from a fundamentally smarter base network, is strongly supported by the evidence. This is why the honest verdict is neither "yes it's all search" nor "no the model got smarter." The gains are largely search, but the search is mostly happening inside the model's own reasoning loop and inside the tool-using scaffold, not exclusively in a web query.

Keeping these two channels distinct also explains why some benchmarks are trivially helped by tools and others are not. If the answer exists on a web page, external search dominates. If the answer has to be constructed from scratch, like a novel proof or an unfamiliar puzzle, external search is useless and internal search is everything. The diagram below fixes the taxonomy the rest of the guide relies on.

The debate over whether this is real progress or a dead end is itself now a public argument between senior researchers. Turing laureate Richard Sutton, whose essay coined the framing, has been openly skeptical that simply scaling language models is the path forward, which is worth hearing in his own words rather than paraphrased.

3. The Web Search Tool Became the Default (and a Metered Product)

The first half of the founder's hunch, that the search tool got better and gets used at far larger scale, is not a matter of opinion. It is visible in the product architecture of every major lab. Between late 2024 and 2026, web search went from a novelty toggle to a first-class, metered feature that labs price as a distinct unit of value on top of tokens. OpenAI put live search into ChatGPT on October 31, 2024, so it would "automatically search the web in response to queries about recent information" and return "an AI-generated answer with sources and links" - MIT Technology Review. Anthropic followed with a web search API on May 7, 2025, pitched explicitly to "augment Claude's comprehensive knowledge with current, real-world data" - TechCrunch.

The pricing tells you how the labs think about search. It is not bundled into intelligence. It is sold separately, per query, because the labs treat "reaching live information" as a capability distinct from "the model thinking." Anthropic charges $10 per 1,000 searches on top of tokens, and its own documentation frames the tool as what gives Claude "up-to-date information beyond its knowledge cutoff" - Anthropic. Google is blunter about the mechanism: grounding with Google Search exists to "reduce model hallucinations by basing responses on real-world information," billed at $14 per 1,000 requests for the current Gemini 3 tier with 5,000 free per month - Google. Perplexity meters search as a per-request fee that scales with context size, from $5 to $14 per 1,000 requests across its Sonar tiers - Perplexity.

When a capability is priced as a separate line item and marketed around factuality and recency rather than reasoning, that is the industry telling you where it believes the value sits. The chart below shows the going rate to let a model look something up, and the fact that a market clears at these prices at all is evidence that search is treated as a product, not an afterthought.

Scale confirms the shift. Perplexity, the consumer face of AI search, processed 780 million queries in a single month by May 2025, growing more than 20% month over month, per figures its CEO disclosed publicly - JustThink AI. The labs also kept iterating the search layer itself rather than treating it as done: Anthropic shipped successive versions of the tool adding dynamic filtering and response controls plus a companion web-fetch tool, and Google ships dedicated Deep Research models that "plan and execute multi-step research across hundreds of sources" - Google. If you want the practitioner's map of this layer, our guide to the Top 10 AI Search APIs for Agents breaks down the backends by capability.

Which raises a practical question the hunch implies: if search does the work, which search backend does it best? The providers are not interchangeable. Some own an independent index, some resell one; some return agent-ready extracted content, some return raw links. The table below scores the leading agentic search and retrieval backends on the criteria that actually matter to a model calling them at scale. It is sorted by weighted final score, highest first.

The four criteria reflect what a model actually needs from a search backend: index independence and scale (30%, because an owned index resists deplatforming and lag), agent-readiness (25%, because extracted content and citations beat a list of blue links for a machine reader), adoption (25%, a proxy for reliability and real-world battle-testing), and access and pricing clarity (20%, because a metered, documented price is what lets a builder plan). Exa ranks first not because it has the biggest index (Google's is larger) but because its index was designed from the start for machines rather than humans, which is exactly the shift this whole article is about. The scores are analytical judgments backed by the cited data, and reasonable readers will weight the criteria differently.

4. Where the Hunch Is Strongest: Tools Move the Number More Than Weights

Now to the sharp end of the argument. On the tasks where the founder's hunch is correct, it is not marginally correct, it is overwhelmingly correct. Return to the opening fact and look at it structurally. On BrowseComp-Plus, researchers held the model fixed at GPT-5 and swapped only the retriever: a keyword-based BM25 retriever produced 55.9%, and a neural embedding retriever produced 70.1% while making fewer search calls. The floor case is even louder: a weak open model paired with BM25 managed just 3.86%. Roughly fourteen points of accuracy, on a browsing benchmark, came entirely from the search component with zero change to the weights. That is a larger swing than the celebrated gap between many consecutive flagship releases.

The picture below isolates the effect. The model is identical, the benchmark is identical, and the only thing that changes is the search component feeding it. The weakest bar shows that the model still matters, since a poor model with the same keyword retriever manages under 4%, but the two GPT-5 bars show that once you have a capable model, the retriever quality decides the outcome.

The same structural story dominates coding, the domain everyone points to as proof that "models got dramatically better at software." SWE-bench, the benchmark behind those claims, cannot even be run on a bare model. The model is dropped into a code repository and must execute commands and edit files, so every SWE-bench headline is by construction a model-plus-tools number. Anthropic's own reported scores use "the same simple scaffold that equips the model with solely the two tools ... a bash tool, and a file editing tool" - Anthropic. There is no tool-free version to compare against. When you read "the model scored 74.5% on SWE-bench," you are reading the output of an agent loop, not a measure of raw coding intelligence. We took this apart in detail in Why AI Coding Benchmarks Lie, and the mechanism is exactly this conflation.

The most damning evidence is what happens when you hold the model fixed and change only the harness. A 2026 paper arguing that agent comparisons must disclose their scaffold found that a single model's score swings enormously by wrapper alone: on SWE-bench Verified Mini, Claude Sonnet 4.5 moved from 68% to 34%, GPT-5 Medium from 46% to 12%, and o4-mini by nearly 48 points, purely from swapping the scaffold - arXiv. The paper's conclusion is the sentence this entire guide could hang on: harness-only variations "routinely dwarf the 2 to 4 percentage point shifts that papers report as meaningful model advances." When the retry logic and tool set move the number by thirty or forty points and a model release moves it by three, "the model got better" is not separable from "the harness got better" on a leaderboard. The chart below shows the same weights winning or losing by the scaffold alone.

Even prompt engineering, which touches neither the weights nor the tools, moves the number. On the tau2-bench telecom agent benchmark, rewriting the policy instructions into decision trees and checklists lifted GPT-5-mini's success rate from 55% to 67.5% with no model change at all - Quesma. And on factual questions specifically, retrieval does most of the heavy lifting: bare frontier models plateau near 52-56% on the no-tools SimpleQA Verified benchmark, with Gemini 2.5 Pro at 55.6% and GPT-5 at 52.3% - Google DeepMind. A vendor evaluation, which should be read as self-interested, reports that adding web search lifts SimpleQA accuracy above 90%, roughly a fifty-point gain from retrieval alone - Tavily. Whatever the exact figure, the direction is not in doubt: on knowledge tasks, the search layer, not the model's memory, determines whether the answer is right. This is the entire premise of retrieval-augmented generation, which we cover from the ground up in Retrieval Augmented Generation (RAG): the only intro you need.

So on browsing, coding, tool-agent, and factual tasks, the founder is right: the system around the model, and especially its search and retrieval layer, explains more of the measured improvement than the weights do. If the article stopped here it would be a clean confirmation of the hunch. It does not stop here, because there is a family of tasks where this whole story collapses.

5. Where the Hunch Breaks: Gains Search Cannot Explain

The cleanest way to test whether raw reasoning improved is to remove search from the equation entirely and see if scores still rise. Benchmark designers have built exactly these tests, using three mechanisms that make retrieval structurally useless, and on all three, scores climbed fast in 2025 and 2026. This is the counter-evidence the hunch has to survive, and it is substantial.

The first mechanism is holding out the answers so they are not on the web. FrontierMath, maintained by Epoch AI, is a set of unpublished research-level math problems whose private sets are never released, so there is nothing to retrieve - Epoch AI. Frontier models went from roughly 2% before the reasoning era to scores in the high tens of percent on the graded tiers. There is an important honesty caveat that cuts both ways: Epoch's protocol lets models "execute arbitrary Python code" but gives them no web search, so the gains reflect reasoning plus a code tool, which supports the broad "tools" reading while refuting the narrow "web search" one. And on the genuinely novel FrontierMath-Erdos set of unsolved problems, the very best September 2026 model solved only 2 of 68, where no prior model had solved any - Epoch AI. The frontier of true mathematical novelty is still near zero, which is its own kind of humbling data point.

The second mechanism is novel puzzles designed to defeat memorization. The ARC-AGI benchmarks forbid tools and the internet, and their history is instructive. ARC-AGI-1 was effectively solved by spending compute, not by retrieval, which we will return to in the next section. When its authors rebuilt it as ARC-AGI-2 specifically to kill the compute-brute-force shortcut, pure language models scored 0% at launch in March 2025 and a leading reasoning preview managed 4%, against a human panel that solves the tasks reliably - ARC Prize. By late 2025, verified commercial systems had climbed to 37.6% for Claude Opus 4.5 and 31% for Gemini 3 Pro, still with no tools or internet - ARC Prize. No web page contains these puzzle solutions, so that rise is reasoning, not search. The chart below shows the trajectory against the human ceiling.

The benchmark authors make the same point with their own framing, and it is worth seeing directly because it is the visual centre of the counter-argument. Their headline chart is titled "Scale is Not Enough," plotting score against cost per task to show that frontier systems stay near the floor no matter how much compute they are allowed to spend, while an ordinary human panel solves the same puzzles cheaply and reliably. If raw scaling or brute-force sampling were the whole story, that gap would not exist.

The third mechanism is auditing search out of the test directly. Humanity's Last Exam, a 3,000-question expert benchmark launched in January 2025, was explicitly built so that any question a model could answer correctly only with search was flagged and removed - Scale AI. At launch, models scored under 10%, with GPT-4o at 2.7% and o1 at 8.0%. By 2026 the official no-tools leaderboard showed Gemini 3 Pro at 38.3%, GPT-5 at 25.3%, and Grok 4 at 24.5% - CAIS / Scale. Because the searchable questions were deliberately excluded, those gains cannot be attributed to retrieval. This is the counter-argument in its purest form, and it is worth sitting with before drawing conclusions, which is why our broader look at whether progress is stalling, Has AI Hit a Wall?, reaches a similarly two-sided verdict.

The single strongest counter-datapoint is the 2025 International Mathematical Olympiad. Google DeepMind's Gemini Deep Think officially earned a gold-medal score of 35 out of 42 points, solving five of six problems, working end to end in natural language, within the same two 4.5-hour sessions as human competitors and with no internet or tools - Google DeepMind. OpenAI separately reported the same 35 out of 42 with an experimental model. The problems were brand new that year, so no search could possibly help. Whatever else is true, a system that solves fresh olympiad problems under time pressure with no tools is doing something the "it's all search" story cannot explain. The hunch, taken as a universal claim, breaks here.

6. The Real Engine of 2026: Test-Time Compute

If the gains on search-proof benchmarks are not from the internet, and not obviously from a bigger pretraining run, where do they come from? The answer is the second kind of search from Section 2: internal, inference-time compute. This is the mechanism that actually powers the 2025 and 2026 reasoning wave, and the labs say so in their own words. When OpenAI introduced its reasoning models, it stated that performance "consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute)," and, crucially, that "the constraints on scaling this approach differ substantially from those of LLM pretraining" - Simon Willison, who reproduced the quote verbatim. That is a lab explicitly saying the new gains are not the next pretraining scale-up.

You can watch the mechanism work on a fixed model. On the 2024 AIME math exam, GPT-4o solved about 12% of problems. The o1 reasoning model, with a single sample, reached 74%; with consensus across 64 samples, 83%; and re-ranking 1,000 samples with a scoring function, 93% - Simon Willison. Same weights, more search through the space of solutions, roughly twenty extra points from sampling alone. This is Sutton's search, made concrete. It is not the model knowing more. It is the model, or a wrapper around it, exploring more before committing to an answer. The chart below shows how accuracy climbs purely with inference-time effort.

Three research results turn this observation into a law. "Large Language Monkeys" showed that repeated sampling scales solve-rate over four orders of magnitude as a power law, taking a coding model from 15.9% at one sample to 56% at 250 samples on SWE-bench Lite - arXiv. A DeepMind study found that compute-optimal test-time compute "can be used to outperform a 14x larger model," meaning you can beat a bigger network by letting a smaller one think longer - arXiv. And the s1 paper demonstrated the cheapest version of the trick: a thousand curated examples plus a trivial device of appending the word "Wait" to force longer thinking pushed a model's AIME score from 50% to 57% with nothing else changed - arXiv. Anthropic confirms the same curve from the vendor side, reporting that accuracy "improves logarithmically with the number of thinking tokens" - Anthropic.

The load-bearing catch, and it matters enormously for the hunch, is that repeated sampling only converts to real accuracy when an automatic verifier can pick the correct sample. Code can be run, proofs can be checked, math answers can be graded, so sampling pays off there. For open-ended tasks with no verifier, selecting the right answer among many is an unsolved problem. This is why the reasoning wave shows up first and strongest in math and code. It is also why the deepest counter-evidence to "it's just web search" is DeepSeek-R1, peer-reviewed in Nature, which showed reasoning "can be incentivized through pure reinforcement learning, obviating the need for human-labelled reasoning trajectories." Its base variant's AIME 2024 score rose from 15.6% to 77.9% through RL with rule-based verifiable rewards and no external tools whatsoever - Nature via PMC. That is a genuine internal capability gain with zero web search involved. The relationship between reasoning cost and quality is the subject of our True Cost of LLM Inference analysis, because thinking longer is not free.

There is a unifying idea underneath all of this, and it is the bridge between the two kinds of search from Section 2. Reinforcement learning with verifiable rewards, the technique behind the recent reasoning models, works because a correct answer can be checked automatically: a proof verifies, a test suite passes, a math answer matches a key. Verification is itself a search primitive, the thing that lets a training loop or an inference-time sampler tell a good exploration from a bad one. This is why the reasoning wave landed first and hardest in domains with cheap verifiers, and why it has been slower to reach open-ended writing or judgment, where no automatic grader exists. The founder's instinct that "the model is searching more" is, at this level, exactly correct. The subtlety is that the search is a compute-driven exploration guided by verification, happening in training and at inference, not a web query fired off to Google. Calling both of them "search" is fair to Sutton and unfair to the intuition most people have when they picture a model looking something up.

The catch is that internal search is not magic either. It buys accuracy with money and time. ARC-AGI-1 is the sharpest illustration: a leading reasoning model hit 75.7% at about $26 per task, but reaching 87.5% required roughly 172 times more compute at $4,560 per task - ARC Prize. The image below plots that trade-off, and it is the clearest single picture of what "smarter" often means in practice: not a better single pass, but permission to spend far more compute per answer.

The moment this test-time-compute breakthrough was announced is itself a useful primary document, because the ARC Prize result was revealed on stage as the headline proof point, with the benchmark's own representatives present.

7. The Attribution Muddle: Why a Leaderboard Cannot Tell You

By now the core difficulty should be clear: a rising benchmark number can come from a smarter model, a better scaffold, more inference compute, or the benchmark itself leaking into training data, and a leaderboard rarely lets you tell which. This is not a minor caveat. It is the reason the founder's hunch is so easy to believe and so hard to prove. Three well-documented mechanisms inflate scores without the underlying model reasoning any better, and each has primary-source quantification.

The first is contamination. Scale AI rebuilt the grade-school math benchmark GSM8K as a held-out set called GSM1k and found accuracy drops of up to 8%, with a measurable correlation between a model's tendency to reproduce the original benchmark verbatim and the size of its performance drop - arXiv. For coding, "The SWE-Bench Illusion" showed models can identify the buggy file from an issue description alone, with no repository access, at up to 76% inside the benchmark versus 53% on outside repositories, a gap consistent with the benchmark's solutions having leaked into training - arXiv. Some of what looks like reasoning is retrieval from memorized training data, which is its own quiet form of "search."

The second mechanism is the harness, covered in Section 4, and it is the one that most directly supports the hunch. The third is eval overfitting. "The Leaderboard Illusion" documented that access to a popular human-preference arena's data can lift a model's win rate from 23.5% to 49.9%, roughly doubling it, by overfitting to the arena's distribution rather than getting broadly better, and that a few large labs tested dozens of private variants before release - arXiv. None of these three mechanisms require the model to reason better, yet all three raise the number a reader interprets as intelligence.

To be balanced, the strongest capability trend cuts the other way, and it is worth taking seriously. METR measures the length of software task an AI agent can complete at 50% reliability, and it has been doubling roughly every seven months for six years, reaching about a one-hour horizon for a leading model in early 2025 and pushing past the 16-hour ceiling of METR's own task suite by mid-2026 - METR. That is a real, large, and economically meaningful gain. But even here the attribution caveat holds: METR measures agents equipped with bash, Python, and browsing tools, and it reports that the same models score significantly differently under different scaffolds, so the metric is explicitly a model-plus-system trend, not a weights-only measurement. The image below is the flagship chart, and it is the best single visual argument that the composite system is genuinely doing more useful work over time.

The honest reading is that all of these effects are real at once. Genuine capability rose, and contamination, harness engineering, and eval overfitting also raised scores, and current public benchmarks rarely separate them. A reader who takes any single leaderboard number as a clean measure of intelligence is making an error the benchmark authors themselves warn against. For teams trying to evaluate models for real work rather than for a press release, this is why we argue for task-specific, private evals in the AI Agent Evals and Benchmarks guide rather than trusting the public scoreboards.

8. Read the Footnotes: How Labs Actually Report Scores

The most direct test of the founder's hunch is not a theory. It is the fine print under the marketed numbers, and for coding and math it is unambiguous: the headline is almost never a bare model. For coding it definitionally cannot be, because SWE-bench requires tools, as an independent host confirms in listing the exact tool set it provides, a bash tool, a file editor, and a patch tool - Epoch AI. But on top of the tools, labs stack test-time compute and disclose it only in footnotes.

Anthropic's higher SWE-bench figures, for example, are not single attempts. The company states it samples many parallel attempts, discards patches that fail the visible regression tests, and uses "an internal scoring model to select the best candidate," which lifts the plain-scaffold number of 72.5% to 79.4% for one model - Anthropic. Roughly seven points of that headline is a parallel-sampling-and-reranking pipeline, not the model thinking harder in a single pass. Google is the most transparent about this: its technical report notes that all scores are single-attempt pass@1 "unless otherwise specified," and reports one benchmark at 67.2% with "multiple attempts" and test-time selection versus 59.6% single-attempt, a 7.6-point gap disclosed in a footnote - Google DeepMind.

Math headlines were quietly tool-assisted too, then partly walked back, which is the most interesting twist for the hunch. The record AIME scores for one generation of reasoning models, 99.5% for the smaller model, were achieved "when equipped with a Python interpreter," with materially lower tool-free scores - Analytics Vidhya. That is the hunch in its purest form: a headline that is really "model plus calculator." Then GPT-5 reversed the practice, reporting a state-of-the-art AIME headline of 94.6% explicitly "no tools," while a tooled variant saturates the benchmark - Runbear. So the tool-free number genuinely rose even as the tool-on number stopped being interesting. Both facts are true, and a fair account has to hold them together.

Even the industry's flagship attempt to measure real economic value bakes tools in. OpenAI's GDPval, which scores models on tasks drawn from 44 occupations by blind expert comparison, states plainly that "for the OpenAI models, we enabled the web search tool and the code interpreter tool," with the top system reaching a 47.6% win-or-tie rate against human experts - OpenAI. We explored what that eval implies for actual jobs in Can AI Agents Do Your Job? GDPval 2026. The pattern across all of it is consistent: the marketed "smarter model" number is, for coding definitionally and for math historically, a "model plus tools plus more compute" number. That is the founder's hunch, confirmed in the labs' own methodology sections. The counterweight, GPT-5's no-tools AIME headline and Google's clearly labeled single-attempt defaults, shows the raw floor is also reported and also rising. The labs increasingly design models to want tools by default, a shift we examined in What the AI Model Wants: The Native Mode.

9. The Economics: Value Is Moving to the System

If the system around the model explains so much of the measured improvement, the money should be moving there too, and it is. This is where the abstract argument becomes a market structure, and the 2026 data is unusually clean. The clearest single proof that the harness, not the weights, decides outcomes comes from LangChain, which held the model completely fixed and moved a coding agent from 52.8% to 66.5% on a terminal benchmark, from outside the top 30 into the top 5, by changing only the system prompt, tools, and middleware - LangChain. A fourteen-point capability swing with zero model change is the empirical backbone of the now-standard investor framing that the base model is a commodity and the harness is the product.

The venture thesis has caught up to the engineering reality. Andreessen Horowitz's house view argues that base models are converging and swappable, that value accrues to the applications that orchestrate them, and warns explicitly against the "bad assumption that the apps layer will be subsumed by models," citing more than $1B of new coding-startup revenue in 2025 - a16z. Capital is flowing to the retrieval layer in particular. Exa raised a $250M round at a $2.2B valuation on the explicit thesis that AI agents will soon search the web far more than humans do, serving over 5,000 companies and 400,000 developers - Exa. Brave's independent 40-billion-page index now serves billions of API calls a week and, by its own account, powers most of the top-ten LLMs' real-time answers - Brave. Perplexity, which owns almost no frontier model and runs on others' weights, reached a $23B valuation as a pure systems-over-models business - Sacra. This is the same dynamic we described in The Big Pipe: How LLM Inference is Eating Software, where the value migrates to whoever orchestrates the intelligence.

The structural logic is straightforward once you accept that intelligence is becoming a commodity input. When an input commoditizes, margin moves to whoever combines it with something scarce, and in this market the scarce things are a good retrieval index, a well-tuned scaffold, proprietary data, and the distribution to put the system in front of users. That is why a company that runs on rented models can be worth more than some of the labs whose models it rents, and why search-infrastructure firms are raising at valuations that assume machines, not humans, will generate most search traffic within a few years. The base model is the part every competitor can call with the same API key, so by definition it is the part that stops being a moat. This is not a prediction that models will stop improving. It is an observation about where the improvement gets captured, and the answer is increasingly the layer that decides how the model is used rather than the layer that trains it.

But the economics also contain the strongest correction to an over-literal reading of the hunch, and it deserves equal weight. Search cannot substitute for reasoning. On OpenAI's BrowseComp benchmark, giving a non-reasoning model a browser raised its score only from 0.6% to 1.9%, while a reasoning-trained agent scored 51.5%, and a reasoning model with no special browsing still beat the browsing-enabled weaker model - arXiv. The paper's own conclusion is that "browsing alone is not sufficient" without strategic reasoning. Tools multiply a strong reasoner and do almost nothing for a weak one. On the GAIA assistant benchmark, even tool-equipped frontier models trailed humans badly, 15% against 92% - arXiv, and on the realistic WebArena environment the best agent reached 14.41% against a human 78.24% - arXiv. Handing a mediocre model a great search tool does not produce a great result. The defensible economic claim is that value is moving to the system, not that the base model stopped mattering.

10. What This Means If You Are Building

The practical payoff of all this is a reallocation of effort, and it is where the analysis stops being academic. If a large share of measured capability lives in the system around the model, then a team's marginal hour is usually better spent on the system than on chasing the newest flagship. The harness data makes this concrete: a fixed model gained fourteen points from harness work alone in the LangChain result, and single-model scores swing thirty to forty points on scaffold in the disclosure paper from Section 4. Those are larger deltas than a model upgrade typically buys, and they are under your control rather than a lab's release schedule. The first implication is to treat retrieval, tools, and orchestration as first-class engineering, not glue.

Concretely, that means investing where the leverage actually is, and the order matters more than any single choice.

• Retrieval quality first - the retriever, not the model, decided a fourteen-point swing on BrowseComp-Plus, so a better index and chunking strategy often beats a bigger model.
• Own your evals - public leaderboards conflate model, tools, and compute, so build task-specific tests on your own data.
• Right-size test-time compute - thinking longer buys accuracy but costs money, so spend it only where a verifier can check the result.
• Route by task - use a cheaper model with good tools for retrieval-heavy work and reserve the frontier for genuine reasoning.
• Design the scaffold deliberately - retry logic, tool selection, and context strategy move the number more than the model version does.

The retrieval-first point deserves emphasis because it inverts the usual instinct. Teams reach for the newest model when their agent underperforms, but the evidence says the retriever, the context window management, and the tool design are more often the bottleneck. Retrieval also fails in specific, knowable ways: the Stanford study of legal AI found purpose-built retrieval tools still hallucinated 17% to 33% of the time despite marketing to the contrary - Stanford RegLab, and the "lost in the middle" effect means a mis-placed retrieved document can be worse than no retrieval at all, with accuracy dropping to 53.8% when the answer sits in the middle of the context versus 75.8% at the start - arXiv. Building retrieval well is a discipline, which is why we wrote AI Search for Enterprise and Context Engineering: 4 Techniques for Agents as practitioner guides rather than overviews.

The build-versus-buy decision then comes down to whether you assemble this system yourself or adopt one that already integrates the model, the tools, the retrieval, and the orchestration. Assembling it yourself gives maximum control and is the right call when the agent loop is your core product. Adopting an integrated platform trades some control for not having to build and maintain the harness, the memory layer, and the tool connections from scratch. This is the category that platforms like o-mega occupy, providing a managed system that orchestrates models, tools, and retrieval so a team operates at the system layer without hand-rolling every component, the same way frameworks and orchestration layers do for engineering-heavy teams. It is one option among many, and the right choice depends on whether the orchestration itself is the thing you want to own. Model routing, in particular, is a discipline of its own, covered in AI Model Routing in 2026, and multi-step delegation in Multi-Agent Orchestration. The memory layer that makes retrieval useful over time is its own architecture, which we break down in AI Agent Memory in 2026.

Whichever path you choose, the meta-point holds: the model is increasingly the least differentiated part of your stack, because every competitor can call the same one. What differentiates the output is the quality of the system you wrap around it. That is not a reason to ignore model quality, since a weak model with a great harness still fails on hard reasoning, per BrowseComp. It is a reason to stop treating the model as the whole answer.

11. The Verdict: A Refined Answer to the Hunch

So, are AI models getting smarter, or just searching more? The honest verdict is that the hunch is directionally right and needs one crucial refinement. A large and growing share of the improvement we call "smarter models" is search and tools and inference-time compute, disclosed in footnotes rather than headlines. On browsing, coding, tool-agent, and factual tasks, the system around the model, and especially its retrieval layer, explains more of the measured gain than the weights do. The BrowseComp-Plus retriever swap, the SWE-bench scaffold, the harness swings that "routinely dwarf" model advances, and the labs' own tool-enabled methodology sections all point the same way. On the pure question of measured, marketed capability, search and its cousins are doing more of the work than the industry's language admits.

The refinement is that "search" mostly does not mean the web-search tool the hunch names. It means search in Sutton's sense: spending more compute at inference time to explore, sample, and self-correct, plus reinforcement learning that teaches the model to do this well. That is why raw reasoning genuinely improved on search-proof benchmarks, from an olympiad gold medal with no tools, to ARC-AGI-2 climbing from near zero toward the human range, to Humanity's Last Exam rising after its searchable questions were audited out. A model that solves fresh olympiad problems under time pressure with no internet is not merely searching the web. It is reasoning better than its predecessors, even if that reasoning is itself a form of internal search. The narrow version of the hunch, that it is all the web-search tool, is false. The broad version, that it is search rather than a fundamentally smarter base network, is largely true.

Pressure-testing the opposite conclusion is instructive. If someone claimed "none of this is real, it's all contamination and harness tricks," the IMO result, the Nature-published pure-RL reasoning gains, and the METR time-horizon trend refute them. If someone claimed "the model just got smarter, tools are a detail," the retriever swaps, the scaffold swings, and the tool-enabled headlines refute them. Reality sits between, and the interesting part is not which camp wins but what the synthesis implies: the locus of progress has shifted from the pretrained network to the system that surrounds it, and that shift is the actual story of 2025 and 2026.

Looking forward, this reframing predicts the near future better than the "smarter model" narrative does. Expect the base model to keep commoditizing while agentic scaffolds, retrieval, and test-time compute absorb more of the value and the differentiation, a trajectory the search economy's valuations already price in. Expect labs to keep reporting tool-enabled and high-compute numbers as headlines, and expect the sophisticated reader to keep asking which layer moved. The models of late 2026, from GPT-6 Astra and Claude Fable 5.1 to Gemini 3.8 Flash and DeepSeek's agent-tuned V4 line, are marketed almost entirely in agentic, long-horizon, tool-using terms, which is the industry quietly conceding the point. The most useful mental shift a reader can make is to stop asking "which model is smartest" and start asking "which system, on which task, at what cost." That question has an answer. "Is the model smarter" increasingly does not.

This guide reflects the AI landscape as of September 2026. Model names, benchmark scores, and pricing change frequently, so verify current details against primary sources before making decisions.