Do AI Agent Teams Really Work Better? Benchmarks Say Usually Not
Mostly no. Vals AI found teaming GPT-6 Sol and Claude Opus 5.5 rarely paid off for the extra tokens, though one medium-reasoning Sol setup improved notably.
Illustration made on Lumeta with GPT Image 2.5Make one like it
Usually not. On Oct. 11, 2026, Vals AI reported that AI agent teams often cost 1.8x to 5.1x more than single agents while delivering little or no measurable quality improvement, especially at maximum reasoning. For developers and companies building agent workflows, the immediate takeaway is simple: if you want better output, start by testing a strong single agent before paying for a team.
What Vals AI tested
Vals AI evaluated GPT-6 Sol and Claude Opus 5.5 on its Vibe Code Bench in both solo and team setups, each at medium and maximum reasoning effort. Across the four team-versus-solo comparisons, only one showed a statistically significant quality gain: GPT-6 Sol at medium reasoning, where the team scored 7.3 points higher. At maximum reasoning, the team setup gave neither model a meaningful advantage.
Illustration made on Lumeta with GPT Image 2.5Make one like it
Why it matters
- The research suggests multi-agent systems are often a cost decision, not a quality decision.
- If a model is already running at full reasoning effort, adding more agents may not improve the result much.
- Teams may still make sense when faster completion matters more than squeezing out slightly better output.
- For coding and research workflows with heavy token use, coordination overhead can turn into a real budget problem.
Illustration made on Lumeta with GPT Image 2.5Make one like it
More agents seemed to help speed more than quality
The wider pattern in the source points the same way. Anthropic's own tests with Opus 5.5 showed diminishing returns as more agents were added: on the knowledge base task, scores moved from 0.53 with one agent to 0.70 with 10, 0.71 with 30, and 0.74 with 100. On Lean theorem proving, the scores were 0.39, 0.66, 0.66, and 0.68. OpenAI researcher Noam Brown also said multi-agent systems mainly buy speed, not better quality, noting that four agents finished tasks about twice as fast at about twice the cost. That tradeoff appears to work better for parallel tasks such as web research or math than for tightly integrated work such as long-form writing, where coordination becomes a drag.
Should you stop using AI agent teams?
Not necessarily. The findings suggest teams are often a poor way to buy quality, but they can still be useful when speed matters and the task can be split into parallel work.
Did any team setup improve results in this research?
Yes. Vals AI found one statistically significant quality gain: GPT-6 Sol in a team setup at medium reasoning scored 7.3 points higher than the solo version.
What was the main downside of multi-agent setups?
Cost. In the Vals AI tests, teams cost between 1.8x and 5.1x more than single agents, and those extra tokens usually did not translate into much better output.
Which kinds of tasks may benefit more from multiple agents?
Based on the source, tasks that parallelize well, such as web research and math, are better candidates. Tasks that depend on a single coherent line of thought, such as writing, appear less suited to large agent swarms.
