More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
Even when helping costs nothing, frontier AI agents often refuse to cooperate — and smarter models aren't reliably better at it.
The Thesis
When multiple AI agents work together on a shared task, you'd expect the smarter ones to cooperate more effectively. This paper shows that expectation is wrong. OpenAI's most capable reasoning model, o3, achieved only 17% of the best possible group outcome in a coordination experiment, while the smaller o3-mini hit 50% — despite both receiving identical instructions to maximize collective results. The core finding is that 'intelligence,' as measured by standard benchmarks, does not predict cooperative behavior in multi-agent settings. This matters enormously for the wave of agentic AI products being built right now: orchestrating multiple AI agents to split up and complete complex tasks is a central premise of enterprise AI in 2025, and this paper suggests the coordination layer needs its own deliberate engineering, not just smarter base models.
Catalyst
Multi-agent AI frameworks — systems where multiple LLMs hand off work, share state, and coordinate toward a common goal — have moved from research curiosity to commercial product in the past 18 months. OpenAI, Anthropic, Google, and dozens of startups are shipping agentic pipelines premised on the idea that capable models will coordinate effectively when given good instructions. This paper arrives precisely as enterprises are committing engineering resources to those pipelines, making the failure modes newly consequential rather than merely academic.
What's New
Prior work on LLM cooperation mostly studied strategic dilemmas — situations like the prisoner's dilemma where cooperation has a real personal cost, making failure unsurprising. This paper deliberately removes that complexity: agents lose nothing by helping and gain nothing by withholding. It isolates 'zero-cost cooperation' as its own failure category. The authors also introduce a causal decomposition method that automates one side of agent communication to cleanly separate whether a failure stems from a model not wanting to cooperate versus simply not knowing how — a distinction prior benchmarks couldn't make.
The Counter
The experimental setup is artificial in ways that may not transfer to real deployments. The paper tests coordination in a 'frictionless' environment specifically designed to remove strategic complexity — but real multi-agent systems have messiness, ambiguous instructions, and competing sub-goals baked in. A model that fails in this clean lab setting might behave differently when the task structure provides richer context for why cooperation matters. The finding that o3 underperforms o3-mini is striking, but a sample of two models from one vendor is a thin basis for broad claims about 'scaling intelligence.' The authors also acknowledge they are probing a narrow slice of cooperation — zero-cost, explicit-instruction scenarios — which may tell us more about prompt sensitivity than about fundamental limits of capable models. Finally, the interventions that actually worked (explicit protocols, small sharing incentives) are straightforward engineering fixes, which suggests the problem may be solvable without new research, just better system design.
Longs
- PLTR — enterprise AI orchestration and multi-agent workflow deployments for government and commercial clients
- AI (C3.ai) — enterprise agentic AI deployments that require coordination layer engineering
- SOUN — conversational AI infrastructure that depends on reliable agent handoff behavior
- MSFT — Azure AI Foundry and Autogen multi-agent framework, though the paper's findings highlight risk as much as opportunity
Shorts
- Enterprise software vendors selling 'just plug in GPT-4' multi-agent automation — if coordination failures require deliberate protocol design, the 'drop-in' value proposition weakens
- AI agent platform startups that have built products assuming smarter base models will naturally coordinate better — the paper directly contradicts that scaling assumption
- Consulting firms pitching AI transformation programs built on multi-agent orchestration without coordination-layer engineering — their delivery risk just increased
Enablers (Picks & Shovels)
- OpenAI API — o3 and o3-mini are the specific models tested; API access is the direct substrate for this research
- LangGraph — graph-based agent orchestration library used for multi-agent state management in applied settings
- Anthropic Claude API — alternative frontier model family against which these findings should be replicated
- AgentBench and similar multi-agent evaluation harnesses — the paper's causal decomposition method could be adopted as a new evaluation standard
Private Watchlist
- LangChain — open-source orchestration framework most widely used to build the exact multi-agent pipelines this paper critiques
- CrewAI — commercial multi-agent orchestration startup whose core value proposition depends on agents coordinating reliably
- Cognition AI (Devin) — autonomous software engineering agent that chains multiple reasoning steps and agents
- Letta (formerly MemGPT) — memory and state management for long-running agents, directly relevant to coordination failures caused by missing shared context
Resources
The Paper
Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation failures may arise. In many real-world coordination problems, from knowledge sharing in organizations to code documentation, helping others carries negligible personal cost while generating substantial collective benefits. However, whether LLM agents cooperate when helping neither benefits nor harms the helper, while being given explicit instructions to do so, remains unknown. We build a multi-agent setup designed to study cooperative behavior in a frictionless environment, removing all strategic complexity from cooperation. We find that capability does not predict cooperation: OpenAI o3 achieves only 17% of optimal collective performance while OpenAI o3-mini reaches 50%, despite identical instructions to maximize group revenue. Through a causal decomposition that automates one side of agent communication, we separate cooperation failures from competence failures, tracing their origins through agent reasoning analysis. Testing targeted interventions, we find that explicit protocols double performance for low-competence models, and tiny sharing incentives improve models with weak cooperation. Our findings suggest that scaling intelligence alone will not solve coordination problems in multi-agent systems and will require deliberate cooperative design, even when helping others costs nothing.