One Giant Agent vs 100 Small Agents

Two years ago, shipping an agent meant weeks of prompt glue and brittle tool loops. Now it often means an afternoon and an API key. The marginal cost of another agent is collapsing toward zero.

So the interesting question is no longer: should we build an agent?

It is: how many agents should a system have?

One camp says software will look like a single extremely capable AI employee. The other says it will look like a company—hundreds of tiny workers, each owning one job, collaborating through shared state.

Both sound plausible. Only one survives contact with production systems that have to stay up, stay cheap, and keep evolving.

Camp 1 — One Giant Agent

Build one central reasoner that owns planning, tool use, memory, and execution. Everything funnels through a single loop.

LangChain and LangGraph are the clearest representatives of this direction—not because they forbid multi-agent designs, but because their default gravity is centralized orchestration. One primary reasoning cycle coordinates the workflow. Branches exist. The brain stays singular.

That design wins on operational clarity. One process, one transcript, one place to look when something breaks. No peer handoffs. Observability collapses to a single graph. Growing context windows make the bet feel safer every quarter—if the model can hold more of the world, why split the world?

Fewer moving parts also means fewer places for your glue to fail. For a personal assistant or a thin internal tool, that trade is usually correct.

The downsides appear when the agent becomes a product.

Context does not just grow—it accumulates conflicting instructions, stale tool docs, and half-finished plans. Responsibilities interfere: the same model must be careful about auth, creative about copy, and aggressive about finishing. Every step pays for the entire context pile, even when only a narrow slice matters.

You cannot upgrade “code review quality” without risking the planner. One confused plan degrades the whole run. Large prompts stop being code you can reason about and start being folklore you hope the model interprets the same way tomorrow.

A monolith of reasoning fails the same way a monolith of services fails: easy until change velocity and blast radius collide.

Camp 2 — 100 Small Agents

Stop asking one model to be an organization. Build an organization out of models.

Compose many small agents. Each owns one responsibility—planner, researcher, memory curator, tool executor, reviewer, critic, router, code generator, evaluator. System intelligence is not a single forward pass. It is a protocol.

Wolbarg is a useful example of this pattern: specialized agents sharing memory and collaborating, rather than one loop absorbing every role. Same treatment as LangGraph above—an architectural shape, not a product pitch.

The advantages are the ones microservices promised—and sometimes delivered. Small prompts. Lower inference cost. Hard specialization. Independent upgrades. Real unit tests against a role. Fault isolation. Parallel execution. Cheaper experiments, because you swap one worker instead of the whole brain.

Then you hit the part demos never show.

Routing is hard. Shared state is hard. Memory consistency is hard. Who may write a fact? Who wins when two agents disagree? Messages, events, a shared store—or all three? Every handoff adds latency. Failures travel through dependency graphs you did not draw carefully enough. Distributed debugging turns a bad answer into forensics.

The glue becomes the product. Multi-agent systems do not fail because the models are dumb. They fail because coordination was an afterthought.

The Real Bottleneck

This debate is not about model intelligence.

Models improve. Context windows grow. Scaffolding gets cheaper. Intelligence is no longer the scarce resource for most teams.

Coordination is.

Scheduling. Routing. Shared memory that is not a junk drawer. Communication contracts that survive a rewrite. Observability across roles. Recovery when research succeeded, planning half-failed, and execution already mutated state.

The monolith-to-microservices analogy helps, then breaks. The pressure is familiar: independent evolution, blast radius, ownership. It breaks because agents are non-deterministic. You cannot “just add retries” the way you do with a flaky HTTP client. A wrong handoff is not a 500. It is confident nonsense traveling downstream.

Planner–executor and generator–critic setups already proved the weaker form: decomposition can raise task quality while raising orchestration cost. The question is not whether decomposition helps. It is whether the coordination layer is serious enough to collect the gains.

Building agents keeps getting easier.

Building systems of agents does not.

Prediction

Here is the prediction, without hedging.

By roughly 2027, complex production automation—delivery pipelines, ops runbooks, research-to-report workflows, enterprise back-office agents—will be dominated by specialized multi-agent architectures. Lightweight personal assistants and single-task tools will stay mostly single-agent. Those workloads do not need an org chart.

The force behind that split is change velocity—the same force that killed the all-in-one application server for large products.

Bigger context windows help the giant agent. They do not cancel interference, cost, or the inability to ship one capability without risking another. Larger windows delay the pain. They do not remove the need to specialize, isolate, and evolve parts independently.

Meanwhile, shared memory, routing, eventing, traces, and checkpoints are becoming infrastructure instead of custom glue. Once coordination is a platform concern, the swarm stops looking like a research demo and starts looking like ordinary distributed software with probabilistic workers.

The industry does not converge on one giant general-purpose agent. It converges on agent systems that look like software systems: small units, explicit contracts, shared state under policy, and traces that explain a failure across roles.

Consumer chat stays one mind.

Production automation becomes a workforce.

Closing

The next bottleneck is not a smarter model. It is whether a hundred narrow minds can cooperate without the glue becoming the outage.

Building one smart agent is difficult.

Building one hundred that cooperate is engineering—and that is the architecture that will own serious automation.