The Second Bitter Lesson

Don’t Build Tomorrow’s AI System Around Today’s Model Weaknesses

Imagine nineteenth-century pioneers refusing to move west until automobiles and interstate highways were invented. They would have waited generations for better tools.

But the opposite mistake would have been foolish too: designing every road, settlement, and supply system as if the limitations of wagon wheels and horses were permanent.

The sensible path was to move with the tools available while building things that would remain valuable when transportation improved, including maps, supply lines, settlements, institutions, and knowledge of the terrain.

AI users face the same choice now.

Today’s models are powerful, uneven, and changing rapidly. When an agent forgets something, misreads an instruction, or fails during a long task, the natural response is to add another rule, checklist, approval, or layer of supervision.

Some of that work is necessary. Some of it is permanent infrastructure built around the limits of horses and wagon wheels.

From the Bitter Lesson to Ours

In 2019, reinforcement-learning pioneer Rich Sutton published “The Bitter Lesson” . Looking across decades of AI research, Sutton argued that general methods capable of using increasing computation, especially search and learning, ultimately outperform systems built around human-supplied domain knowledge.

Encoding our expertise often helps in the short term. It is also satisfying: we get to tell the machine what intelligence should look like. But handcrafted methods tend to plateau, while general methods keep improving with more computation, data, and learning.

Sutton was writing about AI research. AI users and organizations face a second-order version of the same lesson:

We are tempted to encode the weaknesses of today’s models into permanent rules, workflows, and institutions.

The rule helps now. The model improves later. The rule remains.

This is not a theoretical concern. Research on scaling laws found improvements from combinations of model size, data, and training compute. DeepMind’s Chinchilla work showed why parameters alone are an inadequate measure: a smaller model trained on much more data outperformed a far larger one at the same compute. METR’s task-horizon research has measured rapid gains in the length of software tasks frontier agents can complete, though the authors caution that the results may not generalize to every kind of work.

No one knows the exact rate or direction of future progress. It is reasonable to expect that many limitations around which we build today will not remain fixed.

Model-Compensation Debt

Software teams already understand technical debt. AI systems can accumulate a related burden. I call it model-compensation debt.

Model-compensation debt is complexity created to compensate for a limitation that later models may no longer possess.

Imagine a manager whose new employee repeatedly leaves critical deadlines out of client reports. One missed date can cost a contract or damage years of trust. The manager adds a checklist, a mandatory template, and two review steps. Months later, the employee has mastered the task, but every report still passes through the entire apparatus. The organization is no longer preventing a live failure. It is preserving the memory of an old one.

AI systems develop the same institutional scar tissue. A model misses a step, so we add a rule. It applies the rule too broadly, so we add an exception. Soon a stronger agent is navigating instructions generated by models no longer in use. The old protections now consume context, attention, time, tokens, and maintenance.

That does not mean we should stop building and wait. Temporary scaffolding can unlock useful work today. A short-lived workaround can still produce a durable product, relationship, or lesson. The lifespan of the tool is not the same as the lifespan of the value it creates.

The debt appears when scaffolding quietly becomes permanent architecture, with no reason to retest or remove it.

What Is Worth Building?

Some investments substitute for intelligence; others complement it. A substitute compensates for something the current model cannot do. A complement makes any capable model more useful in the real world.

Four complements are especially durable.

Direction and evaluation

A rescue aircraft can fly farther and faster than the one it replaces. It still needs coordinates, mission priorities, and a definition of safe return. A more intelligent agent still needs to know what outcome matters and how success will be judged.

Detailed reasoning instructions may expire: “Think through these seven steps in this exact order.” An evaluation is more durable: “The customer must receive the correct policy before the deadline and leave knowing exactly what to do next.”

OpenAI’s evaluation guidance identifies evals as important when changing models. Research on AI agent evaluation also warns that accuracy alone can hide cost, fragility, and overfitting.

Goals, outcome tests, and real-world feedback gain value as models improve because stronger agents can extract more from them.

Tools and current context

A brilliant surgeon without instruments, records, or access to the operating room cannot perform surgery. Knowledge alone cannot save the patient.

An agent may understand the job but lack reliable access to the files, applications, or databases where the work exists. A future model may know more about the world, but it will not automatically know which draft is current, what your organization decided yesterday, or what promise you made to a customer.

Reliable tools, interfaces, provenance, and current organizational truth are the environment in which reasoning becomes useful action.

Authority and responsibility

A talented executive is not automatically authorized to sign every contract, transfer any amount of money, or issue a public statement in the organization’s name.

A rule created because a model is easily confused may expire. A boundary created because an action is private, costly, irreversible, or legally consequential rests on a different foundation.

Capability answers, “Can the agent do this?” Authority answers, “Who is responsible for deciding that this should be done?”

Better models may justify broader delegation. They do not erase responsibility.

Replaceable systems and real-world assets

Power grids remain useful through generations of furnaces, factory equipment, medical devices, and computers because the system does not have to be rebuilt around every new machine. Good AI architecture should do the same. Models, prompts, tools, and memory systems should be replaceable without rebuilding the organization around them.

Products, relationships, distribution, proprietary data, tested knowledge, and trust endure for another reason: a better printing press does not create an audience or something worth printing.

Waiting for stronger AI can therefore be expensive. You may eventually receive a remarkable tool while having built no territory in which to use it.

Will Your System Capture the Upgrade?

A better model does not automatically create a proportionally better system.

Early factories did not receive the full benefit of electricity merely by replacing a steam engine with an electric motor. Their layouts still revolved around belts, shafts, and centralized power. Larger gains required factories to reorganize around distributed electric motors. Economist Paul David used this history in “The Dynamo and the Computer” to explain why general-purpose technologies can take time to produce visible productivity gains.

Research on the “Productivity J-Curve” makes a related argument: general-purpose technologies need complementary investments before their benefits are fully realized.

AI systems face the same problem. Put a stronger model inside an organization designed around a weaker one, and obsolete scripts and redundant reviews can absorb much of the improvement.

Call this capability capture: the degree to which a system converts improvements in its underlying model into better real-world outcomes. Clear goals, useful feedback, reliable tools, current context, and replaceable components increase it. Yesterday’s compensations reduce it when they become permanent.

Continuous Improvement Must Learn to Subtract

Most improvement systems are additive:

Failure → new rule
Friction → new procedure
Mistake → new check

That feels responsible because every failure leaves behind a visible response. But instruction accumulation is not the same as learning.

A model-aware loop asks:

1. Is this a durable requirement or a temporary model limitation?
2. What is the smallest effective response?
3. What evidence will tell us whether it works?
4. When will we retest whether it is still needed?

For most AI users, this loop does not change the model’s underlying weights. It improves the context, tools, evaluations, feedback, and operating environment through which the model works.

Sometimes the result should be fewer instructions, checks, and handoffs. Preserve the evidence from a failure, but do not grant its rule permanent life.

Agents doing the work will often see conflicting instructions, failing tools, and missing context first. That makes the acting agent the best sensor and often the best generator of improvement proposals. It does not make the agent the final judge of every consequence. A rare safety check may feel like ordinary friction. A locally helpful change may expose someone else to financial, legal, or reputational harm.

Give agents proportional authority: let them make narrow, reversible, locally owned improvements; require evidence for shared changes; independently evaluate changes that affect other people or systems; preserve human authority over values and high-consequence actions.

The agent is the primary sensor, not the sovereign judge.

An Agent Organization as a Test Case

An agent organization makes this tension visible. Each failure creates an opportunity to improve the organization and a temptation to add another permanent instruction.

Project truth, reliable tools, outcome evaluations, owner boundaries, and reusable knowledge should survive model generations. Detailed prompts for current reasoning weaknesses, repeated reminders, and some coordination layers may not.

We should not declare any specific control obsolete without testing it. We should be able to explain why it exists.

The test is simple:

If a 10x more capable agent arrived tomorrow, would this part of the system make it immediately productive, or constrain it around yesterday’s weakness?

Before spending substantial effort on an AI-system improvement, ask:

1. Will better models probably solve this limitation?
2. Is it blocking valuable work now?

Then take one existing rule, review step, or agent instruction. Name the failure it was created to prevent. If that failure has disappeared under the current model, retest the control instead of assuming it deserves permanent life.

Build for the Agent You Will Have

The Bitter Lesson does not tell us to stop shaping AI systems. It tells us to be humble about what we build into them.

Do not wait passively for better models. Do not turn every current weakness into permanent architecture either.

Build the goals, tools, feedback, context, evaluations, relationships, and boundaries that a more capable agent will still need. Use temporary scaffolding where today’s limitations block valuable work. Make it visible, testable, and removable.

Do not build a larger cage for the agent you have today.

Build the world a smarter agent can act within tomorrow.