Perhaps Another Bitter Lesson

This English version was machine-translated from the Chinese original. Technical terms may require verification.

A few things I’ve seen over the past several days have brought The Bitter Lesson back to mind.

On September 8, OpenAI published a proposed proof resolving the Navier–Stokes problem: roughly ten thousand concurrent agents took part in the search, producing a construction of a finite-time singularity under external forcing. An internal research model produced the proof; GPT-6 Astra handled the subsequent Lean formalization and verification. The scale does look a little like brute force, with agents exploring different approaches, exchanging findings, and carrying out verification.

Meanwhile, Astra can model a house in Blender and bring it into Unreal Engine. Robocurve gave it control of robot arms, and it placed a block into a bowl in 19 out of 20 trials. Image input and tool calling let a general-purpose model use tools people have already built. Of course, it completed the puzzle insertion task only 2 out of 20 times. These demonstrations don’t establish that it broadly outperforms specialized 3D models or vision-language-action models (VLAs). Still, some tasks we assumed needed separately trained models seem worth another look. Agentic Gen may be better placed to take advantage of the rapid improvement in general-purpose model capabilities.

Sutton’s 2019 essay, The Bitter Lesson, makes roughly this observation: over the long run, general methods that can keep using more computation, especially search and learning, tend to surpass methods carefully built around human domain knowledge. It’s easy to see why that feels bitter. We study a problem, design structures, add our experience, and achieve improvements. Then computation grows, and a more general approach catches up. The earlier investment becomes worthless.

Looking at that observation today, some capabilities we think require specialized modeling may simply be things general-purpose models aren’t good enough at yet. They can now inspect images, write code, and call tools. They have more computation during training, and can spend more time trying things when solving a problem. With further training and broader general capabilities, they may increasingly encroach on domains served by specialized models, as well as tasks within the scope of human abilities.

Sara Hooker’s “hardware lottery” also fits here. A research direction may succeed partly because it suits the hardware and software available at the time. The original Transformer paper already presented greater parallelizability and shorter training time as advantages. Perhaps today’s LMs have happened upon a path that fits existing hardware and makes it relatively easy to keep increasing investment. Algorithmic capabilities and infrastructure choices become intertwined.

So the question I want to ask now is this: of all the things that seem to demand careful, specialized design today, how many come from the problem itself, and how many fill gaps in current model capabilities? I don’t have an answer yet. Every time general methods advance a little further, though, I feel the question deserves to be asked again. Perhaps this is another Bitter Lesson.

References

1. OpenAI. On the Navier–Stokes Millennium Prize Problem. September 8, 2026; updated September 10, 2026.
2. OpenAI. GPT-6 Astra: A new generation of intelligence. September 3, 2026. Blender and Unreal Engine demonstration.
3. Robocurve. GPT-6 Astra on robotic manipulation. September 4, 2026. Results and limitations for two robot-arm tasks; the comparison covers other general-purpose models, not VLAs.
4. Richard Sutton. The Bitter Lesson. March 13, 2019. A university-hosted copy of the original essay.
5. Sara Hooker. The Hardware Lottery. 2020.
6. Ashish Vaswani et al. Attention Is All You Need. 2017.