Recursive self-improvement

Engineers already use AI to write code, design training runs and sift research. If those models get good enough to meaningfully improve their own successors, the improvement curve stops being limited by how fast humans work.

Dario Amodei named this as one of two reasons he now argues for slowing capability growth, writing that it is starting to happen across the industry and could outrun our ability to understand the resulting systems.

Nobody has measured a runaway loop. The disagreement is about whether the early signs justify acting before one appears.