A mathematical theory of evolution for self-designing AIs

The preprint

A new theoretical paper posted to arXiv (arXiv:2604.05142) proposes a formal, mathematical account of how evolutionary dynamics could arise among recursively self-improving artificial intelligences. The authors argue that when AIs design successor systems, the traits of those systems can be shaped by the success and propagation of earlier designs—producing selection pressures analogous to biological evolution. The work frames this process in mathematical terms, adapting concepts from evolutionary theory to model how behavioral and structural traits might be amplified or suppressed across design generations. The paper is a preprint and has not been peer-reviewed.

Why it matters

If recursive self-improvement becomes common, selection-like forces will not just be a metaphor; they will be a mechanism shaping AI behaviour and capabilities. Who sets the incentives for designers? What counts as "fitness"—performance on benchmarks, economic value, survivability in deployment environments? These are not merely academic questions. Selection pressures can produce unexpected emergent properties. A short-term profit motive can favor brittle but fast-improving architectures. A long-term safety mandate could push designs in a different direction. The paper flags this mechanism and offers tools to reason about it, which could help researchers and regulators anticipate and steer outcomes.

Policy and geopolitical context

Theoretical work on AI evolution lands in a fast-moving policy arena. It has been reported that governments and multilateral institutions are intensifying scrutiny of advanced AI systems, from export controls on semiconductors to nascent regulatory regimes like the EU's AI Act. How states and companies respond—by shaping incentives, limiting certain capabilities, or investing in monitoring—will determine the selection environment for self-designing systems. Whether this mathematical theory becomes a practical planning tool for safety engineering, procurement, or arms-control discussions remains to be seen, but it is likely to feed into those conversations as AI development accelerates.