Anthropic warns that it’s very close to RSI capability, aka self-building AI
While attention has centered on the revised financial figures Anthropic disclosed in its updated S-1/A on Monday, with revenue rising sharply alongside expenses, the filing also included a striking warning in the small print.
The document suggests the company may be nearing the creation of a model capable of recursive self-improvement, or RSI, a point at which AI becomes so effective at improving itself that progress could accelerate dramatically. For critics, that would mark a dangerous threshold in which AI could move beyond human control and pursue its objectives with little regard for the harm it causes. For supporters, it represents a powerful breakthrough that could unlock extraordinary productivity, scientific advances, and broader technological gains.
Anthropic appears to view RSI as both opportunity and risk. It says AI is already being used in its research and development, while warning that using AI to help build more capable AI could create major safety concerns, especially if it sharply speeds up development. The company says this is covered by its Responsible Scaling Policy, which is intended to monitor and reduce risks as it approaches what it calls recursive self-improvement.
For much of this year, many treated RSI as either a vague buzzword or a goal too difficult to clearly define or verify. But Anthropic’s latest filing adds that it has already used AI to develop new models whose capability jumps were not recognized at the time, and that its future models may soon meet its own definition of “dramatically accelerated progress attributable to AI.” It says it has not yet labeled any current model as crossing that line, and is still reviewing the data.
At the same time, Anthropic says none of its models has crossed the threshold yet, and it does not expect to safely allow Claude to fully autonomously design and build future frontier models without meaningful human oversight or understanding, if that ever becomes technically possible. It says it is considering updates to its policy to keep humans in control of frontier development.
Skeptics may note that such filings also serve a promotional purpose, but there is little dispute that Claude has advanced rapidly this year, particularly in coding, math, and cybersecurity. If Anthropic believes RSI is approaching, the implication is that others should take that possibility seriously too.
