Anthropic proposes that the US government be able to stop dangerous AI models

In June, Anthropic published its Advanced AI Framework, a proposal for how governments, chiefly that of the United States, should regulate the catastrophic risks of frontier models: biological weapons, cyberattacks, loss of control and the automation of AI research.

It would cover models trained with more than 10²⁵ floating-point operations, from companies with more than $500 million in AI revenue or $1 billion in research spending. They would have to test them, publish results, undergo independent evaluation and secure their systems. The government could block or deter their deployment, with fines scaled to global revenue, and the framework sets out safeguards, such as only a judge being able to impose those measures.

Anthropic asks that Congress not override state laws unless it passes a federal law at least as strict as the framework. According to Politico, the company says its October draft bill reflects it.