IBL News | New York
Anthropic’s CEO, Dario Amodei, said this Monday that his company has “never advocated for a ban on open-weights models,” in an attempt to counter criticism emerging across the tech industry that the AI lab is trying to exert excessive control over the future of AI.
“Open-weights models that don’t have dangerous capabilities are a public good: they don’t cost anything besides the compute needed to run them, and they provide value to businesses, developers, and researchers,” he explained.
“We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed.”
Amodei published his views in a blog post after a coalition of leading companies, including Nvidia, Microsoft, Meta, and Palantir, released a letter late last week urging policymakers to avoid “premature restrictions” on open-weight models, which users can download, modify, and run on their own infrastructure.
Chinese startups currently dominate the market, and some government officials have started to weigh whether those models should be banned or restricted in the U.S.
In his post, Dario Amodei stated that his “primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. “
• “The most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression.”
• “My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks, and may have serious alignment problems.”
• “Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released, they cannot be withdrawn.”
• “We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling and workarounds used to obtain access to such chips.”
• “We should crack down on industrial-scale distillation operations. Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable, and thus partially evade chip bans. Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier.”
• “All sufficiently capable models, open and closed, should go through mandatory safety testing. The best way to address threat #2 is to just directly test models for cyber, biological, and alignment risks before release. I think this idea is actually close to a consensus.”