Jacob Coxon Departs Anthropic, Warns of AI Risks
- Jacob Coxon announced his departure from Anthropic after three years focused on model pretraining at both OpenAI and Anthropic.
- Coxon criticized both companies for racing towards self-improving superintelligence, claiming it poses a risk to humanity.
- Evan Hubinger, head of Alignment Science at Anthropic, supported Coxon’s view, estimating over a 10% chance that AI could lead to human extinction within the next decade.
- Coxon called for voluntary constraints on AI development to mitigate risks associated with advanced systems.
- He noted that many employees at OpenAI may not fully understand the potential dangers of superintelligence compared to their counterparts at Anthropic.
Coxon’s warnings highlight significant concerns within the AI community regarding the rapid pace of development and its implications for safety and alignment with human values. The call for coordinated safeguards among U.S. AI laboratories reflects a growing awareness of these risks.
The discussion around AI’s potential threats is underscored by Hubinger’s estimate that there is more than a 10% chance of extinction due to AI within ten years, emphasizing the urgency for responsible development practices.
