An Anthropic safety researcher has said there is a greater than 10% chance that AI “could kill all humans” within the next decade, and admitted there is no coordinated plan to prevent it from happening.
U.S. company Anthropic, headed up by Dario Amodei, develops the Large Language Model (LLM) Claude and conducts AI research. Yesterday, one of the company’s researchers, Jacob Coxon, announced his resignation on X / Twitter, accusing both ChatGPT maker OpenAI and Anthropic of “racing straight to self-improving superintelligence and gambling with our lives.”
“Neither company is acting responsible,” Coxon, who previously worked at OpenAI, said. “Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,” he continued. “We have all witnessed the progress in each of these domains, and progress is not slowing.
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.”
Coxon said that staff at OpenAI, headed up by Sam Altman, “have not deeply internalized the civilizational stakes,” which is why work to rapidly improve its AI models continues despite the risks. “At Anthropic, the stakes are well-understood, but they are locked in a race to get there first," he insisted. “They believe no one else will act responsibly, so they must do it themselves, despite the risk.”
“Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.”
Coxon went on to say he was optimistic about the potential for coordination, but doubted AI companies were on track to prevent a global race, “which may require costly actions such as a temporary ban on improving model capabilities.”
He ended his statement by urging AI lab researchers to consider what they were doing. “Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?” he asked. “Should you put your head down because ‘it’s happening anyway’ - or take this moment to call for different conditions?”
سەرچاوەی فەرمی بە زمانی ئینگلیزی: https://www.ign.com/articles/anthropic-safety-researcher-says-theres-a-more-than-10-chance-ai-could-kill-all-humans-within-the-next-decade
