The AI Apocalypse: A Brewing Storm or a Red Herring?

Recent warnings from experts about the potential risks of artificial superintelligence (ASI) have sparked a heated debate about the dangers of unchecked technological progress. Some see this as a catastrophic event waiting to happen, while others are more skeptical, cautioning against throwing out the baby with the bathwater.

At the heart of the controversy is the concern that ASI could pose an existential threat to humanity, potentially surpassing nuclear war in its destructive potential. Des Browne, a former defense secretary, has warned that AI poses “a threat on the same, possibly a greater scale” than nuclear war. This idea has gained traction in recent years, with several prominent figures weighing in on the issue.

However, not everyone agrees. Dr. Andrew Rogoyski of the Surrey Institute for People-Centred AI cautions against getting carried away with apocalyptic scenarios. According to Rogoyski, these systems are still far from being as versatile as humans, let alone humans acting collectively. He suspects we’re heading towards “the great disappointment” where advanced AI turns out to be too expensive and not useful enough to continue in its current form.

Behind the scenes, a well-funded lobby group called ControlAI is pushing for international regulation of ASI. Founded by Jaan Tallinn, an early investor in Anthropic and Google DeepMind, the group aims to curb the most powerful AI models and prevent their potential misuse. Despite being an advocate for AI safety, Tallinn himself has expressed concerns about the views held by some senior AI executives.

What’s striking about this debate is not just the technological risks but also the human dimension. Evan Hubinger, alignment science lead at Anthropic, has stated that his company doesn’t have a plan to ensure ASI is aligned with human values. Meanwhile, Jacob Coxon, a researcher who recently quit OpenAI, claims that neither company was acting responsibly and were “gambling with our lives.” This raises questions about the ethics of AI development and whether those working in this field are taking their responsibilities seriously enough.

As the debate rages on, it’s becoming increasingly clear that governments need to take a more active role in regulating ASI. Labour MP Darren Jones has written to the prime minister calling for a “multinational treaty for the regulated and safe development of superintelligence.” This is not just about banning innovation but rather taking a safety-first approach to the rapid development of this technology.

Some experts, like Prof Stuart Russell, are more alarmist in their warnings. He suggests that we’re facing a catastrophe that could be on the same scale as nuclear war. “The other possibility is a much larger catastrophe, in which humanity loses control irreversibly,” he warns. But what does this mean for us? Should we be panicking or should we take these warnings with a grain of salt?

As the stakes continue to rise, it’s essential that we focus on practical steps towards mitigating the risks associated with ASI. This includes international coordination, regulation, and transparency. The recent resignation of Jacob Coxon at OpenAI highlights the urgent need for accountability and a reevaluation of our priorities.

We can’t afford to ignore the warnings. We must take action now to prevent a catastrophe that could potentially wipe out humanity. The question is no longer if but when – and what we do next will determine whether we avoid the AI apocalypse or succumb to its dangers.