There was a time when the concept of the atomic bomb was just a scribble on a chalkboard, an abstract thought in the minds of scientists, discussed only obscure research labs.
Today, we find ourselves at a similar moment around the concept of artificial super intelligence (ASI), although the ideas are starting to spill into the mainstream. This week an ex-OpenAI safety researcher Leopold Aschenbrenner went public with some of the exposure he’s had to the theory within the leading lab (whose stated goal is to build AGI, the human equivalence milestone). Most of the safety team from OpenAI have quit in recent weeks, several, along with current employees have signed a letter demanding whistle-blower protections given recent revelations around the use of restrictive contracts tied to equity to keep them quiet. Aschenbrenner was fired for leaking but claims the real reason were his views on internal cybersecurity. No doubt the OpenAI blog post on their security infrastructure (or rather a lightweight list of commonplace best-practices) being posted this week was a mere coincidence. Aschenbrenner warns that OpenAI are not taking the geo-political dynamics seriously and that we are getting closer to a much more significant east versus west race-condition, that would go far beyond today’s economic de-coupling. He’s not alone in thinking that the algorithmic secrets and model weights leading us to ASI are at risk or perhaps even already compromised.
The so-called ‘scaling-laws’, or the way AI gets more powerful the more data and compute you feed in, is the theoretical path to ASI. These are not laws but a line on a graph, a trajectory that some see as inevitably leading to machines that can surpass human intelligence and rapidly self-improve, but that others are very sceptical of. But one thing is clear, compute and data are increasing massively. GPT-4 was trained on a ~$100 million dollar GPU run . Microsoft is reportedly planning a $100 billion dollar system called Stargate. It’s feasible we could be heading towards a $1 trillion platform that would see models 10,000x more powerful than today.
Aschenbrenner, who ‘s parents grew-up on opposite sides of the iron curtain, believes that we should not forget what states are capable of. He argues that given the implications of China’s authoritarian government being the first to achieve ASI, the US should rest control from corporations and both secure and accelerate its development through nationalisation. This echoes recent comments by Dario Amodei, CEO at Anthropic the creator of Claude where he stated to the New York Times that “when we get to ASL-4 (Claude is ASL-2 on their safety scale) it may make sense to think about the role of government stewarding this technology”. Anthropic are likely training and testing ASL-3 models in their lab today. Aschenbrenner describes a scenario assuming ASI is the key to global hegemony, where the race is not quickly dominated by one player, this causes more intense competition, more geo-political destabilisation and a higher chance of conflict.
The recent Biden executive order demonstrates there is meaningful governmental action on the safety testing and assessment of specific model capabilities, such as bioweapon risk. But with many governments focussed on more immediate economic and political matters, the appetite for dramatic state-led action seems limited. No action movie scene is playing out where crack government squads throw up a ring of steel around an AI lab and helicopters start flying in the top military brass. What the signals point to is a surprising lack of engagement from the national security apparatus in the US. Right now, despite all of the talk of AI regulation, Chinese AI chip embargoes, and DARPA experiments, the techno-capitalistic complex of big-tech firms is at the controls. They are bankrolling the labs, managing security, and deploying their vast cash reserves to build bigger and bigger GPU clusters, wherever is most commercially viable. The US military might have an $800 billion annual budget, but the globe spanning big tech firms are deploying as much as $200 billion a year on R&D alone (dwarfing DARPA’s $4.3 billion), and to some extent it may no longer be easy to bend their activities and absolute capabilities to the will of states. With this line on a graph being the ultimate exponential, the old adage suggests there are only two times that the US can act… when it’s too early, or when it’s too late.
Takeaways: What does this mean for AI adoption? With the right expertise, current generation models can be run very securely and offer huge value. The next generation will also likely provide many years of benefits for the most valuable and critical use-cases. ASL-4/GPT-6+ level models, and the scale of data centres they will require may end-up being all together more disruptive, and also more reliant on national access to computation and energy generation, putting the likes of the US, the oil rich states and China on different footings to Europe and the UK. Companies should harden their systems and start to experiment now with AI at the edge, multi-cloud, open-weight models and running this capability without over-reliance on a few tech firms (that might one day be compelled by the Defence Production Act to divert all of their efforts to the US war effort). They should also develop a long-term computational resilience strategy to determine how best to manage the uncertainties of the future.
For a long read and or a fascinating listen, check out Aschenbrenner’s recent podcast interview and paper on this topic.