Ilya Sutskever, the prominent AI researcher and co-founder of OpenAI, has announced the launch of his new venture, Safe Superintelligence Inc. (SSI). This news comes in the wake of Sutskever’s recent departure from OpenAI, disappearance from public life, and his long-term struggle with the dilemmas of creating world-changing technology.
Sutskever, widely regarded as perhaps the most gifted researcher in the world of AI, has been at the forefront of the field for years. Early on, Sutskever had a childhood fascination with AI, and the intuition that the reason neural networks, the software structures that underpin AI, weren’t performing well was simply because they were too small. He believed that with significantly larger systems, unprecedented capabilities would emerge. This insight has proven to be crucial in the development of modern AI. He was involved in the early re-purposing of GPU technology to run neural networks, in some ways setting in train the journey that has seen Nvidia become the world’s most valuable company this week. He worked at Google in the mid-2010s at the forefront of AI research, and then left to co-found OpenAI, and was instrumental in the creation of ChatGPT.
However, the past year has been challenging for Sutskever. The Altman affair in November 2023, which saw him place a casting vote leading to the temporary ousting of OpenAI CEO Sam Altman, put Sutskever under immense pressure. In one of the last interviews before the events, he expressed concerns about the rapid pace of AI advancement and the need for stronger safety measures. “If you allow yourself, if you just accept that the progress that we’ve seen, maybe it will be slower, but it will continue,” Sutskever warned, emphasising the importance of developing the science to control future intelligence. In the weeks leading up to the November affair, Sutskever tweeted a cryptic message: “if you value intelligence above all other human qualities, you’re gonna have a bad time.” This tweet, coupled with his subsequent disappearance from public life and eventual split from OpenAI, has sparked fevered speculation about what he might have witnessed or realised about the trajectory of development.
His involvement in the boardroom upheaval has been closely tied to his growing unease about the company’s increasing ‘product’ and commercial focus, and misalignment with his ethical ideals. As an advocate for responsible development, Sutskever argued for balancing safety and alignment using the fruits of commercialisation and growth. This divergence is set to widen with recent stories suggesting that OpenAI will drop their capped-profit status in the coming months. Sutskever’s concerns about the availability of compute resources at OpenAI for his now defunct ‘super alignment’ project played a significant role in his decision to leave to launch SSI. Alignment, in the context of AI, refers to the challenge of ensuring that artificial intelligence systems behave in ways that are consistent with human values, goals, and intentions. As Sutskever puts it, are ‘pro-social’. As AI systems become increasingly powerful and autonomous, the risk of misalignment—AI acting in ways that are detrimental to human interests—grows. “Super alignment,” is the even more daunting task of aligning AI systems that are more intelligent than humans, ensuring that their actions and decisions remain beneficial to humanity even as they surpass our own cognitive capabilities.
The key challenge Sutskever identifies is that future systems will be capable of extremely complex and creative behaviours that will be difficult for humans to reliably supervise, making humans “weak supervisors.” To address this, some propose an analogy that can be studied empirically today; using smaller, less capable models to supervise larger, more powerful ones. By encouraging the stronger model to be more confident and disagree with the weak supervisor, OpenAI’s disbanded team had shown promising results in allowing the more powerful model to perform optimally whilst still being under weaker supervision. While there are many limitations and questions that remain, this approach provides a framework for making empirical progress now, in advance of future major developments.
Sutskever’s new venture, SSI, is a renewed attempt to tackle the challenge, and echoes the idea behind founding OpenAI… to ensure there are ethical competitors in the race to ASI. The company’s mission is to “build safe super intelligence,” which Sutskever describes as “the most important technical problem of our time.” Their website reads; “Our business model means safety, security, and progress are all insulated from short-term commercial pressures.” SSI is co-founded by Daniel Gross, a former AI head at Apple, and Daniel Levy, who previously worked at OpenAI. Of course, the work of SSI and the term “build” in the mission, will be an accelerant in the race towards super intelligence as well as no doubt hugely valuable if successful. This is the fundamental dilemma faced by many ethically minded AI researchers; whether to actively contribute to the development of the state-of-the-art, or to abstain from the field altogether to avoid hastening progress. The strongest argument for doing the former, and getting involved, is that the power of compute, GPUs and datacentres is skyrocketing, meaning an ‘over-hang’ effect could kick in. Slower progress today would result in extra fast progress in the future when the computational power is unlocked, causing rapid change that results in more widespread economic and social destabilisation. A more gradual build-up is likely to be less disruptive, giving everyone more time to adapt.
Takeaways: There are few individuals we should take as seriously as Ilya Sutskever when it comes to the future of AI. His ground breaking contributions and exposure to the latest research make him one of the most significant technical voices. However, it is his genuine awareness and indeed public struggles with the profound implications of his work that mean we should follow his journey closely.