In the last month, after a period of mounting questions about their lack of shipping products, OpenAI have pressed fast forward. This week, from their annual San Francisco developer day to a surprise interface update for ChatGPT, the company is moving from talking to deploying.
Firstly, the introduction of Canvas on Thursday (initially to all Plus users, in beta form), OpenAI’s new collaborative interface for ChatGPT, appears to be more than just a facelift. It’s a recognition that the chatbot format, while flexible, has its limitations. Canvas allows users to work on writing and coding projects in a more intuitive, document-like environment and is modelled on Anthropic’s Claude artifacts. For writing the versioning, partial paragraph level updates, and live editing tools are a welcome update. For software engineering the inline control could dramatically improve the process of iterating and perfecting up to several hundred lines of code. The line-by-line refine and update and reminiscent of the exceptional editing experience of the increasingly popular Cursor IDE. In the screenshot below you can see the classic chat to the left, with handy info, plus on the right the code window and a section of code highlighted for specific questions or editing.
Earlier in the week, OpenAI’s developer day was packed with solid technical announcements. The new Realtime API brings low-latency true speech-to-speech experiences to developers’ fingertips, while vision fine-tuning opens up possibilities for more challenging use-cases. These and the introduction of cost reducing prompt caching, model distillation and reduced API costs, are the building blocks for the next generation of AI-powered systems.
And the o1 model family (AKA ‘Strawberry’) was never far from the discussions. It’s reinforcement learning advancement continue to show promise in planning, mathematical, and coding tasks. As Noam Brown, Ilge Akkaya and Hunter Lightman discussed in a recent Sequoia Capital podcast, o1’s ability to use chains of thought and backtracking mimics human-like reasoning. This isn’t just about solving equations – it’s about creating AI that can tackle complex, multi-step problems across various domains.
Never one to miss the opportunity, Sam Altman’s dev day fireside chat added fuel to the AGI speculation fire. The OpenAI CEO, who is increasingly concentrating his power at the helm of the firm, reiterated the company’s ambitious goals for achieving artificial general intelligence. He suggested that debate around how to define AGI, means we’re materially closer to the threshold. Bold claims as ever, but ones that are backed by a big $6.6 billion funding round that values the company at $157 billion…
Yet, amidst the innovation and investment, OpenAI faces internal challenges. The ongoing leadership attrition raises questions about the company’s culture and direction. The apparently hasty release of o1, coupled with internal friction over safety concerns and product timelines, suggest a shift from the company’s original ethos of safety-first AI research. Mira Murati’s departure, along with other key executives, hints at multiple internal struggles. Thursday saw news that the still-to-be-released Sora video generation model co-lead had left to join Google. The rapid expansion of the workforce, with an influx of commercially oriented hires, marks a cultural transformation from a research-focused entity to a product-driven powerhouse. The product track belies the concern that as a model ‘training shop’ it lacks a moat in an increasingly commoditised space.
Takeaways: OpenAI’s 2024 H2 developments are somewhere between incremental and pivotal. The acceleration is noticeable, but these are parts of the future, not the finished vision. The ball is now in the court of app developers and AI consultants to work out how to make the most of an almost overwhelming array of options. As we’ve covered in recent weeks, Microsoft, Apple and the other horizontal platform providers will gradually and relatively cautiously add AI features built on building blocks from the likes of OpenAI. But far more is already possible today, and too many of the world’s knowledge and productivity problems are going un-connected to this incredible new toolkit.