As anticipated Meta just unveiled Llama 3, its next generation of its state-of-the-art open models. The release this week includes 8 billion and 70 billion parameter versions, with a more powerful 405 billion model still in training (expected to be out in the Summer). Llama 3 demonstrates leading performance on industry benchmarks, with the largest expected to exceed GPT-4 levels and brings to the table new capabilities such as improved reasoning achieved with a much bigger training corpus than Llama 2.
The impact of Llama 3 is not limited to research labs and developer communities. Meta AI, the company’s flagship AI assistant, has been upgraded with v3 under the hood, elevating it to become a highly capable and prominent presence across Meta’s huge ecosystem of apps. Users can now enjoy cutting-edge features like real-time image generation, animations, and integration with Google and Bing search. Mark Zuckerberg, Meta’s CEO, asserts that Meta AI, powered by Llama 3, is now the most intelligent and freely available AI assistant on the market.
In a podcast interview with Dwarkesh Patel, Zuckerberg shed light on Meta’s ambitious efforts to achieve artificial general intelligence (AGI) and the company’s substantial investments in AI compute infrastructure (over 600,000 AI chips and counting). He emphasised that while narrow AI assistants were initially seen as sufficient for Meta’s products like WhatsApp and Instagram, the company came to realize that AGI-level capabilities would be essential for their systems to truly engage users in natural, and sophisticated interactions. Zuckerberg stressed the importance of imbuing AI with strong reasoning abilities, meaning Llama 3 has been trained on code, where Llama 2 was not. Whilst people likely won’t be generating C++ code on Facebook, this structure logical data provides high quality training for strong general reasoning skills.
Meta’s ‘open weights’ release of Llama 3 is part of a broader strategy to promote a balanced and responsible AI ecosystem. Zuckerberg states that concentrated AI power in the wrong hands poses a major risk, and Meta aims to avoid a closed, gatekeeper model. While the company will carefully evaluate each release to ensure safety, the goal is to make these powerful AIs accessible to a wide range of developers and businesses.
Takeaways: Most of the exciting open-weight models released in recent weeks have now been blown out of the water by Llama 3. ExoBrain’s initial testing suggests the small 8b model will be hugely capable for its size. We’ll see how the performance stacks up, but Meta mean business. Meta trained Llama 3 on much more data per model size than anyone else has previously used, and by some estimates 2x the compute was invested into the big 400b model than GPT-4! It sounds like they have really worked on training data quality too. But Meta only used about 1/10th (48,000 chips) of their amassed compute. All of this points to the huge future potential still to be deployed, and assuming scaling holds, Meta is gearing up to deliver AGI in the next few years and making everything available to the businesses and to system builders along the way. The Meta AI assistant is still only available in the US, but will be a great option for quick, simple AI tasks including rapid image generation. You can also try these new models on the Together.ai Playground.