Fresh from launching rockets and filing his law suite against OpenAI (and promising to drop it if they change their name to “ClosedAI”) Elon Musk this week stated he would open up his Grok AI to the world. No details were available as we posted this, but this is likely to be billed as a ‘truth-seeking’ AI and as such may come with open training data and code.
This will add to hundreds of thousands of AI models now available to download from community websites, and runnable on your own laptop or desktop. As the state-of-the-art (SOTA) systems have been getting bigger, many much more compact models are now available for a range of specific uses; writing, coding, chatting, image generation etc.
Takeaways: Look out for the term open source in the context of AI models; its mostly used incorrectly. Models that are available for download such as Meta’s Llama 2 or from Mistral are ‘open-weight’, meaning the file containing the model can be copied and run by anyone. But… this is not open-source software. The code and data used to train them are rarely shared, and the ‘weights’ or patterns encoded into their neural networks are a meaningless mass of numbers to the human eye.
Nonetheless with a medium power laptop or desktop (and if you have a gaming GPU all the better) you can download apps like LM Studio. Once installed search for models which you can run for just the cost of the electricity. LM Studio will find models that are popular and indicate which size (quantisation) of model will run on your machine. Try searching for Google’s Gemma 2 (a ~2 billion parameter model) and get a capable AI assistant that will run in 8GB of local RAM, without the need for an internet connection or subscription fee. Look for LLM360s models for true open-source with everything from training data to underlying code shared with the community. The future is shaping up to see AIs of many sizes and skillsets running on every device from routers, washing machines, phones, and wearables to robots, drones and autonomous vehicles. This won’t be without its risks as this analysis of open-weight foundation models indicates.