The Crossroads of Open-Source AI: Big Tech vs. Free-for-All

Friday - May 12 2023, 09:52 UTC - 1 year ago

In the last few weeks, multiple open-source large language models have been released aiming to counter the grip of the big tech companies on AI. This open-source innovation is likely to benefit the development of AI, but is also precarious and many of these models rely on the big tech to generate the models in the first place.

Last week a leaked memo reported to have been written by Luke Sernau, a senior engineer at Google, said out loud what many in Silicon Valley must have been whispering for weeks: an open-source free-for-all is threatening Big Tech’s grip on AI.

New open-source large language models—alternatives to Google’s Bard or OpenAI’s ChatGPT that researchers and app developers can study, build on, and modify—are dropping like candy from a piñata. These are smaller, cheaper versions of the best-in-class AI models created by the big firms that (almost) match them in performance—and they’re shared for free.

In many ways, that’s a good thing. Greater access to these models has helped drive innovation—it can also help catch their flaws. AI won't thrive if just a few mega-rich companies get to gatekeep this technology or decide how it is used.But this open-source boom is precarious. Most open-source releases still stand on the shoulders of giant models put out by big firms with deep pockets. If OpenAI and Meta decide they’re closing up shop, a boomtown could become a backwater.

For example, many of these models are built on top of LLaMA, an open-source large language model released by Meta AI. Others use a massive public data set called the Pile, which was put together by the open-source nonprofit EleutherAI. But EleutherAI exists only because OpenAI’s openness meant that a bunch of coders were able to reverse-engineer how GPT-3 was made, and then create their own in their free time."Meta AI has done a really great job training and releasing models to the research community," says Stella Biderman, who divides her time between EleutherAI, where she is executive director and head of research, and the consulting firm Booz Allen Hamilton. Sernau, too, highlights Meta AI’s crucial role in his Google memo. (Google confirmed to MIT Technology Review that the memo was written by one of its employees but notes that it is not an official strategy document.) .

If the trend toward closing down access continues, then not only will the open-source crowd be cut adrift—but the next generation of AI breakthroughs will be entirely back in the hands of the biggest, richest AI labs in the world.

The future of how AI is made and used is at a crossroads.

--- Open-source bonanza --- .

Open-source software has been around for decades. It’s what the internet runs on. But the cost of building powerful models meant that open-source AI didn’t take off until a year or so ago. It has fast become a bonanza.

Just look at the last few weeks. On March 25, Hugging Face, a startup that champions free and open access to AI, unveiled the first open-source alternative to ChatGPT, the viral chatbot released by OpenAI in November.

Hugging Face’s chatbot, HuggingChat, is built on top of an open-source large language model fine-tuned for conversation, called Open Assistant, that was trained with the help of around 13,000 volunteers and released a month ago. But Open Assistant itself is built on Meta’s LLaMA.

And then there’s StableLM, an open-source large language model released on March 19 by Stability AI, the company behind the hit text-to-image model Stable Diffusion. A week later, on March 28, Stability AI released StableVicuna, a version of StableLM that—like Open Assistant or HuggingChat—is optimized for conversation.