In many ways, this shouldn’t be a surprise to anyone. The current renaissance in open source LLMs comes hot on the heels of a renaissance in image generation. The similarities are not lost on the community, with many calling this the “Stable Diffusion moment” for LLMs.
In both cases, low-cost public involvement was enabled by a vastly cheaper mechanism for fine tuning called low rank adaptation, or LoRA, combined with a significant breakthrough in scale (latent diffusion for image synthesis, Chinchilla for LLMs). In both cases, access to a sufficiently high-quality model kicked off a flurry of ideas and iteration from individuals and institutions around the world. In both cases, this quickly outpaced the large players.
These contributions were pivotal in the image generation space, setting Stable Diffusion on a different path from Dall-E. Having an open model led to product integrations, marketplaces, user interfaces, and innovations that didn’t happen for Dall-E.
The effect was palpable: rapid domination in terms of cultural impact vs the OpenAI solution, which became increasingly irrelevant. Whether the same thing will happen for LLMs remains to be seen, but the broad structural elements are the same.
Google "We Have No Moat, And Neither Does OpenAI"
from Dylan Patel
- The upshot for the industry at large, is: the **LLM-as-Moat model h...from Steve Yegge
- What I built isn’t an ActivityPub system as much as a Mastodon-comp...from Tom MacWright
- Any software is considered free software so long as it upholds the ...from writefreesoftware.org
- We don't quite know what to do with language models yet. But we...from Maggie Appleton
- These are wonderful non-chat interfaces for LLMs that I would total...from Maggie Appleton
- combining search and AI chat is actually the wrong way to go and I ...from Garbage Day
- the tech am I digging recently is a software framework called **Lan...from Interconnected
- We’re building apps to surround and harness AI, but we need microsc...from Interconnected