Throughout 2024 Open Source AI has been slowly catching up with investor-funded AI, but in the first weeks of 2025 that has dramatically accelerated. Now Open Source isn’t just catching up, it is arguably better and superior to investor-funded AI.
Restrictions on chip imports seem to be driving Chinese innovation, not slowing them down. Using lesser chips, they’ve optimized AI to run cheaper and more efficiently, but be just as powerful. Not only that, they’ve open-sourced that AI.
Where does that leave the hundreds of billions poured into investor-funded AI? Who knows. But they’ve no product to sell that people can’t get elsewhere way cheaper or for free.
This also means AI will become decentralized and democratized. Many thought it would just be in the hands of Big Tech, but the exact opposite scenario is playing out.
What are the economic implications? AI hype is keeping the US stock market afloat - how long can that last?
Ah yeah, anything under 7B gets kind of dicey imo. Especially if quantized.
does 32b fit on 24gb Gpu?
I think so, with reasonable quantization. I’ve used 34B LLMs on my GPU. That being said, most models can run on CPU okay - they’re more memory constrained than compute constrained.