Selfhosted

Maybe but right now, according to the Steam hardware survey, the most commonly owned VRAM tier is 8GB - sitting at around 27% of surveyed systems, with 16GB closing the gap fast.

Even with MoE and llama.cpp tricks, you're not running frontier anything at decent context length without significant fiddling on that - it's possible on 8GB but you're operating with almost no headroom, 16GB might let you scrape by with a Q4 quant MoE.

The very best local coding models (arguably the Qwen3-Coder-30B-A3B MoE or Qwen 3.6-27B) need 16-17GB at Q4 quantisation, and building a system from scratch to run one is probably a $3.5K proposition. With the cost of living crisis hitting everywhere, that means the table ante gear is beyond the reach of many. Even a decent GPU is north of $1K in many local markets.

I adore small LLMs, and know a lot of tricks to leverage them, but 14B is the bare minimum for what I would start to consider competent.

We're boned until 2030ish, when gear gets cheaper (allegedly).

Part of me thinks there's a dark conspiracy at play here. Give people affordable access to frontier LLMs, make self hosting hardware cost prohibitive, then jack up subscription prices.

I think there's a way out of that mess, but it needs people to stop chasing "bigger, better" and start chasing "actually, how can I use what I have to do X instead of needing bigger and better?" but that needs talented devs and a mind shift.

ICBW and YMMV.