this post was submitted on 27 Aug 2026
197 points (99.0% liked)

Technology

87572 readers
4152 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] BassTurd@lemmy.world 8 points 10 hours ago (2 children)

Maybe, but CPUs are coming out with LLM tuned chips, and I can run a basic model on an i5, 8gb ram, and no dedicated card. It's not super powerful, but for most users, it's more than enough for what they use the big models for. Also, if it does take a discrete card to get that needed boost in performance, then at least consumers would be able to get GPUs again.

I think as hardware improve and is further designed around LLM efficiency, and local models are tuned for specific uses and being able to run on lesser hardware, it will make Nvidia obsolete for large swaths of the population. A good GPU will still be necessary for high performance, graphic/physics intense gaming, but that's a really small subset of all users.

Hopefully Nvidia just shits and has to grovel back to the consumer to get there marketshare back when all of the DCs go tits up.

[–] mesamunefire@piefed.social 1 points 55 minutes ago

Bitcoin went through a similar trajectory. Graphic cards to specualty chips.

[–] squaresinger@lemmy.world 9 points 8 hours ago (1 children)

I can run models on my 3yo midrange smartphone. Gemma-4-E2B totally runs on there. Bonsai-8B too. But neither is really good for most tasks.

Heck, "a basic model" even runs locally on the 8MB RAM of an ESP32-S3, but it's utterly worthless at anything.

If you get a bit more into self-hosting AI it quickly becomes obvious that for any actually useful real world tasks you need at least 24GB you can dedicate to the LLM alone, and if this is fully GPU vRAM, the performance is way, way higher than on CPU, even with an NPU.

[–] melfie@lemmy.zip 4 points 2 hours ago

any actually useful real world tasks you need at least 24GB you can dedicate to the LLM alone, and if this is fully GPU vRAM

Agreed. For anyone who doesn’t have a 24GB card, the AMD 7900 XTX is basically the only sanely priced 24GB GPU left in 2026 and can be had for under a grand new, whereas even a used 3090 is $1200 and $2000 new.

Qwen 3.8 27B runs nicely on my XTX. I have to use Claude at work, and Qwen 3.8 in llama.cpp with the OpenCode desktop app leaves nothing to be desired in comparison. 3.8 27B is about on par with Sonnet 5 according to benchmarks, though I even prefer it over Opus because Qwen writes in clear, understandable language. We are at the point now where anyone with a 24GB GPU has little reason to use data center LLMs.