this post was submitted on 07 Sep 2026
513 points (97.2% liked)
Technology
87897 readers
4447 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Correct, that's how I would do it, but then I need another machine to act as a head.
If your MB has onboard graphics, maybe you could mask the GPU and just pass it off to a container running the LLMs I guess
No onboard graphics unfortunately, that would have been the easy way out.
Well it works anyway even with a bit of occupied vram, but you could also buy a cheap videocard to use as an output. I have small intel card like that in my server for jellyfin transcoding, I think I paid 60€ for it, it hardly uses any power
I actually did exactly that previously! I had both an RX 6800 XT and an RX 6600 in my system and I used the 6600 for video output. Unfortunately, this cuts my RX 6800 XT from PCIe 4 16x to PCIe 4 8x and severely slows down model loading for llama-swap. Joys of the X570!
And yes, I do have it running right now with a bit of occupied VRAM, but I need to limit my model to 14 GB to leave 2 GB free for GNOME Shell. I really want one of those 64 GB UMA Mac Mini, I heard they work really well because the GPU has direct access to system RAM.
So I have a framework laptop with ryzen ai cpu that uses 48gb of shared ram, and it does run Q4 llms fine enough, but I'm not sure it compares to a real GPU.
On my desktop I have an RX 7900 XTX but I've only dabbled in image generation so far, so right now I couldn't really tell you the difference.
24 GB VRAM. Damn, jealous! My 16 GB seems pitiful in comparison 😅
I do wonder if the Ryzen AI CPUs compare with Apple's UMA. I'm mostly interested in LLM inference for code generation and automation.