Bought b70 with egpu enclosure and usb4 connection wasn't really planning to actually run anything but now ended up with llama.cpp with openwebui - kids/parents want to/have to use chat, might as well provide local solution than them using industry options. Also started with ollama and Gemma 4 26b a4b - asked it to write script to setup llama.cpp in container.
this post was submitted on 24 Jun 2026
140 points (81.2% liked)
Selfhosted
62069 readers
430 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 3 years ago
MODERATORS
Yeah, I've heard the B70 is good bang for buck. My kids love using chat GPT to generate images and I'm aware that there are some really capable local models that can do that as well now - B70 should make short work of it.
That may be something for me to look at later on if I decide to keep self hosting.
OTOH, I'm also aware that I may end up building something that they don't actually use. Been there, done that, and I don't want to do it again.
Actually, on that topic, one interesting use case for me is my youngest one wants to have a YouTube channel.
So obviously, I'm not going to let her become a YouTuber, but what I'm thinking of doing is providing her my old phone (properly locked down) so that she can video record clips that she wants.
Then, have those clips sent automatically to our jellyfin server so it appears like a channel. Code a fake YT plugin so that AI can do likes, positive comments etc.
It's... work. I dunno...maybe a good enough AI can vibe slop the entire project for me.
I've been testing coding capabilities a bit (mostly scripts - so that work done by ai is reproducible).
- Context size is very much a required thing along with model capabilities.
- Local model can generate good enough script in one shot - but reiterations r crazy
- Use git to keep files tracked (easy to revert) and make a modular script - main script calls function a, function b etc where functions are relatively self contained (no need to look at others) when u need new capability add function c
- If something need changing- try to do it urself (unless it's whole architectural change, then just start new project)
As for image gen + chat . as long as chat model + context + image model fits, u should be fine.
I'm running dwarfstar which is a 2 bit deepseek v4 flash. It's quite capable even at 2 bit.
This dwarfstar looks interesting, can you elaborate on your setup and what kind of inference speeds you are getting?
I have a 5080 and 128gb of ram running on a AMD 9950X.
Depending on the task I can get over 170-200t/s when the MOE only calls a few agents and can fit inside the VRAM or as low as 5-10ts when it calls more agents and has to hit the system memory. But for grunt work that doesn't need professor level tasks, it's more than capable and if you have the time, it's super worth it because it's basically free tokens.
I only use this for overnight work to save on tokens during the day. When I'm pulling analytics for my work and it just needs basic analysis that doesn't touch multiple tooks.
During work hours I'm using GLM5.2 for web development, Kimi k2.7 for complicated data analysis and Minimax m3 if I need the context window to be bigger than what kimik2.7 can give me.
I've been running ministral on CPU on a home-server: works pretty nicely, not very performant for everyday tasks and the savings were not sufficient for it to make sense. It still was cheaper and faster to just use Mistral API and get better models.
No, too expensive. I wish I could but it doesn't make sense financially for me right now, it is much cheaper to buy openrouter credits from time to time
Yeah, mostly for translation purposes.
I think I currently have gemma 4 set up.
I prefer my critical faculties completely intact and un-altered, thank you very much.
I do not require or desire a 400 watt bullshit-artist yes-man or vulnerability coder cooking my GPU.
I'm still messing around with self hosting llm, rn ive settled on using lumo from proton if I use an llm.
When I have run llm, I used koboldcpp. Works pretty well, depends on what you are doing and what models you use. Forget which models ive been using off the top of my head
I would love to run and host a local LLM on my phone just to tinker and learn. I found a tutorial on setting DeepSeek on your Android phone using Termux but it is a year old. I'm sure there are better more efficient LLMs that can run on a phone now.