Selfhosted

As a service? No, not really.

I use llms for specific tasks, usually small models, for things like identification and tagging (immich, paperless), translation or transcription (whisper), etc. I have a bunch of larger models on a Mac mini and my work laptop for various testing purposes, and have used them to make summaries and other such things.

They aren't great for search IMO, and search summaries can be quite bad.

I do find them useful to interrogate an unknown rs232 protocol or similar, though, which is where larger models come in handy.

Corporate LLMs are generally garbage to me not only because they are just data farming, but also because they really aren't good at everything. They might claim it, but the reality is a tailored model is often the better approach.

Using openwebui as a front end for llama.cpp is a decent strategy, combined with sub-agent routing (so generic model hands off to tailored models then returns the right info). This can be useful for some scenarios.

A caveat on coding - IMO, they are mostly bad at it. If you are using it to generate a small part of something that needs to be repeated they are handy, but at scale they aren't great, meaning for the entire project (or even an entire plugin or module, depending on complexity). Maintainability becomes a serious issue. While they have improved somewhat, there are limits to what they can do.

Macs with M series chips are good options for a hardware base since the memory is shared, and IMO is one of the most cost effective options over a dgpu with a ton of memory. A dgpu with a decent bit of memory (say 8-12GB) is a solid choice for a model thats larger (27b with 3 or 4b active as a mixture-of-experts model) especially if you have an iGPU for regular graphics tasks. Since its all shared on the M series macs, just get something that fits entirely in memory there.