this post was submitted on 15 Jul 2026
223 points (97.0% liked)

Technology

86354 readers
3208 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
223
Generative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project. (www.theatlantic.com)
submitted 4 hours ago by Some_Emo_Chick@lemmy.world to c/technology@lemmy.world
23 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] Asafum@lemmy.world 3 points 2 hours ago* (last edited 2 hours ago) (3 children)

Yeah I'm just beginning my local AI journey on a 5080, tried Qwen3.6 27b Q4 and was getting like 1tps because of the vram overflow. Ran it over night at it was still chewing on generating a prompt for a sub agent when I got up in the middle of the night until it simply ended in some kind of "fetch failure" lol. I think I gave it something too large to tackle, but either way 1tps is kinda garbage.

[–] scarabic@lemmy.world 3 points 57 minutes ago

Wow I’m starting to feel bad about that time I asked AI to make a joke about scatology & eschatology sounding similar.

[–] RedstoneValley@sh.itjust.works 5 points 1 hour ago (1 children)

You could use the 35B MoE model, tune it a little bit and get much better results. I have a 5060 ti and 70-80 tok/s are the norm

[–] Asafum@lemmy.world 1 points 30 minutes ago

That's what I generally use. I wanted to see if I could use the 27b to "review" what the 35b put out. The 35b has been working pretty well, but it's not very thorough. I asked it to make a program and then 27b was like "this is a skeleton, there are folders but no contents." Lol

[–] worldclasspun@programming.dev 2 points 2 hours ago (1 children)

Is that quantized? 4 bit Qwen 3.6 can get 22tps on a 1060.

[–] Asafum@lemmy.world 1 points 29 minutes ago

It's the q4 quantization, but it requires 20+GB vram and my 5080 only has 16