this post was submitted on 07 Dec 2025
984 points (98.0% liked)

Technology

77084 readers
2772 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
984
I Went All-In on AI. The MIT Study Is Right. (open.substack.com)
submitted 1 day ago* (last edited 1 day ago) by AutistoMephisto@lemmy.world to c/technology@lemmy.world
239 comments fedilink hide all child comments
 

Just want to clarify, this is not my Substack, I'm just sharing this because I found it insightful.

The author describes himself as a "fractional CTO"(no clue what that means, don't ask me) and advisor. His clients asked him how they could leverage AI. He decided to experience it for himself. From the author(emphasis mine):

I forced myself to use Claude Code exclusively to build a product. Three months. Not a single line of code written by me. I wanted to experience what my clients were considering—100% AI adoption. I needed to know firsthand why that 95% failure rate exists.

I got the product launched. It worked. I was proud of what I’d created. Then came the moment that validated every concern in that MIT study: I needed to make a small change and realized I wasn’t confident I could do it. My own product, built under my direction, and I’d lost confidence in my ability to modify it.

Now when clients ask me about AI adoption, I can tell them exactly what 100% looks like: it looks like failure. Not immediate failure—that’s the trap. Initial metrics look great. You ship faster. You feel productive. Then three months later, you realize nobody actually understands what you’ve built.

you are viewing a single comment's thread
view the rest of the comments
[–] utopiah@lemmy.world 1 points 18 hours ago (1 children)

If I understand correctly then this means mostly adapting the interface?

[–] jj4211@lemmy.world 1 points 18 hours ago (1 children)

It's certainly a use case that LLM has a decent shot at.

Of course, having said that I gave it a spin with Gemini 3 and it just hallucinated a bunch of crap that doesn't exist instead of properly identifying capable libraries or frontending media tools....

But in principle and upon occasion it can take care of little convenience utilities/functions like that. I continue to have no idea though why some people seem to claim to be able to 'vibe code' up anything of significance, even as I thought I was giving it an easy hit it completely screwed it up...

[–] PoliteDudeInTheMood@lemmy.ca 1 points 14 hours ago (1 children)

Having used both Gemini and Claude.... I use Gemini when I need to quickly find something I don't want to waste time searching for, or I need a recipe found and then modified to fit what I have on hand.

Everytime I used Gemini for coding has ended in failure. It constantly forgets things, forgets what version of a package you're using so it tells you to do something that is deprecated, it was hell. I had to hold its hand the entire time and talk to it like it's a stupid child.

Claude just works. I use Claude for so many things both chat and API. I didn't care for AI until I tried Claude. There's a whole whack of novels by a Russian author I like but they stopped translating the series. Claude vibe coded an app to read the Russian ebooks, translate them by chapter in a way that prevented context bleed. I can read any book in any language for about $2.50 in API tokens.

[–] jj4211@lemmy.world 1 points 14 hours ago (1 children)

I've been using Claude to mediocre results, so this time I used Gemini 3 because everyone in my company is screaming "this time it works, trust us bro". Claude has not been working so great for me for my day job either.

[–] PoliteDudeInTheMood@lemmy.ca 1 points 9 hours ago (1 children)

I think it really depends on the user and how you communicate with the AI. People are different, and we communicate differently. But if you're precise and you tell it what you want, and what your expected result should be it's pretty good at filling in the blanks.

I can pull really useful code out of Claude, but ask me to think up a prompt to feed into Gemini for video creation and they look like shit.

[–] jj4211@lemmy.world 1 points 6 hours ago* (last edited 6 hours ago)

The type of problem in my experience is the biggest source of different results

Ask for something that is consistent with very well trodden territory, and it has a good shot. However if you go off the beaten path, and it really can't credibly generate code, it generates anyway, making up function names, file paths, rest urls and attributes, and whatever else that would sound good and consistent with the prompt, but no connection to real stuff.

It's usually not that that it does the wrong thing because it "misunderstood", it is usually that it producea very appropriate looking code consistent with the request that does not have a link to reality, and there's no recognition of when it invented non existent thing.

If it's a fairly milquetoast web UI manipulating a SQL backend, it tends to chew through that more reasonably (though in various results that I've tried it screwed up a fundamental security principle, like once I saw it suggest a weird custom certificate validation and disable default validation while transmitting sensitive data before trying to meaningfully execute the custom valiidation.