Technology

LLMs have some use cases, just far fewer than the hype fawns over. Automating tedium is a good use; we've been using computers for this for years. Automating creativity and services is terrible, and in the latter case, merely an extension of phone trees that make it impossible to reach a real person.

I have a good example from yesterday: I use CashApp for all of my banking needs, and I get distributions twice a month to cover rent and essentials. Well, yesterday, I had an unexpected charge that was partially reversed but left me in overdraft. I reached out to my mom and explained the situation, at which point begins four fucking hours of hell on both ends, and, of course, customer service tries to keep you in an "AI" loop before letting one talk to a real person.

But surprise! This is another "AI" with more elaborate scripts, each more insulting than the last. Yes, I'm sure I've entered all the information in correctly. Yes, I've tried it multiple times. The issue here is that the app is not doing today what it did yesterday under identical circumstances. No matter how I tried to describe the edge case we'd apparently run into, the chatbot insisted it was user error; everything's fine on their end.

Eventually, I get a link to talk with an alleged "real person," and the process repeats. It doesn't much matter if they're real or not when sticking to the script nets the same results as the first two chatbots.

The error message mom is getting when attempting to send money (and she attempted this multiple times) was "Your app is not up to date; please redownload and try again." And, of course, she had the most recent version and was able to confirm that. Her chatbot experience served only to frustrate her, so I looked at what I could figure out on my end, though she's on iOS, so replicating the issue was impossible.

Eventually, after trying to access my account through the Web portal instead, I run into a prompt telling me I need to create a new $cashtag. What's happened to the one I've been using without issue for years? "Customer service" muses that I did something to my account myself, or that there's been fraud I'd have clearly known about. That's the handle people pay me via, and changing it is not in my interest. But the "AI" knows all, and obviously everything is hunky-dory on their infrastructure end, so it's a me problem. Also, I can't have it back.

After further useless steps I'm guided through, we arrive where we were three fucking hours prior, I finally acquiesce and set up a new tag.

This is when the lightbulb goes off: There's a nonzero chance that my tag being canceled had unexpected downstream effects. On the fourth call with my mom, I tell her I had to pick a new one and share it, suggesting she give it one more try.

And it goes through as expected.

So, the error message she was getting and that chatbots were attempting to fix was a complete red herring. An error message of "the $cashtag you selected is no longer active" would have been useful. The "AI" being aware of the incorrect error message would have also been useful. Telling me that my tag had been canceled to start instead of walking me in circles, uninstalling, reinstalling, clearing cache, the whole nine yards, would have been useful.

Instead, two people spent four hours each trying to figure out two problems, one caused by the other. A full workday on a Saturday dedicated to troubleshooting issues the bots were blithely unaware of, even though it's literally impossible this is the first time these specific issues came up at the company. That's more than $200 of free labour to arrive somewhere that should have been known to the system.

This is what you cause when you don't use LLMs as intended.

That said, I still use it as a far more powerful Grammarly, as even on my laptop, I have a nasty propensity for typing totally correct spellings of incorrect words, and it's great as a fresh set of eyes where I'd fill in the word that should have been there upon editing. I generated a server image for a Discord based on an out-of-context line (a comically oversized rooster in an Alpine valley -- taller than the Alps themselves -- looking down on a scale cow, with a far less involved prompt), and there was much mirth and merriment.

But these are no-stakes, low-impact uses. As soon as it's adjacent to something mission critical, not just for a business but also their customers, the level of scrutiny for software needs to be as high as it was pre-ChatGPT. And since that negates imagined cost-savings, ain't gonna happen.

You can eventually work a screw into some materials with a hammer and insistence that it's an improvement over a bespoke fucking screwdriver, but the substrate is damaged as a result.

Just so with LLMs. But more and more people are expected to use them in a work environment without anything approaching sufficient training, often in situations where they aren't domain experts. Garbage in, garbage out.