Fuck AI

TLDR: In short, like a lot of AI use the biggest problem is more how other people are using it that I have little control over.

The thing is that CodeGen utility varies greatly on what is being advocated and what situation it is being applied to. CodeGen getting things going when the operator isn't strongly opinionated about the details for fairly common patterns is fairly strongly in the wheelhouse. It is somewhat worse at amending a project in the same broad ways it was able to do when starting from scratch, even a project the model itself generated. So as you progress, it is more and more likely that a human will need to understand what is going on to be effective in modification. This presents a problem as people work themselves into a mess and can't get out of it and this causes a fair number of projects to just get abandoned because they can't go anywhere.

Even as it has demonstrable utility, the real world implications can be a mess. For example, the other week someone used agentic AI to open up 70 'security findings' on a project I work on. To it's credit, it found one actual issue, and while another issue was incorrect, looking into it I did find a separate issue it didn't notice, so I got two fixes out of it. However I had to deal with 68 completely stupid things that weren't anything. The operator at least up front sent an email that they didn't understand any of it and how much was real, but wanted to share in case any of it is useful. But I guess that's the price I pay for the two valid issues that might have otherwise not been caught. Then a few days after settling those, another user opened up over 60 and they were all dupes. Then the next week another user opened up a bunch that were all dupes too. Now it's ridiculous. One could argue that I could fight fire with fire and put an LLM on triaging the issues and closing out the dupes, but that means random github users can now make me spend my money on LLM services just by opening issues. Further, after being on the receiving end of AI chatbot triage in customer support, I hate inflicting that on the humans opening issues.

Then of course there's the code submissions and expectations around how I should handle them. There was a longstanding understanding that folks try their best, but implementing feature requests takes effort and folks are broadly understanding at delays or being a bit down the priority list. Now someone will Claude up a merge request instead of an issue and be impatient because "Claude already did the hard work, all you have to do is accept it", and it's a mess of code to review. It's not like the code at the hands of the operator is exactly good, for example someone sent a merge request because some feature raised an exception for them and blocked it from working and Claude "fixed" it. Problem is Claude changed the code to catch the exception, do nothing, and just say that it worked. In their test scenario, the feature was trying to make things the way they already were, so a no-op was no problem, but when I made their test case actually start from another state, it failed but still said it worked.

Another project I used to be a part of recently got handed over from the maintainers that long maintained it and lost issue to a super Slop-happy AI user. They got it rigged so it's all agentic and "addresses all issues and resolves all pull requests automatically from anyone". The project had some poor issue hygeine, so this meant the agentic code was doing things like fixing an issue from 2017 where the user complained it didn't work with Ubuntu 16.04. No idea if the fix actually worked, but it did make code changes that realistically no one would care about if they only did what they want. Except now they are getting bunch of new issues because they say "hey, the latest update broke a bunch of stuff I relied on", and another said "hey, it was great to see my issue get fixed last week, but this week the issue is back?" and messages saying "the update pace is ridiculous, why are there 3-4 releases a day, the former cadence was once a quarter". It's become a broiling mess of chaos because some AI enthusiast decided to make a nearly abandoned project that people were using a playground for AI usage. Dude even said point blank it's fine, just open an issue and AI will fix it for you and if it breaks, then someone else just open another issue and it'll get fixed and he is confident that eventually the users will shape the project into some equilibrium that way, so he has to do very little except pay for the tokens.