this post was submitted on 30 Sep 2025
1088 points (98.7% liked)
Technology
75682 readers
3207 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
To preface I don't actually use ai for anything at my job, which might be a bad metric but my workflow is 10x slower if i even try using ai
That said, I want AI to be able to do unit tests in the sense that I can write some starting ones, then it be able to infer what branches aren't covered and help me fill the rest.
Obviously it's not smart enough, and honestly I highly doubt it will ever be because that's the nature of llm, but my peeve with unit test is that testing branches usually entail just copying the exact same test but changing one field to be an invalid value, or a dependency to throw. It's not hard, just tedious. Branching coverage is already enforced, so you should know when you forgot to test a case.
Edit: my vision would be an interactive version rather than my company's current, where it just generates whatever it wants instantly. I'd want something to prompt me saying this branch is not covered, and then tell me how it will try to cover it. It eliminates the tedious work but still lets the dev know what they're doing.
I also think you should treat ai code as a pull request and actually review what it writes. My coworkers that do use it don't really proofread, so it ends up having some bad practices and code smells.
That's what parameterization is for. In unit tests, most dependencies should be mocked, so expecting a dependency to throw shouldn't really be a thing much of the time.
You can get the first half with coverage tools. The second half should be fairly straightforward, assuming you wrote the code. If a branch is hard to hit (i.e. it happens if an OS or library function fails), either mock that part or don't bother with the test. I ask my team to hit 70-80% code coverage because that last 20-30% tends to be extreme corner cases that are hard to hit.
And this is the problem. Reviewers only know so much about the overall context and often do a surface level review unless you're touching something super important.
We can make conventions all we want, but people will be lazy and submit crap, especially when deadlines are close. >
The issue with my org is the push to be ci/cd means 90% line and branch coverage, which ends up being you spend just as much time writing tests as actually developing the feature, which already is on an accelerated schedule because my org has made promises that end up becoming ridiculous deadlines, like a 2 month project becoming a 1 month deadline
Mocking is easy, almost everything in my team's codebase is designed to be mockable. The only stuff I can think of that isn't mocked are usually just clocks, which you could mock but I actually like using fixed clocks for unit testing most of the time. But mocking is also tedious. Lots of mocks end up being:
Chances are, if you wrote it you should already know what branches are there. It's just translating that to actual unit tests that's a pain. Branching logic should be easy to read as well. If I read a nested if statement chances are there's something that can be redesigned better.
I also think that 90% of actual testing should be done through integ tests. Unit tests to me helps to validate what you expect to happen, but expectations don't necessarily equate to real dependencies and inputs. But that's a preference, mostly because our design philosophy revolves around dependency injection.
I think both are essential, and they test different things. Unit tests verify that individual pieces do what you expect, whereas integration tests verify that those pieces are connected properly. Unit tests should be written by the devs and help them prove their solution works as intended, and integration tests should be written by QA to prove that user flows work as expected.
Integration test coverage should be measured in terms of features/capabilities, whereas unit tests are measured in terms of branches and lines. My target is 90% for features/capabilities (mostly miss the admin bits that end customers don't use), and 70-80% for branches and lines (skip unlikely errors, simple data passing code like controllers, etc). Getting the last bit of testing for each is nice, but incredibly difficult and low value.
I use Python, which allows runtime mocking of existing objects, so most of our mocks are like this:
Most tests have one or two lines of this above the test function. It's pretty simple and not very repetitive at all. If we need more complex mocks, that's usually a sign we need to refactor the code.
I absolutely hate dependency injection, most of the time. 99% of the time, there are only two implementations of a dependency, the standard one and a mock.
If there's a way to patch things at runtime (e.g. Python's unittest.mock lib), dependency injection becomes a massive waste of time with all the boilerplate.
If there isn't a way to patch things at runtime, I prefer a more functional approach that works off interfaces where dependencies are merely passed as needed as data. That way you avoid the boilerplate and still get the benefits of DI.
That said, dependency injection has its place if a dependency has several implementations. I find that's pretty rare, but maybe its more common in your domain.
A software tester walks into a bar, he orders a beer.
He orders -1 beers.
He orders 0 beers.
He orders 843909245824 beers.
He orders duck beers.
AI can be trained to do that, but if you are in a not-well-trodden space, you'll want to be defining your own edge cases in addition to whatever AI comes up with.
Way I heard this joke, it continues with:
A real customer enters.
He asks where the toilets are.
The bar explodes.