He fails to mention that he has AI also generate and do all of those tests, which means, A) he has no quality supervision over his quality supervision. Speaking from experience, AI can and will break a test in order to pass it rather than fixing the code if it randomly decides to. It is not always reliable and does not have even the baseline of good sense you would expect a brand new junior dev to have. And B) he's probably spending 3 times as much on tokens used for generating and running tests than he is already using on code generation. That's only going to be sustainable while these tokens remain cheap, which won't be for much longer, my guy.

Also it is always fun when someone asks you basic questions about your code and you have to admit complete and total ignorance because you neither wrote nor read a single line of it. How are you going to sell confidence in your black box that even you haven't peeked inside?

[–] ZILtoid1991@lemmy.world 24 points 20 hours ago (1 children)

The D Language Foundation tried to incorporate LLMs in the unittests, and they created something akin to assert(5 + (funcToTest(846) * 0) == 5) early on, now they want to use it to rewrite the standard library from scratch, because "it has become better since".

[–] kryptonianCodeMonkey@lemmy.world 33 points 20 hours ago* (last edited 20 hours ago)

The other day I had Claude write some code that failed a unit test. It was failing at a step that ran assertEquals(actualMap, expectedMap);. The reason it was failing was because one of the values in the map, a UUID, is automatically generated in the code. In the test, it has been hard coded.

Anyone with any sense would know you need to handle that misalignment. Omit that field from the match assertion? Set that UUID based on the generated value instead of hardcoding it? Anything like that would work just fine.

But Claud's solution was to change that assertion to assertTrue("The actualMap and expectedMap will never be equal because the values of ID will never match");. So the test always passes and it explicitly notes that they will not match. I would have fired a Junior Dev on the spot for writing that solution.