The D Language Foundation tried to incorporate LLMs in the unittests, and they created something akin to assert(5 + (funcToTest(846) * 0) == 5) early on, now they want to use it to rewrite the standard library from scratch, because "it has become better since".

[–] kryptonianCodeMonkey@lemmy.world 33 points 20 hours ago* (last edited 20 hours ago)

The other day I had Claude write some code that failed a unit test. It was failing at a step that ran assertEquals(actualMap, expectedMap);. The reason it was failing was because one of the values in the map, a UUID, is automatically generated in the code. In the test, it has been hard coded.

Anyone with any sense would know you need to handle that misalignment. Omit that field from the match assertion? Set that UUID based on the generated value instead of hardcoding it? Anything like that would work just fine.

But Claud's solution was to change that assertion to assertTrue("The actualMap and expectedMap will never be equal because the values of ID will never match");. So the test always passes and it explicitly notes that they will not match. I would have fired a Junior Dev on the spot for writing that solution.