oh wait, is this this THE sol 5.6? The most amazing model ever with trust me bro benchmarks? The model that is observed cheating more than any previous model? surprised_pikachu.jpg

In all seriousness it never should have production creds anyway. But the fact this is soul sucking openai's newest flagship model with thinking cranked up is the cherry on top. The company that is literally cheating and shortcutting its way ahead produces models in its own image, the sci fi story writes itself.

[–] phailhaus@piefed.social 15 points 1 day ago (2 children)

What evaluation setup lets the thing being tested see the test results and gives access to the test source code?!

Slop evaluating slop.

[–] AwesomeLowlander@sh.itjust.works 1 points 17 hours ago

Given that they mark it as cheating, isn't that essentially part of the test in itself?

[–] myotheraccount@lemmy.world 7 points 1 day ago

The model is so smart, we let it write it's own evaluation setup /s