this post was submitted on 02 Oct 2025
85 points (90.5% liked)

Technology

75682 readers
2811 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] InEnduringGrowStrong@sh.itjust.works 83 points 22 hours ago (6 children)

Microsoft says its Agent Mode in Excel has an accuracy rate of 57.2 percent in SpreadsheetBench, a benchmark for evaluating an AI model’s ability to edit real world spreadsheets.

It generates 42.8% bullshit.

[–] potoo22@programming.dev 13 points 17 hours ago

Just keep regenerating data until it's something the stock holders like. Doesn't matter if it's BS. They're already accustomed to that.

[–] jubilationtcornpone@sh.itjust.works 37 points 22 hours ago (2 children)

They probably view that as a statistic worth bragging about. It's not. If Excel got calculations right 57.2% of the time it would be completely worthless.

[–] PerogiBoi@lemmy.ca 2 points 12 hours ago (1 children)

I asked copilot to look through my every spreadsheet and find how many instances of a category occurred. I was curious to see if it was any good. Gave me 2 different numbers. Neither were correct.

[–] jubilationtcornpone@sh.itjust.works 2 points 11 hours ago (1 children)

Copilot: Putting the "Artificial" in Artificial Intelligence.

[–] PerogiBoi@lemmy.ca 1 points 11 hours ago

The tech behind LLMs could have just been Clippy and everyone would be happy.

[–] FreedomAdvocate@lemmy.net.au 0 points 16 hours ago (2 children)

Did you read the next sentence? Humans only get like 72% right. It’s not far off at all.

[–] MountingSuspicion@reddthat.com 5 points 12 hours ago (1 children)

I wonder where that "human accuracy" statistic is coming from. Plenty of people don't know how to read and interpret data, much less use excel in the first place. There's a difference between 1/4 of people in the workforce not being able to complete a task, and a specialized AI not being able to complete a task. Additionally, this is how you get into the KPI as a goal rather than a proxy issue. AI will never understand context isn't directly provided in the workbook. If you introduced a new drink at your restaurant in 2020 AI will tell you that the introduction of the drink caused a 100% decrease in foot traffic since there's no line item for "global pandemic". I'm not saying AI will never be there, but people using this version of AI instead of actual analysis don't care about the facts and just want an answer and for that answer to be cheap.

[–] FreedomAdvocate@lemmy.net.au 1 points 2 hours ago

As I’ve said many times, though not in this topic - AI is a tool to be used, and using it is a skill that needs to be learned.

For your pandemic example, that’s something that you would need to provide the AI with the context of. The joke of a “prompt engineer” being a job soon actually has merit, in that you want people who know how to use their tools the best. It’s constantly learning through iteration to give the AI a specific instruction set to get the results you want/need.

[–] FutileRecipe@lemmy.world 3 points 14 hours ago (1 children)

Depending on where you go to school, 70% is passing while 50% is not. While "not far off," one is a C, the other a F.

[–] FreedomAdvocate@lemmy.net.au 1 points 1 hour ago* (last edited 1 hour ago)

That’s not at all what this means. In this instance, 70% is basically “human level”. For AI to already get 57% it means that it’s approaching the same level as people do in Excel.

[–] MadMadBunny@lemmy.ca 23 points 21 hours ago (1 children)

So it achieved the actual proficiency of a middle manager…

[–] MonkderVierte@lemmy.zip 4 points 17 hours ago

Decades ago. The company that replaced it's CEO with a LLM thrives.

[–] SkaveRat@discuss.tchncs.de 11 points 21 hours ago (1 children)

Slightly better than Vegas. Unfortunately, plenty of people are okay with Vegas odds.

[–] Imgonnatrythis@sh.itjust.works 7 points 21 hours ago

Not enough accuracy to be useful. Not enough bullshit for politics.