Microsoft says its Agent Mode in Excel has an accuracy rate of 57.2 percent in SpreadsheetBench
Agent Mode in Word turns document creation into vibe writing
“Productivity is our DNA, we’re Office,” Chauhan says. “While others will try to replicate us, there is no substitute for the real thing.”
Somebody vibe kill me please.
Why would you deliberately ship a feature that has a 57% accuracy rate? Are the engineers desperate to appease Ed Zitron’s Business Idiots or are have they awakened to the realization that most people with computers don’t do productive labor, and anyone who embraces these features probably had a worse-than-57% accuracy rate to begin with?
Gotta pump those usage numbers up. Who cares about how useful it actually is?
The Cult of the Next Quarter has finally succeeded in summoning its deity
You also gotta imagine that they tailored the implementation to succeed in known benchmarks and they still got 57%
Evidently, the benchmarking is based on a set of 912 user questions covering a range of scenarios. I have to assume that a decent chunk of these are covering situations where the original spreadsheet was set up poorly or excel isn’t the ideal platform for the desired output. Given that, I wonder if the accuracy metrics even mean anything or if it’s just a measurement of how well the software pounds square pegs into round holes.