Microsoft says its Agent Mode in Excel has an accuracy rate of 57.2 percent in SpreadsheetBench
Agent Mode in Word turns document creation into vibe writing
“Productivity is our DNA, we’re Office,” Chauhan says. “While others will try to replicate us, there is no substitute for the real thing.”
Somebody vibe kill me please.
Evidently, the benchmarking is based on a set of 912 user questions covering a range of scenarios. I have to assume that a decent chunk of these are covering situations where the original spreadsheet was set up poorly or excel isn’t the ideal platform for the desired output. Given that, I wonder if the accuracy metrics even mean anything or if it’s just a measurement of how well the software pounds square pegs into round holes.