M365 Copilot Fails to Up Productivity in UK Government Trial
upstart writes:
A UK government department's three-month trial of Microsoft's M365 Copilot has revealed no discernible gain in productivity - speeding up some tasks yet making others slower due to lower quality outputs.
The Department for Business and Trade received 1,000 licenses for use between October and December 2024, with the majority of these allocated to volunteers and 30 percent to randomly selected participants. Some 300 of these people consented to their data being analyzed.
An evaluation of time savings, quality assurance, and productivity was then calculated in the assessment.
Overall, 72 percent of users were satisfied or very satisfied with their digital assistant and voiced disappointment when the test ended. However, the reality of productivity gains was more nuanced than Microsoft's marketing materials might suggest.
Around two-thirds of the employees in the trial used M365 at least once a week, and 30 percent used it at least once a day - which doesn't sound like great value for money.
In the UK, commercial prices range from 4.90 per user per month to 18.10, depending on business plan. This means that across a government department, those expenses could quickly mount up.
According to the M365 Copilot monitoring dashboard made available in the trial, an average of 72 M365 Copilot actions were taken per user.
"Based on there being 63 working days during the pilot, this is an average of 1.14 M365 Copilot actions taken per user per day," the study says. Word, Teams, and Outlook were the most used, and Loop and OneNote usage rates were described as "very low," less than 1 percent and 3 percent per day, respectively.
"PowerPoint and Excel were slightly more popular; both experienced peak activity of 7 percent of license holders using M365 Copilot in a single day within those applications," the study states.
The three most popular tasks involved transcribing or summarizing a meeting, writing an email, and summarizing written comms. These also had the highest satisfaction levels, we're told.
Read more of this story at SoylentNews.