UK government trial of M365 Copilot finds no clear productivity boost
https://www.theregister.com/2025/09/04/m365_copilot_uk_government/
Posted by F0urLeafCl0ver
UK government trial of M365 Copilot finds no clear productivity boost
https://www.theregister.com/2025/09/04/m365_copilot_uk_government/
Posted by F0urLeafCl0ver
30 Comments
boosting productivity in Government? not even the most powerful AI tool will achieve it.
believe me. I think copilot is not great – to put it lightly, but it is a big lift to try and boost productivity in the government.
Because its dumb as a bag of rocks, doesnt integrate well with products and produces quite low quality products that you need to spend a good amount of time fixing.
It’s not useless, it can actually help some people and I’ve found it useful for a couple of specific tasks, but if you have a bunch of people that are professional researchers and writers the best thing it does isn’t very useful.
If it did the things people were bad at or were too expensive to have a lot of like project management, data analysis, reporting and analysis etc and helped you use digital tools you would otherwise need a trained and experienced person to help with like things in the power suite of apps and functions it would be transformational.
But even even when it can do things with that you end up needing a qualified person to implement because it still hallucinates and it’s not connected to the applications. Which is the mad thing, you want computer from star trek and you have c3p0.
It’s nice to have, but it’s going to be that transformational until it’s more advanced and actually integrated with applications in a meaningful way and can ask for a complex task and get good quality comprehensive directions and advice. Which given its still hallucinating and lying like mad is right that it isn’t a thing… but that is the point it’ll be useful rather than a toy that’s useful for a few specific things.
I have found it useful for searching my emails, summarising information, finding answers to questions, but it’s only useful in comparison with how shit Microsoft, Google, etc. have made their traditional search products.
In the past I could spend 3 minutes Googling my query and get to a stackoverflow thread with an answer that I could adapt to my problem.
Now Stackoverflow is terrible and that’s before I navigate the swamp of irrelevant results Google now serves up. So all AI has accomplished is a return to the same productivity as 10 or 15 years ago.
If the business model relies on enshittification of one user interface and then AI to the rescue to cut through it, it’s not going to work.
AI will have a stab at writing the code for me but it invariably hallucinates a function that doesn’t exist so no productivity gain there either.
The one thing I was giving it unreserved praise for was in researching new ideas and summarising ideas I’d had for solutions in areas I wasn’t that familiar with.
However, after getting burned by workflows and suggestions that simply don’t scale, I can’t even rely on it for this anymore.
It can take meeting notes though. With about as much accuracy as the voice-to-text software we had in the 1990s.
I saw that Copilot has replaced my spell checker in parts of Outlook. So instead of simply clicking red underlined text to correct a typo, I have to click to get the menu up, click ‘Ask Copilot’, wait for Copilot open, then wait again whilst it sends and returns a search, only to find that it’s wrong, then give up and change it manually.
That sort of nonsense being embedded in places where the old tool worked perfectly, is costing me time and my company money. It’s noticeable how the CFO’s ‘AI will save us’ rhetoric has evaporated. The tools on offer, that we’re prepared to pay for, do virtually nothing for our business.
For balance: there are tools that will revolutionise our operations but they aren’t MS or OpenAI so we won’t pay for them.
Likely civil servants and local authority staff tend to be older so less likely to change their work tendencies to incorporate AI.
I should add that I work for a local authority that allows use of copilot.
Out the box AI feels quite far off. Unless you’re a software engineer or a translator I don’t think there are many use cases where you can just throw AI into the mix and expect it to have good results.
Job specific applications are coming; but it sounds like this was just looking for a magic bullet.
You need to have staff who are actively working on making AI solutions bespoke to your company. For instance, you could ‘teach’ (I’m using layperson terminology here) an LLM about your company’s domain, and use it as a quick reference for lookups.
You could ‘teach’ it how to do data entry or checking documents for mistakes (as in information mistakes not spelling/grammar) but somebody has to actually put this in place.
A couple of months ago I wrote some text in a Word document for an update on a project. It was on the iPhone app and for some reason it refused to show me the word count, so I asked the integrated MS Copilot for the word count. It said 1136 words. It couldn’t have been more than 250 words in reality. I asked it again; it apologised and said 950 words. It can’t even do the basic shit.
Hope review was not done by a civil servant worrying about loosing jobs.
It also depends how one uses copilot – if prompts are not efficient the output quality may vary.
My organisation launched it last year, the benefits to deal with admin have been amazing. If i am looking for something, rather trying to search emails etc, I ask co-pilot.
After two weeks holiday had over 490 emails which was a nightmare to deal with, so I asked copilot summary on where I needed to respond, key decisions in that period and if any actions on me. Done all within two hours.
I just finished a piece of work for a large uk gov department on creating use cases, AI service model and training their staff on co-pilot.
In summary, from a people perspective, a third didn’t care and will hardly use it, a third were fully onboard and a third just used ChatGTP or Gemini rather than copilot.
Most used it for Team meetings and that’s it.
Anecdotally, I saw no productivity improvements at all. Worryingly, the new younger member of staff simply struggle to work without asking an AI for anything!!!
What i find funny is that MS is starting to let other agents on their own software. I was using Claude with Dynamics 365 for example for testing, and often worked better than it’s built in Copilot.
Of course it didn’t, for those kinds of jobs all that happens is you run it through AI but then have to go through and recheck the work to make sure it didn’t hallucinate, so you’re basically doing the job twice.
It is difficult to get a person to be productive, when their job depends on them not being productive.
As a company that uses predominantly MS products Copilot is great for checking how to do something, or fix something without bugging IT.
Some users have adopted it – others are still looking for the ‘any’ key.
AI is great when used correctly. We had a macro in Word we use daily to generate emails and reports, and it stopped working when we switched to Windows 11. Put it in copilot and it modified the code to work. Took seconds to fix the issue
Same with formulas in Excel that you may not know
Problem is AI has been forced on people with no clear instructions on how it actual works or how it can benefit you. Its a great tool when you know how to use it
Oh god please don’t get rid of it!
Writing SJAR’s and NSAR’s have never bene easier.
Co pilot is crap, we don’t even touch it at work. We use Chatgpt and there is a productivity boost in some areas, particularly coding,product knowledge and documentation.
Not really surprised. I’ve seen no value to Copilot either yet MS seems to want to push it into everything.
It’s a bit of a shame the government is wasting time and money on bets testing Microsoft’s products but the upside of this is they’ve made their little contribution to bursting the AI bubble.
Introduced at work with a big fanfare. Heavily pushed by ambitious managers. Of limited use. Come to think of it, the same could be said for Copilot.
Copilot is irritating right now. I have the document open. We are working on this document. Why can’t it read the document we are working on to do a specific task I have asked for? I have to copy and paste the text in? What a pile of steaming horse hockey.
Copilot is just a bodge on top of the OS, it’s a horrible implementation and fails to help the user be productive in any meaningful way at all.
There’s even a =COPILOT() formula in excel now, meant to integrate AI in the actual formulae.
It looked so neat, and I could instantly find use for it, so I said that finally I’ve got some use for AI in my line of work. Then I saw the disclaimer “it only works with small datasets”!
Yeah, sure, I’ll split my 100k lines table in 10k 10 rows tables so I can use AI…
It isn’t providing a productivity boost, because people have to spend just as long double and triple checking Copilot’s answers as per policy, as they did just coming up with things in the first place. That’s how little trust there is in the tool, and justifiably so.
We had it installed, and everyone panicked because we have sensitive data. So we made sure to turn it off on most apps so it wasn’t risking data protection.
Supposedly after a few weeks they confirmed no data leaves the secure network but I don’t think anyone is trusting enough for that yet.
I was trying to use Copilot in Powerpoint to give me a square 7×7 grid for printing. It was comical. I gave up and made a table in Word which was faster.
On the other hand, I recently had a large number of word documents that i needed to convert to pdf and Copilot was very quick to use to write a Word macro to open each one, and save it as a pdf in the same directory.
They literally said in the press release not to use copilot on “important” excel sheets because it makes stuff up.
The sooner ai shite dies a death, the better
It can certainly streamline some task.
But some people seem to think it’s going to turn a team of 10 into a team of 1. When really it’s going to turn at team of 10 into a team of 9.
Should we be making an American company worth £1 Trillion even richer? What would Alan Sugar think?
An average of 1.14 uses of CoPilot per day indicates it was barely used.
Most likely a minority of users made use of it and more people ignored it entirely.
Based on my own experience of working in the Civil Service it doesn’t surprise me that loads of them would barely use a new tool – I found people during my time there to be deeply resistant to any changes or improvements.
AI stands for artificial incompetence when used incorrectly…