TEARDOWN Published 21 November 2026 at 08:16. Evidence-based. Source-cited. No sponsored content.

National AI guidance for social care rests its proof on one council's 63% time-saving pilot. It never mentions the only study of that exact trial, which found the tool misspelling names and inventing details.

4 out of 5 stars4/53 documented mistakes in this teardownHow ratings work

Estimated reading time: 6 minutes

Peterborough Town Hall on Bridge Street, a stone civic building with a Union flag flying from its facade.
Peterborough Town Hall, where Peterborough City Council, developer of the "Hey Geraldine" AI assistant cited in the guidance, holds its council meetings, 8 January 2015. Photo: Julian Dowse / Wikimedia Commons, CC BY-SA 2.0.

In short. The Department of Health and Social Care and the Department for Education published "AI in social care" on 20 November 2026, billed as the first cross-sector AI guidance for children's and adult social care [[1]]. Its headline case study, "AI in Swindon Borough Council's Adult Social Care", credits a pilot of the Magic Notes tool with "a 63% reduction in admin time" and a Care Act Conversation time that "fell from 1.5 hours to 30 minutes" [[1]], a figure the accompanying press release repeats [[2]]. The guidance names one source for this: Swindon Borough Council's own article [[1]]. That article actually gives the figure as "35 minutes", not 30 [[3]], and dates the trial to April to July 2024, 19 social workers across 184 meetings, with no mention anywhere in the guidance of either the date or the scale [[3]]. The one outside look at this exact pilot, commissioned by Beam, the tool's vendor, and reported in trade press on 10 February 2025, found practitioners encountering misspelt names, invented detail and a tool that struggled in meetings covering more than one case [[4]], a category of risk the guidance warns about only in the abstract, never alongside its own case study. Three documented mistakes: four stars.

Social care in England runs on paperwork. The Department of Health and Social Care's own press release puts the stakes in plain terms: AI "can free up care staff to spend more time supporting people and families by cutting paperwork", and the guidance itself says nearly 90% of people receiving care are already covered by some form of digital record [[1]][[2]]. On 20 November 2026, DHSC and the Department for Education published what they called the first cross-sector guidance covering both children's and adult social care, a single document meant to tell tens of thousands of practitioners and dozens of local authorities how to adopt AI safely [[1]].

The document itself is not a thin one. It runs to an annex of ethics and risks, a section headed "Accuracy and hallucination" that warns AI "produces incorrect (but often plausible) or incomplete content, including misunderstandings of transcripts or context", and a bias section that states plainly: "research has found some AI tools produce different summaries depending on individual characteristics, such as gender" [[1]]. Under "Main points", near the top of the document, it states that "AI is not infallible" and that "practitioners should challenge AI outputs" [[1]].

None of that caution appears anywhere near the two case studies the guidance and its press release actually lean on to make AI's case.

The claims, tested

What the guidance says What the evidence shows Verdict
The case study "AI in Swindon Borough Council's Adult Social Care" lists "faster assessments" and states "the time to conduct a Care Act Conversation fell from 1.5 hours to 30 minutes", naming its source as "the news article: Council completes AI trial to enhance Adult Social Care services, Swindon Borough Council" [[1]]; the press release repeats "from 1.5 hours to just 30 minutes" [[2]] Swindon Borough Council's own article, the guidance's only named source for this figure, states the time "dropped from one hour 30 minutes to just 35 minutes" [[3]] The national guidance's headline statistic does not match the one source it points readers to for that statistic
The case study presents the 63% admin-time reduction and the assessment-time figures with no date attached, as supporting evidence for guidance billed as new and "first ever" [[1]][[2]] Swindon's own article dates the pilot to "between April and July 2024", involving "19 social workers and a single leadership support officer" across "184 frontline meetings" [[3]], two and a half years before the guidance that cites it The guidance's sole outcomes evidence is both a single council's small trial and, by publication, old, and discloses neither fact
The guidance warns in general terms that AI "is not infallible", can "hallucinate" and may "misunderstand... context" [[1]], without naming a tool this has happened to The only outside look at the exact Swindon pilot, commissioned by Beam (the tool's vendor) and conducted by Alan Turing Institute researchers across Swindon, Barnet and Camden, found practitioners reporting "name misspellings, repetitions and assumptions" and a tool that "struggled" whenever "multiple different clients" were discussed in one meeting [[4]] The specific tool named in the guidance's own flagship case study is the one independently shown to produce the exact failures the guidance only warns about in the abstract

A pilot doing a lot of work

The 63% figure is not a small part of the guidance's pitch. It appears in the document itself, in the ministerial press release, and in the trade coverage that followed [[1]][[2]]. Minister of State for Social Care and Mental Health Alison McGovern said the national guidance "will clearly set out how providers can best adopt AI to improve the care patients receive", with the Swindon figures as the lead evidence for that claim [[2]]. Children and Families Minister Josh MacAlister made a similar case for children's services, built on the same release [[2]].

What the guidance does not tell a reader is that the Swindon trial has already been looked at once before, by people other than the council that ran it or the company that sold it the tool. Community Care, a specialist outlet covering the profession, reported on 10 February 2025 that researchers Rob Procter and Stanislav Zhydkov of the Alan Turing Institute had evaluated Magic Notes across Swindon, Barnet and Camden: 91 staff, a trial running April to September 2024, with 11 in-depth follow-up interviews [[4]]. The research was commissioned by Beam, the company that makes and sells Magic Notes, not by DHSC, DfE or any of the three councils [[4]]. Even so, it found real problems. One practitioner told the researchers: "It's not perfect yet. I wouldn't just copy and paste a summary straight over because sometimes it makes assumptions. I think it can't pick up on nuances." Another: "Sometimes it can add in detail based on what's not been discussed", calling the tool "presumptuous" [[4]]. A third practitioner reported the tool failing outright in team meetings: "When you're talking about multiple different clients, it doesn't capture all the information" [[4]]. In adult social care, a summary that invents detail or loses track of which client is which is not a cosmetic bug.

Credit where due

The guidance is not a thin sales document. Its Annex A sets out hallucination, bias, automation bias and over-reliance as named risks with named mitigations, states outright that "AI is not sentient and does not understand the personal contexts of a child, family, individual or their lived experiences", and is explicit that "AI should not replace practitioners' judgement or make automated decisions about risk, need, eligibility or intervention" [[1]]. Its first principle is that "practitioners remain responsible for their casework, recording and decision-making" and that AI output must be "reviewed, challenged and amended" [[1]]. The Peterborough "Hey Geraldine" case study is also a modest, specific claim, "answered over 1,200 questions and saved around 15 minutes per interaction", credited plainly to in-house development rather than oversold as transformative [[1]]. And the guidance names its sources precisely enough, the exact council, the exact tool, a direct link, that this piece was possible to check at all; a vaguer claim would have been harder to test.

Three documented mistakes. (1) The guidance's case study and the accompanying press release both state the Swindon Care Act Conversation time "fell from 1.5 hours to 30 minutes", but the guidance's own named source, Swindon Borough Council's article, gives the figure as 35 minutes. (2) The case study presents its 63% figure and time savings with no date attached; the pilot it describes ran from April to July 2024, 19 social workers across 184 meetings at a single council, two and a half years before this "first ever" guidance was published, and the guidance discloses neither the age nor the scale. (3) The guidance warns in general terms that AI can hallucinate and misunderstand context, but never mentions that the exact tool in its own headline case study was found, in the only outside evaluation of that pilot, to misspell names, invent unstated detail and fail in meetings covering more than one client. Rating: four stars.

Sources

  1. Department of Health and Social Care / Department for Education, "AI in social care", 20 November 2026
  2. Department of Health and Social Care / Department for Education, "New AI guidance to cut paperwork and free more time for care", 20 November 2026
  3. Swindon Borough Council, "Council completes AI trial to enhance Adult Social Care services", 1 October 2024
  4. Community Care, "AI tool improves direct work in adult social care despite accuracy concerns, practitioners report", 10 February 2025
SHARE THIS ARTICLEXBlueskyMastodonLinkedInRedditEmail