NHS warned over use of AI to treat patients with depression

Using artificial intelligence (AI) to treat depression could backfire on the NHS and leave patients more vulnerable than ever, AI experts have claimed.

Industry figures have sounded the alarm over relying on AI to help fix the UK’s mental health crisis, following the government announcing it is investing £3.6m across 17 mental health tech projects.

Smart glasses, which help people struggling with severe depression to complete everyday tasks and AI filter apps making therapy for debilitating anxiety less daunting, are among the cutting-edge tech projects the government hopes will ease pressure on the NHS.

Lord Vallance, Science Minister, said: "These projects are shining examples of how innovation can transform people’s quality of life, by helping those with severe mental health conditions to take on everyday tasks that can otherwise feel impossible.

“From smart glasses helping those with debilitating depression to navigate through the day to games helping children to build their social skills, we are supporting teams across the UK to build cutting-edge tech that unlocks opportunity, supports the NHS and grows our economy.”

But Colette Mason, founder and AI systems architect at Clever Clogs AI, said the tech on which the products and systems were founded remained untested on a large scale group of users with severe mental illness.

She said there was also an issue of 'hallucinations', creating genuine risks for vulnerable users. 

Mason said data protection issues could also arise: “Data governance appears overlooked. These devices continuously monitor eye movements, voice patterns and location data, with comprehensive psychological profiling requiring explicit GDPR consent and protection assessments. Yet funding announcements remain silent on these safeguards.”

Another fear is that NHS staff may have to pick up the pieces if the tech malfunctions. Mason added: "The greatest risk is burden transfer. When glasses malfunction during mental health crises, front-line NHS staff inherit problems they didn't create to services already strained by a 40% rise in mental health demand."

Mitali Deypurkaystha, founder/CEO at Newcastle upon Tyne-based Impact Icon AI said, if used well, the devices would help patients. 

She urged against rushing in without guardrails including data protection and against over-simplified care.

“Without NHS-funding and clear duty-of-care, we dump the burden on schools and families. “Simple test: does it lighten the load for clinicians and carers or not? They should be an extra layer, not a replacement.”

“Safety must come first: parental consent, school safeguarding and clinician oversight are non-negotiable. They could be a lifeline for people stuck on long waiting lists. But unless the NHS designs with inclusion in mind, some demographics risk being left behind."

Baroness Merron, Minister for Mental Health, said: “Technology is transforming healthcare and we are putting patients at the heart of this revolution.”

“By embracing new tech, we’re improving lives and reducing pressure on the NHS to make healthcare fit for the future, as part of our Plan for Change.

CrossSense said its technology helps to reframe negative thoughts while offering step-by-step instructions for carrying out daily tasks, such as keeping vulnerable users safe from harm with prompts like advice to stay away from boiling water on the hob. 

It adapts to the wearer’s needs over time and helps to prevent cognitive decline, including dementia, so complex mental health conditions are not an unnecessary barrier to fulfilled and independent lives.

Szczepan Orlins, CEO CrossSense, said technology is evolving. He said: “There's still a potential for some hallucinations [but] there's a range of things that we are doing about this.”

Orlins said patients who used the glasses would be able to “build some kind of mental health resilience so if the glasses malfunction then the patient will have already developed some coping strategies.”

He said ultimately the roll out of AI would prevent other long-term conditions which were placing a huge stress on the NHS.

“Depression increases the chance of getting dementia by 50% chance. So there are things that will help the NHS hugely if the software works.”

Please credit Samantha Downes.