Are financial firms ready for the AI they are using?

AI can be useful in specialist finance, where a single case can involve valuations, company accounts, leases and legal documents. Helping someone find and organise the relevant information has obvious value.

The concern is when a summary becomes an accepted fact without anyone checking the original. A missed qualification in a valuation or an incorrect reading of a lease can change how a deal looks.

Many borrowers approach specialist lenders because their circumstances need individual consideration. Firms need to preserve that judgement as they introduce more automation.

As a small publisher using AI, I’ve found that a confident answer still needs checking. In lending, someone needs to own the decision, be able to explain it and give the borrower a practical way to challenge an error.

AI is already transforming financial services, from mortgage processing and fraud detection to customer service and financial planning.

But here's my concern: financial firms are racing to automate decisions before proving they can control them.

AI can improve efficiency, but it can also misunderstand someone's circumstances, reproduce bias or give inappropriate answers to vulnerable customers.

The biggest danger is allowing technology to replace judgement and accountability.

At Roxton Wealth, we're investing heavily in developing our own technology and AI-powered systems. But we're building around our processes, compliance requirements and client needs, not simply handing decisions to a chatbot.

AI should make financial services more human, not less.

Customers shouldn't be worried about firms using AI. They should be worried about firms using it without proper oversight, transparency and accountability.

The firms that get this right will have a significant advantage.

Firms that use AI to handle tasks, allowing them to spend more time with their clients, will be the winners going forward. However, this doesn't mean putting your entire client base into a £20-a-month subscription. A policy, a strategy, and guardrails are needed to ensure client data is secure and functions aren't compromised. Staff also need training to highlight where fraudsters may use AI, including voice manipulation, to gain access to client information and funds.

AI has enormous potential in financial services, from summarising documents and handling administration to fraud detection and helping advisers work through huge amounts of information. The danger comes when the helping quietly becomes decision-making. Mortgages, protection and investments often involve messy human circumstances that don't always fit neatly into an algorithm, particularly where customers are vulnerable.

Firms need proper human oversight, testing for bias and clear accountability for every important decision. If AI declines someone, flags them as risky or recommends a course of action, there needs to be a human being capable of understanding and challenging why. Customers shouldn't be frightened of AI, but nobody wants to explain their financial life to a chatbot that has plenty of answers but very little understanding.

New starters have replied to me with the AI prompt left in. To me that's the biggest risk in financial services too: staff who stop checking what the machine hands them. The FCA has flagged that human review of AI can become box-ticking rather than real oversight. A check that just waves the answer through is worse than none, because everyone assumes someone looked. With no special AI rules, keeping it in check comes down to what I'd ask of any number: who signed it off, and do they understand it? The UK Finance and PwC report is right that firms should measure how often human reviewers challenge the AI's answers, because a rate falling towards zero may be a warning sign, not a success. Customers don't need to fear the technology itself, because the firm and its senior managers remain responsible for the outcome, whoever supplied the AI. If what a bot tells you about your money doesn't sound right, ask for a person.