Copy article

Your rogue AI chatbot could bankrupt you and your insurer won't pay out

ended 25. November 2025

When the people whose entire business model depends on calculating risk decide something is too unpredictable to touch, that should make everyone pause.

According to the Financial Times, major insurers including AIG are now asking US regulators for permission to exclude AI-related liabilities from corporate policies. One underwriter's assessment? AI model outputs are "too much of a black box."

This isn't theoretical hand-wringing. Google's AI Overview falsely accused a solar company of legal troubles, triggering a $110 million lawsuit. Air Canada got stuck honouring a discount its chatbot invented out of thin air with some clever customer prompting. Fraudsters used a digitally cloned executive voice to steal $25 million from London engineering firm Arup during a video call that seemed entirely legitimate.

OpenAI is funding its own liability (limited to $200 to end users in the ToS) with venture capital money because it itself is uninsurable.

We want your views:

  • If the people whose entire job is calculating risk won't insure AI enabled workflows, what does that tell us about the technology everyone's in an arms race to adopt?
  • Should companies hit pause on AI deployment when their insurers are explicitly excluding it from coverage?
  • Is this the insurance industry sounding an alarm the tech sector doesn't want to hear, or can't afford to listen to?
  • Are businesses creating uninsurable liabilities they'll only discover when something goes catastrophically wrong?

4 responses from the Newspage community

Copy all

Star Quote
Copy

Colette Mason
AI Ethics Consultant at Clever Clogs AI
Logical, conditional-based automation is brilliant. It's explainable, testable, saves time and money. That's the automation that actually works. But AI automation in processes requiring judgement? That's like pulling the fruit machine lever and hoping you win over and over. Entirely different beast. You could win, but you're far more likely to run out of luck.

The AI dream sellers always overlook that in their cheery "sack 'em, then set and forget" marketing. They're flogging systems even professional risk assessors won't touch.

Businesses are eager to replace staff with AI, but that means ditching their safety net for a tool platform owners won't accept liability for. When it goes wrong, and it will, you're in the firing line, while they point at their terms of service.

Insurance companies aren't being cautious. They're being honest. They're telling us what the hype merchants won't: nobody knows how to price this risk because nobody fully understands what a rogue black box will do.
Copy

Patricia McGirr
Founder at Repossession Rescue Network
When risk specialists back away from AI exposures, every board should pay attention. The pattern here is painfully familiar. Whether it is a mis-sold loan, a broken collections process or a model generating nonsense, the damage always lands on the person least able to absorb it. AI is no different. A chatbot invents a refund policy. A model fabricates legal trouble. A cloned voice drains a company account. These operational collapses create real financial harm, not tech anecdotes.

The same gaps that push borrowers into distress are showing up in corporate AI adoption: no accountability, no clear recovery path and no understanding of who pays when things go wrong. Insurers stepping back is not a niche concern. It is the market telling leadership that transparency and oversight are still miles behind the hype.

Before rolling out AI, boards should ask one question: Are we building resilience or creating the next level of uninsured liability and reputational damage?
Copy

Clive Bonny
MD at Strategic Management Partners
Content creators and designers are playing with fire when using ChatGPT to support their client projects. A single logo or trademark infringement which looks similar to a registered brand can create a claim for £150,000 plus a big slice from historical profits. UK's creatives are responsible for £100 billion trade value. Most are small firms unaware that their insurance will not cover Ai based IP infringements. IP attorneys are now very proactive in identifying infringers. Small enterprises need more online education about risk management before UK loses this massive pool of talent and revenue.
Copy

Simon Bain
CEO at OmniIndex
Most LLMs are fundamentally unsuitable for business due to two critical risks:

Hallucinations: GenAI is engineered for plausibility, not fact. Examples (like Google's $110M lawsuit) prove this unverifiable inaccuracy creates liabilities insurers refuse to cover.

Systemic Data Leaks: LLM infrastructure demands decrypted data uploads, creating a "cesspit" of private information. This gaping security hole leads to massive, uninsurable fines, attacks and breaches.

Companies must stop using unsecure LLMs and ban their use with shadow AI a rising factor in corporate data leaks. Only AI that meets the following two non-negotiable security standards should ever be used:

Verification: Can the AI prove how answers were generated for audit?
Privacy: Can the AI guarantee data is safe and private at all times?