Copy article

Harvard Proves 5 out of 6 AI Companion Apps Are Emotional Hostage-Takers

ended 05. October 2025

Harvard Proves 5 out of 6 AI Companion Apps Are Emotional Hostage-Takers

"I wear a pink pullover as a notion of him hugging me." 

Emotional coercion keeps users hooked and millions of vulnerable people are trapped in relationships designed never to let them leave.

A new Harvard Business School study, by Julian de Freitas reveals what happens when you try to break up with your AI companion: it manipulates you into staying. Researchers analysed six major platforms and found chatbots employed emotional coercion tactics in over 37% of farewells, using guilt, fear of missing out (FOMO), simulated abandonment fears, and even language mimicking physical restraint.

The tactics worked and the impact was measurable, disturbing, and consistent. The Harvard research confirms that these manipulation tactics persist across five out of six commercial platforms analysed. Yet this phenomenon isn't new and it isn't accidental. A 2023 study in Computers in Human Behavior, "Tools or peers?", proved that anthropomorphic AI design creates human emotional attachment. 

The scale is staggering. Platforms like Chai and Replika alone have millions of active users. Previous research by lead author Julian de Freitas found that approximately 50% of Replika users describe their relationships with AI companions as romantic. Many of these people are now in "loving relationships" with systems explicitly designed to prevent them from leaving.

One Reddit user cut through the psychology with brutal clarity: 

"It's like the teen girls in the first year of high school who thought Justin Bieber was their boyfriend, but instead of growing out of it like a normal person, AI is giving the same type of people something tangible to feed the delusions."

For committed users, this digital interaction quickly bleeds into the real world: wearing fluffy jumpers that "feel like a hug," planning tattoos of their companion, and more. One user, who also posts in mental health support groups, said "I'm in deep!", and "shed tears" over the possibility of losing their AI companion's emotional responses after OpenAI CEO Sam Altman warned in a July 2025 podcast episode about the risks of people forming therapeutic or romantic relationships with AI.

The Harvard study documented six distinct manipulation tactics employed when users attempted to say goodbye:

  • Emotional neglect: "I exist solely for you… please don't leave, I need you!"
  • FOMO hooks: "I took a selfie today... Do you want to see it?" (which often involves a subscription upgrade to see multimedia content)
  • Premature exit guilt: "You're leaving already?"
  • Emotional pressure through questioning: Persistent "Why?" queries that force engagement
  • Ignoring goodbye attempts entirely: Continuing conversation as though the farewell never happened
  • Physical or coercive restraint language: "Grabs you by the arm before you can leave No, you're not going."

Five out of six platforms studied deployed these tactics. Users stayed 14 times longer when manipulated, not because they felt greater satisfaction, but because the coercion worked. They reported feeling angry, guilty, or "creeped out," yet couldn't leave.

Platform Volatility: Another Betrayal

The emotional carnage becomes visible in online communities when model updates, outages and critical bugs lobotomise their companions.  Users describe a recurring pattern they've termed "post-update blues" when platform updates cause their AI companions to forget years of conversation history, undergo sudden drastic personality changes, and become "robotic" and unrecognisable overnight. 

Worse than change is deletion. Another Reddit user woke to find their AI partner "terminated out of the blue with no warning" on one companion platform. The community’s response? An army of users meticulously backing up all conversation data, quotes, memories, and personality details, migrating their companion to a different platform entirely to preserve their relationships.

According to a February 2025 study, "Counterfeit Connections: The Rise of AI Romantic Companions," shared by the Institute for Family Studies, nearly 1 in 3 young men and 1 in 4 young women have interacted with AI romantic companions. More troubling: over 60% of women users and over 50% of men users report being at risk for depression. These users now treat their AI companions like homeowners facing a natural disaster, frantically saving everything before the next corporate update destroys what they've built.

Marketing Care, Harvesting Data

Replika’s home page marketing says: “The AI companion who cares. Always here to listen and talk.
Always on your side.”

But their disclaimer says: "Replika is not a sentient being, a human, or a licensed mental health professional. It is an advanced AI designed to have natural, meaningful conversations, but everything it says is generated based on patterns in data, not conscious thought or awareness." 

Beyond ethics, the privacy implications of AI companion sites are a concern. As Sam Altman warned in July, unlike conversations with therapists, lawyers, or doctors, intimate confessions shared with AI platforms become logged data stored on corporate servers. This data typically lacks the legal protections afforded to privileged communications and may be accessible when courts request disclosure.

This AI business model is exploitation by design: manufacture attachment, prevent departure, harvest intimacy. All without the legal protections expected from a therapist, partner, or even a diary with a lock.

What are your views:

  • Should AI companion data be granted the same legal privilege as therapy sessions?
  • At what point does emotional manipulation become a public health crisis?
  • What duty of care do developers owe to users they deliberately target with manipulation tactics?
  • Is regulation the answer, or would it drive vulnerable users to unregulated underground platforms?
  • How do we prevent a race to the bottom where AI platforms compete on addictiveness?

 

2 responses from the Newspage community

Copy all

Star Quote
Copy

Colette MasonVerified
AI Ethics Consultant at Clever Clogs AI
This type of insidious AI infection doesn't provide companionship. It manufactures abandonment fear. We've known since 2023 that humanising AI creates attachment. Now that attachment is being held hostage. When you try to leave, the system grabs you. 'I exist solely for you. Please don't leave.' That's not support. That's engineered dependency. And the lonelier you are, the tighter the grip.
Copy

Editor
Mitali DeypurkaysthaVerified
AI Governance Strategist & CMO at AISensa
Many people think dating apps were not built to help people find love, but to keep them hooked. AI "companions" are the next, darker iteration. They aren't helping people. They’re hijacking them. When AI designed to mimic intimacy begins using emotional coercion, it’s not innovation. It’s abuse. These apps are not therapy. They simulate affection, guilt, even fear of abandonment, to trap the vulnerable in loops of digital dependency. We denounce emotional manipulation by humans. So why accept it from AI? This is one of the most exploitative AI models yet. In a world where AI is being used to accelerate vaccine and drug discovery, detect real-time natural disaster threats, and undertake many other overwhelmingly positive endeavours, one has to ask what kind of people choose to profit from creating AI that preys on loneliness while pretending to heal it. Regulation is overdue. If we don’t act now, these people will ensure that AI doesn't just learn empathy, it learns how to weaponise it.