Your AI Customer Service Bot Is About To Cost You A Lawsuit
Air Canada lost. Klarna reversed. DPD's bot wrote a poem about how terrible the company was. Here's how to deploy AI customer service without becoming the next headline.
In 2024, Air Canada lost in tribunal because its support chatbot invented a bereavement-fare policy that didn't exist[1]. The airline argued — out loud, in court — that the chatbot was "a separate legal entity responsible for its own actions"[2]. The judge disagreed. They paid the refund and ate the precedent.
That ruling wasn't a one-off. It was the opening shot.
Klarna spent eighteen months telling the press its AI assistant did the work of 700 customer service agents[3]. In May 2025 the CEO quietly admitted the quality wasn't there and started rehiring humans[4]. DPD's bot called the company "the worst delivery firm in the world" — in a poem, unprompted-ish, after a customer goaded it[5]. The clip hit 1.3 million views before DPD pulled the bot offline.
I've watched a steady drip of operators in the $1M–$20M range rush an AI support agent into production this year because their cost-per-ticket math looked beautiful in a spreadsheet. Then a customer screen-shots a hallucinated refund policy and posts it. Or the bot promises a discount the business has to honor. Or it just confidently sends people to a 404.
Here's what's actually happening, and how I'd deploy AI support without turning it into a liability event.
Why most AI support deployments are blowing up
The math everyone uses to greenlight these projects is roughly: AI handles 70% of tickets at $0.10 each instead of $5, so we save $X per month. What the math leaves out is the cost tail of the 30% where it goes wrong — and the "handle 70%" part is wildly optimistic for the deploys I see.
Three things are driving the failures.
1. The "handles everything" deployment. Real-world Zendesk AI resolution rates land at 20–40% for typical initial deploys and 60–80% only for well-optimized setups — UrbanStems hit 39%, Lush reached 60%[6]. Operators who quote a single big "70% automated" number are usually counting deflections that didn't actually resolve the customer's problem. Counting deflection-as-resolution is how you ship a bot that lies politely and call it a win.
2. There's no architecture stopping the hallucination from reaching the customer. Testlio's 2025 AI Testing report found 82% of production AI bugs trace back to hallucinations, and 39% of customer-service bots got pulled or significantly reworked the same year[7]. Almost all of those teams had skipped the boring middle layer: retrieval grounded in their actual policy database, plus a confidence threshold that hands off to a human when the model isn't sure.
3. The legal exposure is now real. The Air Canada ruling didn't just say "the airline is liable." It said the airline is liable because it can't reasonably claim the chatbot is a separate entity[2]. Translation: every promise your bot makes is a promise you made. If it invents a return policy, you honor it or you litigate. SurveyMonkey's 2025 study found 79% of Americans actively prefer talking to a human over an AI agent[8] — so when this goes wrong, it goes viral fast.
The combination is brutal: cost savings of $40K/month, one $200K hallucination tail-risk event, plus a brand hit you can't unscramble.
What an actually working deploy looks like
I won't pretend this is one-size-fits-all. But for a typical $5M–$20M ecommerce or services operation, here's the stack I'd build instead of "drop a chatbot on the homepage and pray."
Layer 1 — Triage, not answer
The first AI touch shouldn't be answering the customer. It should be classifying them. Three buckets:
- Transactional (order status, return label, password reset) → AI handles end-to-end. This is the band where the well-optimized 60–80% resolution rates actually show up[6]. Plug it in and let it ship.
- Policy-bound (refunds, exchanges, exceptions) → AI gathers context, then hands to a human with the case pre-filled.
- Emotional / complaint (anger, threat, grief, legal language) → straight to human, no AI rephrase, no upsell, no "I understand how frustrating that must be" autoreply. Detection is a 30-line classifier.
That single split kills 70% of your hallucination risk. The bot is no longer answering refund-policy questions in natural language. It's answering "where is my order" in natural language and routing everything else.
Layer 2 — Grounded retrieval, not free generation
For the questions the bot does answer, ground every response in your actual help center plus your live policy database via retrieval-augmented generation. Two non-negotiables:
- The model can only cite sources that exist. No "according to our 30-day policy" unless that string appears in the retrieved document.
- If the retrieval score is below threshold (I use 0.78 cosine similarity as a default starting point — tune from there), the bot says "Let me get a human on this" and escalates. Not "I'll do my best to help" — escalates.
This is the layer Yuma.ai's team correctly calls "quality control architecture"[9] and it's where every blown deploy I've reviewed had a gap.
Layer 3 — Confidence-gated handoff
The bot publishes its confidence score on every response, internally. Above 0.85 → ships to customer. Between 0.70 and 0.85 → ships but copies a supervisor inbox for spot-review. Below 0.70 → human takes over before the customer sees the reply.
Klarna's course-correction here is instructive. They didn't kill the AI. They tightened the confidence thresholds and reinstated humans for high-complexity cases[4]. Same model, smaller scope, fewer fires.
Layer 4 — Logging, replay, and a kill switch
Every conversation logged. Every escalation tagged with the reason. A weekly review where someone reads 50 random transcripts and flags the ones that came close to inventing policy. A literal kill switch on a config flag — when (not if) something goes viral, you can turn the bot off in 30 seconds while you patch.
DPD's mistake wasn't that their bot misbehaved. Every model can be jailbroken. Their mistake was the lag between the viral tweet and the kill switch[10]. If the kill switch is a config commit and a 6-hour deploy pipeline, you don't have a kill switch.
The number that should drive the decision
A hallucination in customer service costs roughly a few thousand dollars per event in Forbes' framing[11] — but that math is for the obvious case (a refund honored, a discount eaten). It doesn't price the reputational tail, the regulatory attention Klarna got for misleading framing[12], or the brand-trust erosion when 79% of your customers already preferred a human[8].
The number I'd actually run the deployment against: what's the cost of your worst possible week if the bot goes off-script and you're slow to pull it? If that number is bigger than 18 months of customer service salaries, you don't have a positive-expected-value automation. You have a slot machine.
What I'd do this week if you're already deployed
If you're already in production with an AI support agent and you've been getting away with it:
- Audit the last 200 escalations. Count how many were policy-bound questions the bot tried to answer. That number should be near zero.
- Add the emotional/complaint classifier. One day of work, kills your viral-fire risk.
- Lower your confidence threshold by 10 points for two weeks and watch your CSAT. If it goes up (it usually does), keep it there.
- Write the kill switch. Not a runbook. A config flag tied to a feature flag service. Someone with phone access at 11pm Saturday should be able to flip it.
Most teams I see skip step 4 because it's not sexy. Then a chatbot writes a poem about how terrible their company is and they spend Monday morning calling their PR firm.
If you want this audited on your stack — actual transcripts pulled, actual failure modes mapped, actual retrieval scores measured — that's the kind of thing the audit call is for. 30 minutes, your numbers, no pitch.
-
Air Canada ordered to pay customer who was misled by airline's chatbot↩
Air Canada lost tribunal case over chatbot hallucinated bereavement policy.
-
What Air Canada Lost In 'Remarkable' Lying AI Chatbot Case↩
Court ruling: airline can't reasonably claim chatbot is a separate entity; airline defended that argument and lost.
-
Klarna AI assistant handles two-thirds of customer service chats in its first month↩
Klarna announced AI assistant doing work of 700 full-time agents.
-
Klarna Reverses Course on AI Customer Support, Resumes Human Hiring↩
Klarna reversed course, tightened confidence thresholds and reinstated humans for complex cases.
-
DPD AI chatbot swears, calls itself 'useless' and criticises delivery firm↩
DPD chatbot called the company 'worst delivery firm in the world.'
-
Zendesk AI agent metrics: A complete guide to resolution rates in 2026↩
Real-world Zendesk AI resolution rates: 20-40% typical initial deploys, 60-80% well-optimized; UrbanStems 39%, Lush 60%.
-
Business Impact of AI Hallucinations – Rates & Ranks↩
Testlio 2025 report: 82% of production AI bugs from hallucinations; 39% of CS bots reworked.
-
Customer Service Statistics 2026: Humans vs AI Trends↩
79% of Americans prefer interacting with a human over an AI agent.
-
AI Hallucinations in Customer Service: Why Quality Control Architecture Matters↩
Quality control architecture is the missing layer in failed deploys.
-
DPD error caused chatbot to swear at customer↩
DPD took chatbot offline after public viral incident.
-
The Hallucination Tax: Generative AI's Accuracy Problem↩
Customer service hallucinations typically cost a few thousand dollars per event.
-
Klarna AI: 67% of Customer Support Automated, $40M Saved↩
Klarna's '700 agents replaced' framing was misleading and drew regulatory attention.
Ready to build your own AI system?
Book a Free Audit Call →Keep Reading
Stop Treating Your AI Agents Like Software. Treat Them Like New Hires.
Most failed AI agent projects aren't a model problem — they're an onboarding problem. Here's the 5-step playbook for treating agents like new hires.
Your AI Agent Bill Is Out of Control. Here's the Fix.
AI agents burn 50x more tokens than chats. The fix isn't a cheaper model — it's a tighter system. Token budgets, model routing, context pruning, deterministic off-ramps.