AI Customer Service Agents: A Criteria for Success
By: Michael Zhao
The Ivey Business Review is a student publication conceived, designed and managed by Honors Business Administration students at the Ivey Business School.
AI Isn’t on Hold, But You Might Be
Customer service has always been expensive. Long hold times, high agent turnover, inconsistent quality, and the sheer volume of repetitive inquiries have made it one of the most operationally burdensome functions in any large organization. For decades, companies have looked for ways to reduce that burden. The introduction of AI seemed to be a potential solution to that problem.
The global AI for customer service market was valued at $12.06 billion in 2024 and is projected to reach $47.82 billion by 2030. Today, 89% of contact centers report using AI chatbots. The business case is compelling: Companies using AI customer service tools have cut first response times by 37% and improved ticket resolution speed by 52%. AI agents now resolve more than 45% of queries before they ever reach a human. But the numbers hide a more complicated story. Only 25% of contact centers report having successfully integrated AI automation and 64% of executives openly acknowledge they are making active trade-offs between cost savings and customer satisfaction when deploying these tools.
The problem is that large language models (LLMs) have a much narrower practical range than expected. LLMs are trained on text datasets and learn from statistical patterns. They are very good at generating fluent, human-sounding text, but lack judgement when a situation falls outside of their training data. When the latter happens, AI will hallucinate: they won't acknowledge their own knowledge gaps but instead present a plausible guess. The pressure to find new AI use cases has never been more intense due to competitor frenzy. Given the repetitive and costly nature of customer service, the space has become ground zero for this AI experiment.
But implementing AI customer service agents without considering the impacts on customer trust, sentiment and brand perception is a risky venture which, despite initial cost savings and efficiency gains, can crash and burn. On the other hand, when done right, AI can improve the workflow of human agents and the customer service experience of users. Two Swedish companies, Klarna and IKEA, learned this lesson recently. One the hard way, the other the right way.
Klarna Crashes the Checkout
Klarna is a Swedish fintech company founded in 2005. Its core product is buy-now-pay-later (BNPL), stepping in to absorb all credit risks for merchants while earning a small cut on each transaction.
When using Klarna, users can immediately see all relevant information about their finances with no hidden fees. The app’s polished interface and emphasis on transparency make it particularly appealing to younger Gen Z and millennial users, who account for nearly 90% of the platform’s active base as of 2025. Since Klarna operates at the intersection of credit and personal finance, customers often contact support about sensitive and high-stakes issues. For example, a user may be charged for an item they returned but still face an upcoming payment deadline, risking late fees or damage to their credit if the issue is not resolved quickly. These situations make trust, clarity, and ease of use central to Klarna’s value proposition.
Klarna was also preparing itself for an IPO, which meant they wanted results and fast. In 2024, Klarna drew attention when it announced plans to replace more than 700 customer service employees with an in-house AI assistant built with OpenAI. The system quickly became central to the company’s support operations, handling 2.3 million conversations in its first month, or roughly two-thirds of all customer interactions. It also cut resolution times from 11 minutes to under 2 minutes and reduced repeat contacts by 25%. Klarna claimed these gains would translate into a $40 million profit improvement for the year through cost savings.
Klarna's AI worked fine for what AI is good at: straightforward, predictable questions with consistent answers. The issue is that BNPL customers are usually calling about much more complex, urgent, and emotional requests about deeply personal matters, such as their finances. Common complaints such as payment mistakes, delayed refunds, or misunderstandings about spending limits are frustrating and sensitive issues for customers, which require human judgment and empathy to resolve. Judgement and empathy are two things that AI customer service agents do not have. As a result, Klarna’s AI customer service initiative resulted in customer complaints, mistrust, and damaged brand perceptions.
The implementation of an AI customer service tool went directly against Klarna's two key selling points: transparency and ease of use. In May 2025, after much customer backlash, Klarna backtracked on its AI customer support strategy, reversing past cost-cutting decisions and rehiring for the customer service agent roles they had cut. Klarna CEO’s admitted that the company had prioritized efficiency and cost over quality.
IKEA Assembles for AI Success
IKEA is the world's largest furniture retailer, generating 45.1 billion euros in sales in fiscal year 2024. In 2021, IKEA introduced an AI chatbot named Billie, after their iconic Billy bookcase. The problem Billie was built to solve was clear and feasible from the start. IKEA hosts over 882 million customers annually and has seen a surge in online inquiries that was overwhelming its call centre agents. The bulk of those inquiries were exactly the sort of thing AI handles well, things like "where's my order?", "What's your return policy?", "Is this product available in my store?" All questions required the same answer every time. Between 2021 and 2023, Billie resolved approximately 47% of customer inquiries directed to call centres, around 3.2 million interactions, saving IKEA nearly 13 million euros in the process. Customer satisfaction with the bot sat at 85%. This means that, by every operational measure, Billie was doing its job.
At this point, IKEA had 8,500 call centre employees whose day-to-day workload had been cut nearly in half. Instead ofmass layoffs, IKEA transitioned these employees from call centre roles into interior design consultants. Customers could now book a 45-to-60-minute video consultation with one of these advisors, walk through their space, explain what they were trying to achieve, and come away with a product list, a floor plan, and a 3D rendering of what their finished room could look like. The results were impressive. By the end of fiscal year 2022, IKEA's remote interior design channel had generated 1.3 billion euros in sales, accounting for 3.5% of the company’s total revenue. IKEA has since set a target of growing that share to 10% in the coming years.
Klarna Crashed, IKEA Assembled: Lessons in AI Customer Service
On the surface, both companies took steps to automate their customer service business lines. But what did IKEA do differently to succeed? Where did Klarna go wrong? The answer lies in the relationship each company has with its customers.
The difference wasn't that IKEA's customer service was simpler than Klarna's across the board. The real difference was that IKEA was precise about what they asked the AI to do. Billie was never handed a dispute, a complaint, or anything requiring a judgment call. It was handed a very specific, well-defined slice of customer volume, the repetitive same-answer-every-time queries, and it handled that well. The moment a conversation moved beyond that, the call escalated to a human agent. IKEA clearly defined a line between what the AI could handle and what it couldn't, and stuck with it. Klarna's AI, on the other hand, was expected to handle whatever came through the door, including difficult multi-step financial disputes that required human judgment and empathy. A Harvard Business School study analyzing over 250,000 customer service chat conversations found that while AI improved outcomes for simple, low-stakes requests, the gains were significantly weaker when customers reached out with repeat complaints or complex grievances, exactly the kinds of interactions that make up the bulk of a BNPL company's support queue.
There is also the issue of brand alignment. Klarna built its entire identity around two things: transparency and ease of use. Directing a frustrated user who's worried about their finances to a slow, robotic, and ineffective AI tool was a branding misstep for Klarna. IKEA, by contrast, never needed their chatbot to carry any emotional weight. Billie handled the predictable, repeatable volume so that humans could focus on their own work.
Chat Your Way to the Bot-tom Line
As more companies roll out AI customer service tools over the next few years, the same divide is going to keep showing up. 44% of organizations have already experienced negative consequences from AI customer service implementations. What actually separates a company that's ready for this from one that isn't comes down to a few honest questions worth asking before pulling the trigger.
Question #1: What does your typical customer contact actually look like?
74% of customers prefer chatbots for simple questions. That preference flips immediately the moment issues get more complicated. Across all industries, virtual agents perform significantly worse than live chat agents on task completion, effort, and customer satisfaction. If your support queue is dominated by order tracking, policy questions, and basic troubleshooting, customers are more likely to appreciate the speed and availability of an AI agent. If your queue is dominated by disputes, billing issues, or account problems that require pulling context from multiple systems and making a judgment call, implementing AI will likely result in more frustration and customer complaints.
A good rule of thumb is that if the same customers are reaching out multiple times about the same issue, your queries are probably too complex for AI to close on its own.
Question #2: How much does your customer trust you with?
There's a meaningful difference between a customer who contacts support about a delayed bookcase and a customer who contacts support because of their personal finances. Nearly one in five consumers who have used AI for customer service saw no benefit from the experience, a failure rate almost four times higher than AI use in other contexts. For more personal matters, such as finances, health or legal issues, the higher the bar an AI agent has to clear before it earns the right to handle those conversations unsupervised.
Industries like e-commerce, logistics, telecommunications, and utilities tend to have a better profile for AI customer service. Customer queries are high-volume, the answers are consistent, and the emotional stakes of each interaction is relatively low. In telecom and utilities specifically, AI is already proving effective at monitoring sentiment in conversations, flagging frustration early, and rerouting customers before the situation escalates.
For sectors like financial services, healthcare and insurance, where customer trust is paramount, the automation process for customer service should be done with added thought. AI certainly has a role to play in these industries. For example, Bank of America's Erica handles balance checks and transaction lookups for tens of millions of users effectively. But, the cost of a bad interaction is measured in lost trust, not just a one-star rating.
Question #3: Are you measuring success properly?
When deploying AI in customer service, success must be measured beyond purely cost-based or efficiency-based metrics such as resolution time or cost per ticket. Focusing on non-financial metrics, such as user experience, brand image and customer sentiment are equally as important for business success and profitability. Research shows that a one-point increase in customer satisfaction can increase shareholder value by 1%, and 96% of consumers say they trust a brand more when the user experience is more seamless.
Conclusion: Glitches Behind, Happy Customers Ahead
Klarna and IKEA are two Swedish companies that found themselves at the same crossroad. Both rolled out implementation plans for high-volume customer service operations that could be automated: One walked away with a billion-dollar revenue channel; The other spent two years learning a lesson about the risks of automation when done without considering customer sentiment, experience and brand perception.
AI customer service is here to stay. The LLM tools will continue to improve, becoming more efficient and cost-effective for companies to implement. The companies that can successfully integrate AI into their customer service operations are ones that ask themselves the right questions before they hit deploy.
Editor(s): Mabel Zhao
Researcher(s): Emily Chen
