AI for Customer Service and Success: Agentic Support, Health Scoring, and Sentiment

Customer service is the best-evidenced enterprise AI use case there is: a randomized field experiment across 5,172 support agents found a generative AI assistant lifted issues resolved per hour by 15% on average, with the largest gains for the least experienced agents.^[1] The open questions are no longer whether AI helps support. They are which altitude to deploy at — assisting humans, deflecting contacts, or taking action autonomously — what the escalation contract looks like, and whether the signal your service organization generates flows into health scoring, renewals, and product decisions or evaporates in the ticket queue.

By the numbers

The one controlled experiment worth anchoring on

Most AI-for-support claims are vendor telemetry. The exception is the Brynjolfsson, Li, and Raymond study, which tracked the staggered rollout of a generative AI conversational assistant to 5,172 customer support agents and measured what actually changed.^[1] Beyond the 15% average productivity gain, three findings should shape how you deploy. First, the gains were sharply uneven: less experienced and lower-skilled agents improved both the speed and the quality of their output, while the most experienced, highest-skilled agents saw small speed gains and small declines in quality.^[1] The assistant effectively distributed the behavior of top performers to everyone else — which means the ROI case is strongest where your bench is junior, turnover is high, or ramp time is long, and weakest (possibly negative on quality) for your senior tier.

Second, the study found gains were largest for relatively rare problems, where human agents have the least baseline training and experience — the opposite of the intuition that AI should take the routine tickets and leave the hard ones to people.^[1] Third, the assistant changed the texture of the work itself: the authors find evidence that AI assistance facilitated worker learning, improved English fluency particularly among international agents, and improved the experience of work on the customer side of the conversation.^[1]

The decision this evidence supports: deploy assistive AI to your newest and most junior agents first, measure quality (not just handle time) for your senior tier before forcing adoption there, and do not assume the routine/complex split maps to the automate/human split. It often runs the other way.

Pick your altitude: assist, deflect, or act

Every first-party stack now ships all three altitudes, and conflating them is how support-AI programs get mis-scoped. An agent-assist copilot, a self-service conversational bot, and an action-taking agent have different risk surfaces, different evaluation regimes, and different failure costs. Google's Agent Assist, for instance, "provides in-the-moment coaching and next-best action guidance to customer care representatives" — the human stays the actor.^[4] Amazon Lex V2 sits at the deflection altitude: speech recognition plus natural language understanding for intent-and-slot bots, now with LLM-backed "Assisted NLU" that improves intent classification while staying within the bot's configured intents and slots — a deliberately bounded design.^[5] The third altitude is where the market has moved: AWS now describes Amazon Connect as "a portfolio of agentic solutions,"^[6] with AI agents that "understand, reason, and take action," navigating organizational resources to resolve customer needs directly.^[3]

The portable rule: altitude should be assigned per workflow, not per program. Order-status lookup can run fully agentic on day one; a refund above a threshold may stay assist-only for a year. Anthropic's customer-support guide frames the same discipline as breaking the ideal interaction into discrete tasks — greeting, product Q&A, staying on topic, tool-backed quote generation — and prompting and evaluating each task separately rather than shipping one monolithic bot.^[7]

Agentic support: what changes when the bot can act

The chatbot-to-agent shift is an architecture shift, not a model upgrade. A deflection bot needs a knowledge base and an escalation path. An action-taking agent needs tool definitions against your order-management, billing, and CRM APIs; per-tool authorization scoped to the authenticated customer; idempotent and reversible operations wherever possible; and an audit trail that records every tool call the agent made and why. AWS's own framing of Connect AI agents is explicit that they "dynamically navigate your organization's resources to find solutions and take action to resolve customer needs," while working "in collaboration with your workforce"^[3] — the collaboration clause is the part enterprises underbuild. The escalation contract — which confidence levels, which intents, which customer tiers, and which dollar thresholds hand off to a human, and with what context attached — is the single highest-leverage design artifact in an agentic support program; the full pattern language is covered in /guides/human-in-the-loop-agents rather than duplicated here.

The vendor-published outcomes are directionally useful even discounted as marketing. Zepz reports 67% of inquiries resolved instantly through self-service on Amazon Connect; Orbit Irrigation's customer-care lead expects the assistant to "create 10%–15% time savings on every contact" by replacing multi-source knowledge searches with generated, knowledge-base-grounded responses.^[3] Note what those numbers are: one is a deflection rate, the other an assist-time saving — two different altitudes, quoted on the same product page. When a vendor shows you a case study, ask which altitude produced the number before you put it in your business case.

Model-side capability is rarely the constraint. Anthropic's guide lists the indicators that support chat is a fit — high volumes of repetitive queries, 24/7 availability requirements, rapid scaling during peak periods, consistent brand voice — and notes multilingual coverage in over 200 languages from a single system, which collapses what used to be a per-language bot program into a routing decision.^[7] The constraint is your side of the integration: whether your policies are written down unambiguously enough for an agent to execute them, and whether your APIs expose safe, scoped actions. Voice channels add a real-time layer with their own latency and barge-in constraints; that stack is covered in /guides/enterprise-voice-ai-guide.

Sentiment: the perception layer, and a lifecycle warning

Sentiment analysis is the perception layer of a service stack: it converts conversation exhaust into signals that route, prioritize, and escalate. The mature API pattern is well represented by Azure AI Language, which assigns "negative," "neutral," and "positive" labels using the highest confidence score, evaluates sentiment at both the sentence and the document level, and returns confidence scores between 0 and 1 for each label.^[8] The feature that earns its keep in support is opinion mining — aspect-based sentiment analysis that ties polarity to specific targets in the text^[8] — because "the customer is angry" is not actionable, while "the customer is angry about the billing migration, specifically the proration" is a routable, reportable fact. On the assist side, sentiment analysis ships as a documented feature of Google's Agent Assist alongside Smart Reply, knowledge assist, summarization, and live transcription, feeding supervisor intervention rather than reporting decks.^[9]

Two cautions, neither of which needs a statistic to be true. Sarcasm, mixed sentiment, code-switching, and dialect all degrade classifier output, and the errors are not uniformly distributed across your customer base — so a sentiment-driven escalation policy should be audited by language and segment before it gates anything that matters. And a three-label score on a single message is weak signal; sentiment becomes decision-grade when aggregated across a conversation, an account, or a cohort, which is exactly where it stops being a contact-center feature and becomes a customer-success input.

Health scoring and renewal prediction: the memory layer

If sentiment is perception, customer health scoring is memory: a longitudinal model of whether an account will renew, expand, or churn, built from product telemetry, support-ticket volume and severity, billing history, and engagement signals. This is classic supervised machine learning — gradient-boosted trees and logistic regression remain entirely respectable here — and the vendor category (customer-success platforms such as Gainsight, Totango, and ChurnZero, plus CRM-native scoring) is mature. What generative AI changes is the feature set, not the algorithm: the majority of the leading indicators of churn are locked in unstructured text — ticket narratives, QBR notes, email threads — and LLM extraction plus aspect-level sentiment turns that text into features a scoring model can actually consume. The stack decision is therefore less "which churn model" and more "which pipeline gets conversation-derived features into the model within days, not quarters."

Be skeptical of dashboards over honest baselines. Churn events are sparse in stable B2B books, which makes overfitting to historical churn patterns easy and makes headline accuracy meaningless — a model that predicts "renews" every time is 95% accurate in a book with 5% churn. Evaluate on precision and recall at the intervention threshold you will actually staff, calibrate the threshold to the cost of a wasted save-play versus a missed save, and re-validate quarterly for drift. Demand explainability in the operational sense: a health score a CSM cannot decompose into "support escalations up, champion went quiet, usage of the sticky module down" will be ignored, and an ignored score has an ROI of zero regardless of its AUC. And treat any vendor-quoted churn-reduction percentage as unverifiable until they show the counterfactual — a save-play attribution without a holdout group is a story, not a measurement.

Closing the loop: service exhaust as product signal

The least-deployed and highest-leverage piece of this stack is the loop from support back into product. Every resolved ticket encodes a claim about where the product fails, and modern summarization makes that claim cheap to structure: Agent Assist, for example, automatically summarizes each interaction, its key takeaways, and next steps when the conversation ends.^[4] Pipe those summaries through topic clustering and aspect-based sentiment, and you get a ranked, quantified defect-and-friction backlog that updates continuously — the input product management says it wants and support has always had. The practical pattern is unglamorous: a weekly job that clusters the period's conversation summaries, diffs the cluster volumes against the prior period, and writes the top movers into the product backlog tool with linked exemplar tickets.

Govern this loop like the analytics pipeline it is. Define the KPIs that connect service signal to product outcomes — recurring-contact rate per feature area, time from cluster emergence to backlog triage, ticket-volume decay after a fix ships — and assign an owner on the product side, because a loop with no consumer is a report. This is also where customer service stops being a cost-center use case and joins the revenue motion: the same conversation-derived account signals feed sales prioritization (/use-cases/ai-for-sales-guide) and the shared data layer of a unified go-to-market stack (/guides/unified-gtm-ai-stack).

Honest objections

The steelman against aggressive support automation deserves airtime. The best evidence is narrower than the enthusiasm: the randomized study is one firm, one assistive (not agentic) tool, and chat-based technical support — and even there, the highest-skilled agents saw small quality declines, which is a warning about mandated adoption, not a rounding error.^[1] The headline vendor numbers — 67% self-service resolution, 28% more conversations handled — are published by the vendors selling the products, drawn from customers they selected.^[3][4] Deflection metrics can be gamed by making humans harder to reach, converting cost savings into silent churn that surfaces two quarters later in the renewal book. And an action-taking agent is a new attack and failure surface: prompt-injected refunds, cascading wrong actions, and regulatory exposure in jurisdictions with strong consumer-protection regimes. None of this argues for abstaining; it argues for shadow-mode evaluation before autonomy, per-workflow risk gating, deflection metrics paired with recontact and CSAT metrics, and keeping the human tier genuinely reachable.

The read

Treat AI in customer service and success as one system with four layers: an action layer (agentic support, gated per workflow by risk), an assist layer (copilots deployed junior-first, where the experimental evidence says the gains concentrate^[1]), a perception layer (aspect-level sentiment feeding routing and supervision), and a memory layer (health scoring consuming conversation-derived features, closing the loop to product and revenue teams). Buy the contact-center substrate from a first-party stack you already trust, insist on model and classifier swapability at the API boundary, and spend your scarce engineering on the two artifacts no vendor can ship for you: the escalation contract and the evaluation harness. The organizations that win here are not the ones with the most autonomous bots — they are the ones whose service data compounds.