Amazon Case Study
AI Health Benefits Assistant
Overview
I designed a GenAI-powered chatbot for Amazon's Benefits Management Console, from zero to shipped product. Worked with engineering, data science, and product to build a conversational assistant that helps admins resolve complex benefit queries faster and with more confidence.
Goals:
- Reduce time-to-resolution for complex benefit queries.
- Build trust in AI-generated responses through transparency and source attribution.
- Create a scalable conversational pattern for future AI integrations across Amazon benefits.
The Challenge
Benefits admins handle thousands of queries daily across coverage, eligibility, and plan details. They're stuck navigating fragmented docs, 4+ disconnected systems, and edge cases that need deep domain expertise.
Reduce operational cost of benefit query resolution while maintaining accuracy.
Get reliable answers without switching between 4+ systems and outdated documentation.
The Benefits Management Console before AI integration
Problem to solve
Administrators lack a unified, trustworthy source of answers, leading to slow resolutions, inconsistent responses, and declining satisfaction scores.
Admins had to switch between disconnected tools to resolve a single employee query
Research
I ran contextual inquiry with 12 benefits admins, mapped their existing workflows, analysed 3 months of query data, and benchmarked AI assistants for trust patterns. Key findings:
- Trust must be earned incrementally; admins verify before acting
- Source attribution is critical: "Where did this answer come from?"
- Complex queries need multi-step reasoning, not single answers
- Existing tools required switching between 4+ systems per query
How might we
How might we design an AI assistant that administrators trust enough to integrate into their daily workflow while maintaining accuracy and accountability?
Participated in contextual inquiry, revealing trust as the primary barrier to AI adoption
Design Strategy
Weekly iteration cycles with stakeholder reviews. I explored three interaction models (sidebar panel, full-page, and embedded) and tested each with 8 admins in moderated sessions. Three questions guided every decision:
- "Can I trust this answer enough to act on it?"
- "Where did this information come from?"
- "What do I do when the AI doesn't know?"
AI Assistant integrated into the benefits portal with intro prompts and suggested actions
Solution
A graduated confidence system, built into the existing benefits portal workflow:
- High confidence: Direct answer with sources linked
- Medium confidence: Answer with caveats and suggested verification steps
- Low confidence: Graceful handoff to human expert with context preserved
AI Trust Framework
- Transparency: Always show confidence levels and source attribution
- Control: Users can override, edit, or reject any AI suggestion
- Graceful fallback: Clear escalation paths when AI isn't confident
- Explainability: Show reasoning, not just answers
Response with interactive links, reasoning indicator, and source attribution
Conversation flow with feedback controls and expandable sources
Impact
What I learned
Trust is earned in small, consistent steps
The confidence indicator drove the most positive feedback. Users needed to know when the AI was unsure, not for it to be perfect.
Ship the smallest valuable thing first
Starting with FAQ-only scope let us validate trust patterns before expanding to complex multi-step queries and edge cases.
Design for the skeptic, not the believer
The admins who trusted AI least became our strongest advocates once they could verify every source and override any suggestion.