Project essay

Building Sally: a healthcare voice agent that checks before it confirms

Sally talks with patients in real time. Making it sound natural is the easy part. The hard part is making sure everything it tells a patient is backed by something that actually happened.

Thomas Howard · 7 min read

My role

Lead, Applied AI architecture & development

Focus

TypeScript · Voice · Tool use · Evaluation

Project stage

Live production · Patient outreach calls at pilot scale

Picture a patient calling to book a follow-up. They give a date, pause, correct it, and ask whether Thursday works. In the next couple of seconds Sally has to answer naturally, remember the correction, check real availability, and avoid saying “you’re booked” until a booking actually exists.

Each failure feels different to the caller. A slow response sounds like silence. Stale context sounds like Sally wasn’t listening. A confirmation without a booking is a broken promise, and in healthcare that can mean a missed appointment.

I lead the architecture, development, and operation of Sally, Snap eHealth’s conversational AI system, which is live in production at pilot scale, making patient outreach calls for primary care clinics. Booking into live clinic schedules isn’t switched on yet; the booking path below is how Sally is built and tested for it. The question I design around is easy to say and hard to build: what may the model say, what may the application do, and what can the final record support?

Split the work by deadline, not by feature

It is tempting to ask one model call to do everything. Sally runs three separate workloads, each with its own deadline.

• The speaking loop answers the current turn using context that has already been accepted. It has to be fast.
• Background analysis keeps that context current while the conversation continues. It can take longer, as long as it stays off the routine speech path.
• Extraction runs after the call and turns the transcript and recorded actions into a visit brief. It has to be precise.

Separating the workloads also changes how I choose models. The speaking model is judged on turn-taking, analysis on useful context, and extraction on exact structured output. Asking which single model is best would be the wrong question.

Phone and browser share the same TypeScript conversation logic. Retell handles the audio session and Twilio supplies the phone line. Sally owns the prompts, state, tools, and extraction. Keeping the transports thin means the same authorization rules apply however a patient reaches Sally.

Let the patient change their mind mid-sentence

Patients correct themselves. If background analysis is still working from an older version of the conversation, its result can quietly overwrite the correction. That is how an agent ends up confidently repeating the wrong date.

So every analysis result is checked against the transcript snapshot it read before Sally accepts it. New turns may extend that snapshot. If an earlier turn was rewritten, evidence was removed, or the result would rewind progress, Sally rejects it. Accepted context then becomes available to later turns.

Routine speech never waits for analysis. A regression test holds an analysis response open while speech completes, so if anyone couples the two, the test fails.

Speak about a booking only after it exists

For booking and availability requests, Sally withholds its draft reply, runs the tool through the runtime, and only then asks the model to respond from the committed result. That is an extra model round on every booking. I accept the cost because the alternative is an agent that occasionally confirms appointments that don’t exist.

Before any tool has an effect, the runtime rechecks that the call is still connected, the response belongs to the current turn, the session is in the right state, calling is enabled, and the recipient is verified. A response from an interrupted turn loses its authority. AI disclosure, consent, and recipient verification are application policies, so nothing a caller says can talk Sally out of them. If verification is unavailable, Sally fails closed.

The visit brief gets its own deterministic checks. A confirmed booking requires a successful tool result. The output must match its schema, every transcript reference must resolve, and every quotation must match the turn it cites. Context from the chart keeps its source label instead of quietly becoming something the patient said.

Make the unglamorous failures recoverable

A live voice agent fails in ordinary ways: a deploy in the middle of a call, a dropped database connection, a downstream system that is briefly down. These are the failures that place a duplicate call or lose a visit brief.

Sally runs as a Node service on AWS ECS Fargate behind an HTTPS load balancer, with private PostgreSQL storage. Conversation state and scheduled work commit together, so an interrupted alarm can retry, and an uncertain call-creation attempt goes to reconciliation instead of placing a second call.

A database advisory lock ensures one active process. If that process loses its connection, it marks itself not ready, stops scheduling, and closes its sockets. The tradeoff is that deploys must drain active calls before a process is replaced. I chose explicit ownership over faster rollouts.

Releases build one immutable image in staging and promote that same tested image to production. Finished calls are delivered through a transactional outbox, a queue, private artifact storage, and retried callbacks, so a temporary downstream failure is recoverable and duplicate deliveries are handled. Operational logs contain metadata; conversation content sits behind a separate access boundary.

Test the agent where it actually breaks

Reproduce failures at the boundary

Many voice-agent bugs are timing bugs: speech while analysis is pending, a tool request that arrives after the caller interrupted, a reconnect, a closed session. Provider-stubbed tests control timing and outputs so those cases are repeatable. Extraction tests feed in bad references, drifting quotations, and transcripts that claim a booking without a successful tool result.

Live-model scenarios cover unavailable appointments, caller-chosen slots, attempts to override instructions, AI disclosure, and conversational continuity. They use synthetic cases, never patient records, and they assert on tool activity as well as behavior, so failures stay inspectable as prompts and models change.

Compare models at a stable decision point

To compare speaking models fairly, I freeze the conversation at one caller turn and ask each candidate for its next reply. Replaying a whole conversation doesn’t work: once a candidate says something different, the caller’s next line would have changed too.

Debug the evaluator as carefully as the model

Two of the most useful lessons came from tests that were wrong, not models. In one, the model failed an unavailable-slot scenario because it had never been told the offered slots were the full availability. Fixing the test context fixed the result, with no prompt change.

In another, a faster speech path let the test move on before background analysis arrived, so part of the score was measuring the test’s pacing. Blocking speech on analysis would have made the test pass and the product worse. I made turn pacing explicit instead. Reports keep scenario identifiers, prompt hashes, tool activity, and timing spans, so I can tell whether to change the application, its context, the evaluator, or the model.

Give the care team a record they can check

The visit brief is where Sally’s work becomes useful to clinicians. It keeps four things distinct: what the patient said, what came from the chart, what the patient consented to, and which actions actually completed. Transcript references and committed tool results make each line traceable after the call.

Own it across every boundary

I own Sally across model behavior, runtime state, transport integration, deployment, and evaluation. That matters most when something goes wrong. A failure that crosses those boundaries can be traced through the recorded turn, the accepted context, the tool result, and the extraction evidence, and every fix becomes a regression test.

What I’d tell a founder

The model should never be your system of record. Let it decide what to say and what to ask for. Let code decide what happened.

Design a voice agent around deadlines. Conversation, analysis, and extraction have different latency budgets. Forcing them into one model call trades responsiveness against accuracy on every turn.

Budget time to debug your evals. Some “model regressions” will be test bugs, and the easy fix for the test can quietly damage the product.