Retell AI and Voice Agent APIs

Every other integration in this series fails quietly. A voice agent fails in front of a person who is holding a phone to their ear, in real time, waiting.

That changes what good handling looks like. A retry that takes four seconds is fine for a webhook and unacceptable here. A fallback that logs an error and moves on leaves someone listening to silence. So the first thing to do with Retell is look at the timing defaults, because they are built for safety margins rather than for a good call.

Change these two numbers before you ship

end_call_after_silence_ms defaults to 600000. That is ten minutes. If your agent stalls after speaking, the call stays open for ten minutes of silence before anything ends it, with a human on the other end who said hello and got nothing back.

max_call_duration_ms defaults to 3600000, a full hour. A looping agent will happily hold a line for that long, and you are paying for every minute of it.

Neither default is wrong as a backstop. Both are far too generous as the thing that actually catches a failure. Pick numbers that match a real conversation: tens of seconds of silence, and a duration ceiling that reflects how long your use case should ever take.

What a stalled model actually looks like

Retell reports a disconnection_reason on calls that end badly, and the values tell you where the failure was. error_llm_websocket_lost_connection and error_llm_websocket_runtime point at your model layer. error_no_audio_received points somewhere else entirely.

Group your failures by that field rather than treating them as one bucket called "call failed." The three reasons above need three different fixes, and a dashboard that merges them tells you nothing.

The call lifecycle itself runs through call_status: registered when the call ID is issued and it is starting, then not_connected, ongoing, ended or error. Note that registered means your request worked, not that anyone answered. An agent that treats a successful create as a completed call will report outcomes that never happened.

Webhooks are the real interface

Set webhook_url on the agent and you receive call events; an agent-level URL overrides the account-level one. The available events are call_started, call_ended, call_analyzed, transcript_updated, and four transfer events. If you do not set webhook_events, you get started, ended and analyzed.

The number to respect is webhook_timeout_ms, which defaults to 10000. Your endpoint has ten seconds. If it kicks off an agent run and waits for the result before responding, you will blow through that, and the delivery will be treated as failed even though your agent is working fine.

So acknowledge immediately, queue the work, and process it outside the request. That also protects you from duplicate deliveries starting two agent runs for one call, which is the inbound half of the problem we cover in best webhook infrastructure for agent triggers. Dedupe on the call ID before any run begins.

An idempotency key that actually exists

Retell's POST /v2/create-phone-call accepts an idempotency_key, and the documented behaviour is that you can retry the same request for up to an hour without placing a second call.

That is a genuinely good primitive and you should use it on every outbound call, because the alternative is phoning a customer twice. Build the key from the thing the call is about, not the attempt: the ticket ID, the appointment ID, the lead ID. Then a retry from your queue, your agent or a human re-running a step all collapse onto one call.

Note the one-hour window. A retry tomorrow is a new call, which is correct behaviour and also something to be deliberate about in a scheduler. Writes are only auto-retried when a key is set, so an unprotected create fails visibly rather than quietly dialling again; background in API idempotency: the complete guide and the idempotency docs.

Setting it up

npx swytchcode
swy login
swy init --mode=sandbox

swy get retell
swy auth connect retell           # key stored outside the project
swy auth status

swy discover "create a phone call" --project retell
swy discover "get a call" --project retell
swy list methods retell

swy info <canonical_id>
swy add method <canonical_id>     # reads freely; outbound calls gated below

The key is injected at call time and never enters the model's context, so there is no RETELL_API_KEY in your project for a reasoning step to read out. See the authentication docs.

Policy: phoning a human is not a normal write

Placing an outbound call is the most intrusive thing in this entire series. It interrupts someone, it is regulated in most jurisdictions, and it cannot be undone. Gate it.

{
  "id": "outbound-calls-need-a-human",
  "target": ["<retell create-phone-call canonical_id>"],
  "when": { "field": "to_number", "operator": "!=", "value": "<your-test-number>" },
  "action": {
    "type": "REQUIRES_APPROVAL",
    "message": "Agent wants to place a real outbound call"
  },
  "approval_timeout": "1h"
}

Calls to your test number flow freely so development is not painful; everything else stops and asks. Held calls go to Slack or Telegram and fail closed, so an unapproved call is simply never placed, which is the right default when the alternative is cold-calling a stranger by accident. Setup in the human approval docs, schema in the policy reference, and the general argument in human-in-the-loop approval for AI agent tool calls.

Two more things the API gives you that belong in your compliance story. honor_internal_dnc exists for a reason: do-not-call handling is a legal obligation in most places, not a preference, and an agent that assembles its own call list from a CRM export has routed around whatever list was protecting you. And ignore_e164_validation is a flag to leave alone unless you have a specific reason, since format validation is one of the few cheap guards against dialling something unintended.

From an MCP client, carefully

Reading call records, transcripts and analysis over MCP is genuinely useful: you are reviewing yesterday's calls and want them summarised. Placing calls from an editor session is not something we would set up, because an exploratory conversation is the worst possible context for an irreversible action aimed at a human being. Allowlist the reads for MCP and keep call creation in a service with the approval gate. Setup in the MCP guide, reference in the MCP docs, and skills.md for a coding agent doing the wiring.

What you can build

1. Post-call analysis, with no dialling at all. Consume call_analyzed events, summarise outcomes, tag themes, and write them into your CRM or Notion. Read-only, immediately valuable, and the obvious first project.
2. Appointment reminders. A narrow, expected, opt-in call where the recipient already has a relationship with you. Key on the appointment ID so a retry cannot call twice, and keep the script short.
3. Call-to-ticket handoff. When a call ends unresolved, open a ticket in FreshDesk or Intercom with the transcript attached, so a human picks up with context instead of starting over.
4. A failure-mode dashboard. Group calls by disconnection_reason and watch the model-layer errors separately from the audio ones. This is the thing that tells you whether your voice agent is actually working, and almost nobody builds it.

FAQ

What happens if my model is slow mid-conversation?

The caller hears silence, and with the default silence timeout that can last ten minutes. Lower the timeout, add a filler phrase for anything over a second or two, and make sure your model-layer errors surface as a graceful handoff rather than dead air.

Retell or ElevenLabs?

Different layers rather than competitors. Retell orchestrates a live phone conversation, including turn handling and telephony. ElevenLabs synthesises speech. You may well use both, and the character-budget thinking in that guide still applies.

Can the agent decide who to call?

It can propose. We would not let it decide, because an agent assembling a call list from raw data has bypassed every consent and do-not-call protection you have. Let it select from a list your systems produced, and gate the call itself behind approval.

How do I prove what was said on a call?

Retell's transcripts cover the conversation; swy audit network covers what your agent asked the API to do and when, and swy audit policy covers the calls that were held or refused. For a regulated voice workflow you want both halves; see API compliance for AI agent integrations.

Related reading

• ElevenLabs API for Voice Agents
• How to Connect Aircall to Claude
• Best Webhook Infrastructure for Agent Triggers
• A Prompt Is Not a Policy
• Browse all 300+ API integrations