Making API calls boringly reliable

What we saw

Most API clients are tested against the happy path. The request succeeds, the response comes back, done.

Production is mostly the other paths. Providers return 429s. A connection drops after the request has already reached the server. Specs get upgraded. Settings get overwritten. An agent retries something that should never have been retried.

None of these failures are interesting on their own. But they decide whether people trust an execution layer or not. Reliable API calls aren't about making the happy path work. They're about making the ugly paths predictable.

These are the ones we ran into. None were big outages. Each one could cost someone an afternoon:

• Upgrading a provider reset its settings. Retries and timeouts tuned for a slow API went back to the defaults without any warning.
• A typo like "dynamc" turned idempotency off, again with no warning.
• Two versions of the same provider overwrote each other in the manifest.
• A call would wait for a long Retry-After, then time out anyway, with an error that hid the real cause.
• Providers that run on your own address, like Jira Cloud, failed with errors that didn't say what to set.
• Usage events could get stuck when a login token was rotated.

The settings

Each provider's entry in .swytchcode/integrations/manifest.json has an execution_policy. Every field is optional. These are the defaults:

"execution_policy": {
  "max_retries": 3,
  "base_delay_ms": 500,
  "max_delay_ms": 30000,
  "http_timeout_ms": 30000,
  "connect_timeout_ms": 5000,
  "total_timeout_ms": 90000,
  "retry_on": [429, 503, 504],
  "non_retryable": [400, 401, 403, 404, 422],
  "on_401": "fail",
  "max_concurrent": 5,
  "max_response_bytes": 102400,
  "idempotency": { "mode": "none", "header_name": "Idempotency-Key", "scope": "call" }
}

The point is that each provider gets its own. A flaky third-party API can have more retries than an internal service you trust. Every field is described in the manifest docs.

When the provider fails

The rule here: don't turn a provider's failure into a confusing failure of ours. Retry what's worth retrying, and when we give up, keep the real reason.

Errors in retry_on are retried with exponential backoff and some randomness. Anything in non_retryable comes back straight away, since a 422 will fail the same way every time.

A real run against a mock that fails the first two GETs with a 503:

{
  "status_code": 200,
  "request": { "method": "GET", "url": "http://127.0.0.1:18765/v1/customers?limit=1" },
  "data": { "object": "list", "data": [{ "id": "cus_mock", "object": "customer" }], "has_more": false }
}

The mock received 3 requests. The caller only sees the success.

When all the retries fail, the CLI exits with 4 and says what happened:

{
  "error": "HTTP 503 after 4 attempt(s)",
  "category": "network",
  "retryable": true,
  "suggested_action": "check connectivity; retry is safe"
}

When a provider sends Retry-After, we wait exactly that long. We read both formats: a number of seconds and an HTTP date. But each call also has a total time limit. If the provider asks us to wait longer than the call has left, we stop straight away:

HTTP 429: the provider asked to wait 2m0s before retrying, longer than this call's time limit

Before this change, the call would wait, time out anyway, and the timeout error hid the real cause. The provider had told us exactly what was wrong, and we threw it away.

Retrying isn't always safe

Retries can cause real damage. Say a payment request reaches the provider and the payment goes through, but the response gets lost. If we retry, the customer is charged twice.

So the rules are:

• A GET can be retried, because reading twice doesn't change anything.
• A POST is never retried unless idempotency is on. With it off, a POST that gets a 503 is sent once and comes back with "retryable": false, so an agent reading the output knows not to send it again either.
• With idempotency on, the retry carries a key that lets the provider recognise the repeat.

To turn it on:

"execution_policy": { "idempotency": { "mode": "dynamic" } }

Now calls that change data carry an Idempotency-Key. Every retry of the same call uses the same key, so the provider knows it's a repeat. Against the mock, failing twice with a 503:

3 requests, keys: {'4b79c73ae1b8726afc8809d6f4d1e071'}

Three attempts, one key, one customer created.

Workflows need one more step. If a workflow run is restarted, a random key per call would make the repeated step look new. With "scope": "workflow", the key is built from the workflow run, the step and the request, so the same step in the same run always sends the same key.

When configuration changes underneath you

A setting that quietly changes behavior is dangerous. Execution settings are part of how every call behaves, so they need the same care as code.

We fixed two ways these settings went wrong without anyone noticing:

• Upgrades reset them. A new version of a provider used to start with the defaults. Now it copies the settings from the version you already have installed.
• Typos were ignored. "mode": "dynamc" meant no key was ever sent. Now doctor reports it, and get and sync show the same warning:

$ swytchcode doctor
! [execution_policy:[email protected]] [email protected]: idempotency.mode "dynamc" is not one of none, dynamic - no idempotency key is sent

The manifest is now keyed by provider.library@version, so two installed versions of a provider no longer overwrite each other. Older manifests are updated to the new format the first time they're read.

The address a provider runs on is configuration too. Some providers run on each customer's own address. Jira Cloud is at your own *.atlassian.net, so the downloaded spec can only have a placeholder. We never send a request when the address isn't set. The call stops with exit code 1 and says exactly what to set.

To show the real output, we put a placeholder into the Stripe spec's sandbox address, the same way a Jira spec ships:

{
  "error": "base URL for [email protected] (sandbox) is not set: \"https://{{site}}.example.com\" (it has a placeholder to fill in)",
  "category": "base_url_not_set",
  "suggested_action": "this provider runs on your own instance - set sandbox_endpoint for \"[email protected]\" in .swytchcode/integrations/manifest.json, or set SWYTCHCODE_BASE_URL_STRIPE_STRIPE_SANDBOX"
}

For Jira, you'd set the address in the manifest, or with an environment variable:

export SWYTCHCODE_BASE_URL_JIRA_JIRA_PRODUCTION=https://acme.atlassian.net

The same variable is how we test against a local mock: every output in this post came from the real Stripe spec pointed at http://127.0.0.1:18765. doctor lists every provider that still needs an address.

Everything else

There are a lot of smaller failure modes that aren't worth a section each:

• Large responses can flood an agent's context. max_response_bytes cuts a large JSON response and marks it "truncated": true. The body comes back as a string, so a cut-off response never breaks JSON parsing. A 10 MB hard limit applies everywhere.
• Binary data shouldn't end up in a model's context. Binary responses need --output <file>. A download that would be cut off fails, instead of saving a broken file.
• An interrupted install shouldn't leave half a provider behind. Providers are written to a staging folder first, then moved into place.
• Two commands at once shouldn't corrupt the config. Config writes are locked.
• A burst of calls shouldn't hit the provider's limit. max_concurrent limits parallel calls to one provider within a process, like an MCP server taking many calls from an agent at once.
• A redirect shouldn't get around host limits. If tooling.json limits which hosts the agent can reach, every redirect is checked against the same list.

None of these are exciting. That's the point.

Usage events

Usage reporting has to be reliable without ever slowing a call down:

• Every live command is first written to a local SQLite queue.
• Events are sent in the background in batches, either when there are enough of them or when the oldest has waited long enough. That way, even light users' usage arrives quickly.
• If sending fails, the events stay in the queue and go out next time. They're only removed once the server confirms it got them.
• The queue is keyed by account, not by login token, so rotating a token doesn't leave events stuck.
• The full audit log is uploaded the same way, in daily batches, kept separate per customer.

Commands never wait for any of this. A dropped connection doesn't lose events: the queue only starts dropping the oldest ones if a very large backlog builds up.

Finally

Reliability work is hard to prioritise because there's nothing to demo. Nobody asks for "settings that survive upgrades" until theirs get reset.

The thing that helped most was our end-to-end tests. They run the real CLI against a fake provider that fails when we tell it to. That's how we know the outputs in this post are real, and how we can change the retry code safely.

If users need to read our source code to understand what happened, the bug is on our side.

That's the rule we try to follow at Swytchcode. When you run API calls with Swytchcode, retries, idempotency and timeouts are handled for you, and when something does go wrong, the error tells you what to fix.