Fireworks AI API from an Agent

There is a category of agent integration nobody writes about properly: your agent calling another model. Not the model running the agent, a second one it calls as a tool, because it is cheaper, faster, or better at one specific job.

Fireworks is a good place to do that. It is also the point where your cost control quietly stops working, because the thing deciding how many tokens to spend is now a language model rather than a line of your code.

The shape of the API

Chat completions live at https://api.fireworks.ai/inference/v1/chat/completions with a standard Authorization: Bearer header. Keys come from the dashboard or firectl api-key create.

Models are named by path, not by short slug: accounts/fireworks/models/<model-name>. So you reference something like accounts/fireworks/models/kimi-k2-instruct-0905 rather than a bare name. A model asked to "call the Kimi model" will confidently invent a short slug, so resolve real model names from the dashboard and keep them in config.

The parameters follow OpenAI's conventions closely enough that most code ports over, which is the appeal. It is the places where it deliberately differs that matter.

The default that silently eats your prompt

This is the one to know about. context_length_exceeded_behavior defaults to truncate, which the docs note differs from OpenAI's behaviour.

So when your agent sends a prompt that is too long, you do not get an error. You get a successful response, computed from a prompt that was quietly cut down to fit. The model answers based on whatever survived, which may be missing the actual question.

For an agent this is nasty, because agents accumulate context. The same call works for weeks on short inputs and then starts producing subtly wrong answers once a conversation gets long, with a 200 every time. Nothing in your monitoring will flag it.

Set it to error explicitly, and handle the failure, rather than letting truncation happen invisibly. This is the exact shape of problem we wrote about in 200 OK but it failed: the call succeeded and the outcome is wrong.

Token accounting, and what you can actually rely on

The usage object gives you prompt_tokens and total_tokens as required fields, plus completion_tokens which is optional and nullable. There are also detail objects for prompt_tokens_details.cached_tokens and reasoning tokens.

Two practical consequences. First, do not write accounting code that assumes completion_tokens is present; derive it from total minus prompt if you need it guaranteed. Second, a nice detail for agents: streaming responses include usage by default, which the docs flag as unlike the OpenAI spec. So you can account for a streamed call without a separate lookup.

Log the usage object on every call. When your bill jumps, the question is always "which agent step got expensive," and the only way to answer it is per-call records. API observability for AI agents covers what that trace needs to carry.

Setting it up

npx swytchcode
swy login
swy init --mode=sandbox

swy get fireworks
swy auth connect fireworks        # key goes to the local store, not your .env
swy auth status

swy discover "chat completions" --project fireworks
swy list methods fireworks
swy info <canonical_id>
swy add method <canonical_id>     # inference only; not the quota-update methods

That last comment matters. Fireworks exposes account-scoped quota endpoints that can list, retrieve and update quotas. An agent that can raise its own spending limit is not a cost control, it is a suggestion. Leave those methods out of the allowlist entirely, and then no prompt, no injected instruction and no confused tool call can reach them.

The key is stored outside your project folder, encrypted at rest, and injected when the call runs, so it never enters the model's context. See the authentication docs. Keep --project fireworks on every discover call, since "chat completions" matches a dozen model providers, and run swy get before swy info or info will report a real method as missing.

import { exec } from "@swytchcode/runtime";

const res = await exec("<fireworks chat-completions canonical_id>", {
  body: {
    model: "accounts/fireworks/models/<exact-model-name>",
    messages: [{ role: "user", content: input }],
    max_tokens: 512,                            // always set a ceiling
    context_length_exceeded_behavior: "error"    // never truncate silently
  }
});

// completion_tokens is optional; derive it so accounting never breaks
const { prompt_tokens, total_tokens, completion_tokens } = res.usage;
logSpend({ prompt_tokens, completion: completion_tokens ?? total_tokens - prompt_tokens });

Policies: a spend cap the model cannot argue with

This is the clearest use of argument-level policy anywhere. The risk is not destruction, it is an agent in a loop spending money, and the arguments it sends are exactly where the cost lives.

{
  "id": "cap-output-tokens",
  "target": ["<fireworks chat-completions canonical_id>"],
  "when": { "field": "max_tokens", "operator": ">", "value": 2048 },
  "action": {
    "type": "DENY",
    "message": "Agent requested more than 2048 output tokens on a sub-call"
  }
}

A deny rather than an approval request, because nobody wants a Slack message every time an agent gets ambitious about max_tokens. Add a second rule pinning which models may be called, so your agent cannot wander onto an expensive one mid-task. Check available operators in the policy reference, and keep the file in version control so raising a cap is a reviewed change rather than a quiet one. That is the policy-as-code argument.

Retries and idempotency on an inference call

Inference is a POST with no side effect: run it twice and you get two answers, not two charges against some external record. So correctness does not need an idempotency key here. Your wallet does care, because each retry is billed.

Rate-limit errors get exponential backoff automatically, which is what you want, and the retries guide covers the behaviour. Two things to add yourself: a cache keyed on model plus a hash of the messages, because agents re-ask identical questions constantly inside multi-step plans, and a retry ceiling, because a loop retrying an expensive generation is the worst possible failure mode. The broader discussion of when keys matter is in API idempotency: the complete guide.

From Claude Code or Cursor

Model-to-model calls over MCP are useful for experimentation: you are comparing how three models handle the same prompt without writing a harness. Connect through the MCP server and your allowlist and token caps travel with it. Setup is in the MCP guide, reference in the MCP docs, and skills.md hands a coding agent the conventions so it can wire this itself.

What you can build

1. A cheap first-pass router. Send every request to a small fast model first, and escalate to an expensive one only when the cheap one is not confident. The economics usually work out, and the whole thing is two allowlisted methods.
2. A bulk classification step. Tag thousands of support tickets or documents overnight with a small open model rather than paying frontier prices for a job that needs a label, not prose. Pairs with Hugging Face if you want a dedicated classifier instead of a general model.
3. A model bake-off harness. Run your real prompts against several models, record usage and latency per call from the audit log, and produce an honest cost-per-outcome table rather than a vibe.
4. A fallback path. When your primary provider is degraded, route to a second one so the agent keeps working. Worth building before you need it, and the allowlist makes the blast radius of the fallback obvious.

FAQ

Can I point OpenAI SDK code at Fireworks and move on?

Mostly, and the gaps are the interesting part. The parameters follow OpenAI's conventions, but streaming usage and the context-length behaviour differ deliberately, and the model names are paths rather than slugs. Port the code, then test the long-prompt and streaming paths specifically rather than assuming.

How do I stop an agent spending without limit?

Three layers, and you want all of them. A max_tokens ceiling on every call, a policy that denies calls above a threshold so the ceiling is not optional, and a provider-side quota the agent cannot modify because you never allowlisted the quota-update method.

Should the agent choose which model to use?

Within a list you approved, yes, and it is often a good judgement call since it has the context. Across any model it can name, no. Pin the allowed set in policy so the choice is bounded.

Where do I see what it actually spent?

swy audit network gives you every call with its arguments, which maps to your bill, and swy audit policy shows the calls a cap stopped. If the second number is large, your prompts are asking for more than you think. More in how to log every API call an AI agent makes.

Related reading

• Fireworks integration page
• Using the Hugging Face API from an AI Agent
• Luma API for AI Agents
• Securely Connecting the Claude API to Your Agent
• Browse all 300+ API integrations