Luma API for AI Agents

Most APIs answer in under a second. Luma does not. You ask for a video, you get an ID back, and the actual work happens somewhere else for the next few minutes. That one difference breaks more agent integrations than any auth problem, because agents are built around request and response, and this is neither.

Two failure modes show up immediately. The agent sits in its reasoning loop polling every two seconds, burning tokens and wall-clock time to discover nothing has changed. Or it decides the job failed because it is not done yet, resubmits, and you pay for two generations of the same prompt.

The fix is architectural, not a prompt tweak. Here is the shape that works.

What the API actually gives you

The Luma Agents API lives at https://agents.lumalabs.ai/v1 and takes a bearer token. The flow is two calls:

• POST /v1/generations to start a job. You get an ID back more or less instantly.
• GET /v1/generations/{generation_id} to check on it. The state field is what you care about, with completed and failed as the terminal values. A failure also carries failure_reason and failure_code, which are worth logging because a content-policy rejection and a transient infrastructure error need completely different handling.

The important omission: as of this writing the documentation does not describe webhooks or a callback URL. Completion is detected by polling. So the question of who does the polling, and where, is your main design decision.

Also note the SDKs read a LUMA_AGENTS_API_KEY environment variable automatically. Convenient for a script, and exactly what you do not want in an agent, because an environment variable in the agent's process is readable by anything in that process, including a reasoning step that got talked into printing its config.

Rule one: the agent starts the job, something else waits for it

Do not poll from inside the agent's reasoning loop. Submit the job, write the generation ID somewhere durable, and end the turn. Then let a scheduler, a queue worker or a durable workflow do the waiting and wake the agent when there is a result.

import { exec } from "@swytchcode/runtime";

// canonical_ids come from `swy list methods luma`
async function startRender(brief) {
  const job = await exec("<luma create-generation canonical_id>", {
    body: { prompt: brief.prompt, model: brief.model },
    // same brief submitted twice collapses onto one paid generation
    idempotencyKey: `render:${brief.id}`
  });

  // persist OUTSIDE the agent's context, then stop.
  await db.renders.insert({ briefId: brief.id, generationId: job.id, state: "pending" });
  return job.id;
}

// Runs on a timer, not inside the model's reasoning.
async function checkRender(generationId) {
  const g = await exec("<luma get-generation canonical_id>", {
    params: { generation_id: generationId }
  });

  if (g.state === "completed") return { done: true, assets: g.assets };
  if (g.state === "failed")    return { done: true, error: g.failure_reason, code: g.failure_code };
  return { done: false };   // still working: do nothing, check again later
}

The line that saves you money is idempotencyKey. More on why below. For the waiting half, a durable execution engine is the right tool, and we compared the options in best idempotency and retry infrastructure for agents and best durable execution platforms.

Rule two: a pending job is not a failed job

This is the expensive mistake. A generation that is still running returns successfully with a non-terminal state. An agent that treats "not completed" as "failed" will resubmit, and video generation is not cheap. Do that in a loop and you have a bill nobody budgeted for.

So be explicit in your code and in your prompt: only two states are terminal. Everything else means wait. Treat a timeout on your side as "unknown, check again" rather than "failed, retry," because the job is almost certainly still running happily on Luma's side.

This is a close relative of the 200 OK but it failed problem, inverted: here the call succeeds and the work is unfinished, and naive handling reads unfinished as broken.

Idempotency, because every duplicate has a price tag

With most APIs a duplicate write is a data problem. Here it is a billing problem, and the ambiguous case is common: your submit request leaves, the connection drops before the ID comes back, and you genuinely do not know whether a generation started.

Swytchcode only auto-retries a POST when an idempotency key is set on it, so an unprotected submit surfaces the failure rather than quietly starting a second paid job. Derive the key from the work, not the attempt: a brief ID, a scene number, a content hash of the prompt. Then a retry from your worker, from the agent, or from a human re-running the step all collapse onto the same generation. The reasoning is in the complete idempotency guide and the implementation in the idempotency docs.

Belt and braces: before submitting, check your own table for an existing generation ID for that brief. The cheapest generation is the one you do not run twice.

Retries on the poll, and politeness

The GET is safe to retry as often as you like, which is exactly why you should not. Rate-limit errors get exponential backoff automatically, which handles the recovery; the design job is to not generate the problem. Poll on a sensible curve: a few seconds for the first checks, then widen to fifteen, then thirty. A job that takes two minutes does not need forty checks. Details in how Swytchcode retries failed API calls and the retries guide.

Set a ceiling too. If a job has not reached a terminal state after a generous timeout, stop polling and raise it to a human rather than checking forever. An orphaned poller is a quiet cost that runs for weeks.

Setting it up

npx swytchcode
swy login
swy init --mode=sandbox

swy get luma
swy auth connect luma           # key goes to the local store, not your .env
swy auth status

swy discover "create a generation" --project luma
swy discover "get a generation" --project luma
swy list methods luma

swy info <canonical_id>
swy add method <canonical_id>   # add the create and the get, nothing else

swy exec <luma get-generation canonical_id> \
  --input '{"params":{"generation_id":"<id>"}}'

No LUMA_AGENTS_API_KEY in your project and no .env file. The key is stored outside the project folder and injected when the call runs, so it never reaches the model's context. See the authentication docs for the mechanics.

The policy worth writing: a spend cap

Generation APIs are the clearest case for argument-level policy, because the risk is not destruction, it is an agent in a loop spending money. Gate on whatever your workflow treats as expensive: duration, resolution, batch size.

{
  "id": "long-renders-need-a-human",
  "target": ["<luma create-generation canonical_id>"],
  "when": { "field": "duration", "operator": ">", "value": 10 },
  "action": {
    "type": "REQUIRES_APPROVAL",
    "message": "Agent wants a generation longer than 10 seconds. Check the prompt first."
  },
  "approval_timeout": "2h"
}

Use whichever fields the method actually accepts, which swy info will show you. Held calls go to Slack or Telegram and fail closed, which is the right default when the alternative is an unattended spend. The schema is in the policy reference and the broader argument in a prompt is not a policy.

From an MCP client

Generation is a fun MCP tool and a slightly awkward one, because a long job does not fit neatly into a chat turn. The pattern that works: one tool submits and returns the ID, another checks status, and you ask for the status when you feel like it rather than making the assistant wait. Setup per client is in the MCP setup guide, the reference in the MCP docs, and skills.md has the conventions if you want a coding agent to build it.

What you can build

1. A brief-to-asset pipeline. A marketing brief lands, the agent writes the prompts, submits one generation per variant with a key per variant, and a worker collects the results and posts them for review. The agent never waits and never resubmits.
2. Storyboard expansion. Take a script, split it into shots, generate a reference image per shot, and assemble a contact sheet. Batch work where latency does not matter, which is the ideal shape for an async API.
3. Product video variants for testing. Same product, five framings, generated overnight on a schedule with a spend cap in policy, results dropped into Notion or your asset store for a morning review.
4. A failure-triage agent. Read failure_code across recent generations and group them: content-policy rejections need a prompt rewrite, transient errors need a retry, quota errors need a billing conversation. Three very different actions that all look like "it failed" without reading the code.

FAQ

Can I get a webhook instead of polling?

The documentation we reviewed does not describe callbacks, so assume polling and check their current docs before designing around a webhook. If you do get one later, the same dedup rule applies to inbound deliveries, which we cover in best webhook infrastructure for agent triggers.

How often should the agent poll?

The agent should not poll at all. A scheduler or workflow step should, on a widening interval, with a hard ceiling. Keeping the wait outside the model is the entire trick, and it also makes the cost predictable.

What if the same prompt genuinely needs running twice?

Then put the thing that makes them different into the key: a variant number or attempt counter you control. The point of a deterministic key is that accidental repeats collapse and deliberate repeats do not, and that distinction has to be explicit.

How do I keep an eye on spend?

swy audit network shows every generation that was actually submitted, which is the number that maps to your bill, and swy audit policy shows the ones a rule stopped. If the two numbers differ a lot, your policy is doing real work. More on what to capture in API observability for AI agents.

Related reading

• Luma integration page
• Using the Hugging Face API from an AI Agent
• Making API Calls Boringly Reliable
• JavaScript runtime SDK (docs)
• Browse all 300+ API integrations