FFmpeg for AI Agents: MCP Server and REST API

Agents write FFmpeg. They need a place to run it.

Language models know FFmpeg well. Ask one to make a 720p proxy, cut the first 30 seconds or burn in a logo, and you get a usable command. The problem is the next step: a chat client has no FFmpeg binary, and sandboxes and serverless functions have time, disk and memory limits that video jobs reach quickly.

Xora takes the same arguments the agent wrote and runs them in the cloud. The agent sends one create_job call, polls get_job until the job is done, and gets a signed URL for the output. Same args, different machine.

More on this: FFmpeg in AI agent sandboxes · LLMs write good FFmpeg commands

One job, from prompt to file

Agent writes FFmpeg args
→
create_job → { id, state: "queued" }
→
get_job until "completed"
→
Signed URL for the output

Three ways in

Pick the one your agent already speaks. All three reach the same jobs API, with the same limits and billing.

MCP server

Connect https://api.xora.sh/v1/mcp and the agent gets five tools: create_job, get_job, create_upload, list_recipes and cancel_job. Sign in with OAuth, or send an API key in the Authorization header.
REST API

POST /v1/jobs with JSON, then GET /v1/jobs/{id}. Agents with a shell or an HTTP tool can read /llms.txt and the OpenAPI spec for the full workflow.
Agent skill

A SKILL.md file for Claude: which recipe fits which ask, when to probe first, how to handle local files and how to branch on errors.

Connect in one command

In Claude Code, add the server with your API key. In Cursor, add the same URL and header to mcp.json. In claude.ai and Claude Desktop, add it as a custom connector and sign in to your Xora account — there is no key to copy.

The server sends its own instructions during the MCP handshake, so most agents find the workflow without help: list_recipes once, then create_job, then get_job until the state is terminal.

Step-by-step guides: Claude · Cursor · agent SDKs · all clients

claude mcp add --transport http xora https://api.xora.sh/v1/mcp \
  --header "Authorization: Bearer YOUR_API_KEY"

{
  "mcpServers": {
    "xora": {
      "url": "https://api.xora.sh/v1/mcp",
      "headers": { "Authorization": "Bearer YOUR_API_KEY" }
    }
  }
}

Raw FFmpeg, not a short list of actions

Some media APIs give agents a short list of fixed actions. Xora gives them FFmpeg. In mode: "ffmpeg" the agent sends an argument array, so filter graphs, multiple inputs and codec flags pass through as written. Each array element goes to FFmpeg as one argument — there is no shell string to quote or escape.

For the common jobs there are 9 recipes, such as compress, trim, thumbnail and webReady, and a probe mode that returns ffprobe metadata before the agent picks a command.

{
  "mode": "ffmpeg",
  "input_files": {
    "in_1": "https://example.com/talk.mp4",
    "in_2": "https://example.com/logo.png"
  },
  "output_files": { "out_1": "talk_branded.mp4" },
  "ffmpeg": {
    "args": [
      "-i", "{{in_1}}",
      "-i", "{{in_2}}",
      "-filter_complex", "[0:v][1:v]overlay=W-w-24:H-h-24",
      "-c:v", "libx264", "-crf", "23",
      "-c:a", "copy",
      "{{out_1}}"
    ]
  }
}

Built for an agent's loop

An agent can only recover from what it can read. Every part of a Xora job is explicit.

States the agent can branch on

A job moves through named states and ends in exactly one of completed, failed, rejected or cancelled. get_job also returns progress from 0 to 100. Pass webhookUrl to get a callback in place of polling.

Structured errors

Errors come back as JSON with a code: VALIDATION_ERROR means fix the request, LIMIT_EXCEEDED means the plan allowance is used up, PLAN_REQUIRED means the feature is not on the plan. The agent does not have to parse FFmpeg’s log to know what to do next.

Safe retries

Send an idempotencyKey with create_job. If the call times out and the agent tries again with the same key, Xora returns the job it already made — not a second job and a second bill.

Local files too

When the agent has a file path and no URL, create_upload gives it a presigned URL for one HTTP PUT. It then passes the uploadId to create_job. The file size is checked against the plan limit before the upload starts.

Same price as a REST job

An MCP job is a normal Xora job. It bills credits for machine time and output size, the same as a curl request, and the same plan limits apply. Failed, rejected and cancelled jobs cost nothing, so an agent that gets a command wrong does not pay for it.

The Free plan gives 25 credits every month with no card. See pricing for the paid plans.

What a job costs

Completed → credits for the work done
→
Failed → 0 credits
→
Rejected → 0 credits
→
Cancelled → 0 credits

Frequently asked questions

Does Xora have an FFmpeg MCP server?

Yes. Xora runs a remote MCP server at https://api.xora.sh/v1/mcp over Streamable HTTP. It has five tools — create_job, get_job, create_upload, list_recipes and cancel_job — and they call the same job routes as the REST API.

Which agents and clients can use it?

Any client that supports remote MCP servers over Streamable HTTP. The docs show setup for Claude Code, claude.ai and Claude Desktop, Cursor, and the MCP TypeScript SDK. Agents without MCP can call the REST API and read https://xora.sh/llms.txt for the full workflow.

Can the agent run any FFmpeg command, or only presets?

Any command. In mode "ffmpeg" the agent sends the FFmpeg arguments as a JSON array, including -filter_complex and multiple inputs. Recipes such as compress, trim and thumbnail are there for the common jobs, and probe mode returns ffprobe metadata.

What if the file is on the agent's machine and has no URL?

The agent calls create_upload to get a presigned URL, sends the file with one HTTP PUT, and then passes the uploadId to create_job in place of a URL. One upload can feed several jobs for three days.

Do MCP jobs cost more than REST jobs?

No. An MCP tool call creates the same job as a REST request and bills the same credits. Plan limits are the same too. Failed, rejected and cancelled jobs cost nothing.

FFmpeg for AI agents