Convoy Is Live: Stop Building Batch Infrastructure

Today we're launching Convoy: a managed batch-processing layer for AI inference. You send your requests to a single API. We group them into batches automatically, run them through the model you choose, and deliver the results to your callback URL. You get batch pricing — up to 50 percent cheaper than real-time across every major provider — without building or maintaining a single queue.

Here is the whole thing:

curl -X POST https://api.cnvy.ai/cargo/load \
  -H "Content-Type: application/json" \
  -H "X-API-Key: convoy_sk_your_key_here" \
  -d '{
    "params": {
      "model": "claude-sonnet-4-5",
      "max_tokens": 1024,
      "messages": [{"role": "user", "content": "Hello"}]
    },
    "callback_url": "https://your-server.com/callback"
  }'

You get back a cargo_id, we handle everything from there, and the result lands on your webhook when it is done.

Why we built it

Batch inference is the obvious way to cut AI costs — providers hand you a 50 percent discount for work that does not need to happen in real time. And most AI work fits that profile. Data enrichment, document classification, embeddings backfills, and evaluation runs can all tolerate minutes or hours of latency, yet teams routinely pay full on-demand prices to run them.

The reason is that using batch APIs directly means building infrastructure: a queue to hold requests, batch windows to decide when to fire, retry logic, status tracking, and monitoring. Most teams either never get to it or build it once and maintain it forever. Either way, they leave money on the table. Convoy removes that work. You send a request and get a result, at batch prices.

What you get

• 30 to 80 percent lower AI spend — batch pricing plus model routing, no quality tradeoff.
• Zero batching logic — no queues, no timing windows, no infrastructure to manage.
• No minimum volume — send one request or a million; we pool requests so even small workloads get batch rates.
• Every leading model, one API — Claude, Llama, Mistral, Nova, DeepSeek and more, switchable per request with no code changes.
• Built-in reliability — automatic retries, exponential backoff, and status tracking on every job.

Two ways to run it

Convoy Cloud is live today with a free tier and no credit card required — sign up, grab an API key, and start sending. Convoy Enterprise self-hosts in your own AWS account via a Terraform module, so your spend and data stay with you, with SSO, RBAC, and audit logging included.

If your AI bill is bigger than you would like, start for free or read the docs. To go deeper on the economics, see why your AI bill is probably one setting.