Your AI Bill Is Too High, and It's One Setting

If you run anything at volume through an LLM, there is a good chance you are paying roughly double what you need to. Not because you picked the wrong model, and not because your prompts are bloated. Because of a single quiet default: you are running everything in real time.

Here is the thing most teams never stop to check.

Most AI work doesn't need to be instant

Think about what you actually send to a model. A chatbot answering a user mid-conversation needs to be fast. But that is usually the small slice. The bulk of AI workloads look more like this:

• Summarizing yesterday's support tickets
• Tagging or classifying a backlog of documents
• Generating embeddings for a knowledge base
• Extracting fields from invoices or forms
• Drafting content variants for next week's campaign
• Scoring call-center transcripts for QA

None of that needs a sub-second response. Nobody is refreshing a page waiting on the model. These jobs can finish in minutes or overnight and nothing breaks. Yet most teams run them through the same real-time API they use for live traffic, and pay the real-time premium on every token.

The setting: batch inference

Every major provider — Anthropic, AWS Bedrock, OpenAI — offers a batch (asynchronous) mode, and they price it the same way: 50 percent off their on-demand rates. Same models, same quality, same tokens, half the price. The only trade is that results come back asynchronously, usually within an hour for moderate volumes and guaranteed within 24 hours.

That is the setting. For any workload that tolerates a little latency, batch mode is a flat 50 percent discount hiding in plain sight.

The savings can go further than 50 percent, because two things stack on top:

1. Model routing. Not every job needs your most expensive model. A tagging or classification task often runs well on a smaller open-weight model. Route each job to the cheapest model that clears the quality bar and the savings compound.
2. Prompt caching. If your batch jobs share a large system prompt or context, caching that portion drops its cost to a fraction of full price, and it stacks with the batch discount.

Between the 50 percent batch base, smart model choice, and caching, real-world savings land in the 30 to 80 percent range depending on the workload. There is no quality tradeoff. You run the same models, you just stop paying an urgency tax you do not owe.

So why doesn't everyone do this?

Because batch APIs make you do the work. To use them directly, you have to collect requests and hold them somewhere, decide when to fire a batch, format and submit the job, poll for completion or build a system to track status, and handle partial failures, retries, and timeouts.

That is a real piece of infrastructure. It is why so many teams know about batch pricing and still do not use it: the discount is real, but the engineering to capture it never reaches the top of the sprint. The teams that do build it end up maintaining a queue system forever. We wrote more about that tradeoff in when to use batch inference.

The shortcut

This is the gap Convoy fills. You send individual requests to one endpoint. Convoy holds them, groups them into batches automatically (100 requests or one hour, whichever comes first), runs them at batch pricing, and delivers results to your callback URL. There is no queue to build, no batch window to tune, no retry logic to write.

curl -X POST https://api.cnvy.ai/cargo/load \
  -H "Content-Type: application/json" \
  -H "X-API-Key: convoy_sk_your_key_here" \
  -d '{
    "params": {
      "model": "claude-sonnet-4-5",
      "max_tokens": 1024,
      "messages": [{"role": "user", "content": "Summarize this ticket: ..."}]
    },
    "callback_url": "https://your-server.com/callback"
  }'

You get a cargo_id back immediately, and the finished result arrives on your webhook. Switch models per request with no code changes, and send one request or a million — there is no minimum volume.

Where to start

Pick your highest-volume, least time-sensitive workload: the batch job you already run on a schedule, or the backlog you process in bulk. That is the one quietly costing you the most. Move it to batch and you capture most of the savings with almost none of the effort.

Your AI bill is too high, and it is probably one setting. Create a Convoy account and run your first batch in minutes, or see the pricing.