Prompt Caching
Prompt caching lowers the input cost of repeated context by caching a stable prefix across requests.
Definition
Prompt caching lets a provider reuse a stable prefix of your prompt (system instructions, reference docs) across requests instead of re-billing it at the full input rate.
Why it matters for cost
Cached tokens are billed at a much lower rate than fresh input tokens—often cutting input cost by 50–90% for repetitive workloads.
Practical tip
Keep stable content at the top of your prompt and avoid changing it between calls so the cache keeps hitting.
Related reading
Batch APIs process large async request sets at a discount. Ideal for jobs that can tolerate delayed results.
The context window is the maximum tokens a model can see in one request. Bigger windows cost more per call when filled.
Embeddings turn text into vectors for search and retrieval. They're usually far cheaper per token than generation.
Fine-tuning trains a model on your data. It can cut prompt size and calls, but has upfront training cost to weigh.