Temperature
Temperature controls output randomness. Lower values are cheaper to reason about and more predictable for cost planning.
Definition
Temperature sets how random a model's output is. Near 0 is deterministic; higher values are more varied.
Why it matters for cost
Temperature doesn't directly change the token rate, but high randomness can lead to longer, less focused outputs—raising output token cost. Lower, focused settings tend to produce shorter, more predictable responses.
Practical tip
Use low temperature for structured, repeatable tasks to keep outputs tight and costs stable.
Related reading
Batch APIs process large async request sets at a discount. Ideal for jobs that can tolerate delayed results.
The context window is the maximum tokens a model can see in one request. Bigger windows cost more per call when filled.
Embeddings turn text into vectors for search and retrieval. They're usually far cheaper per token than generation.
Fine-tuning trains a model on your data. It can cut prompt size and calls, but has upfront training cost to weigh.