Token
A token is the unit LLM providers bill by—roughly a word fragment. Token count drives nearly all AI API cost.
Definition
A token is the chunk of text an LLM reads and generates. In English, a token is about four characters or roughly three-quarters of a word. Other languages vary.
Why it matters for cost
Providers price by tokens, split into:
- Input tokens (what you send)
- Output tokens (what the model returns)
Your bill is essentially input tokens × input rate + output tokens × output rate, multiplied by how many calls you make.
Practical tip
Reducing token usage—shorter prompts, cached context, capped outputs—is the most direct way to lower AI API cost.
Related reading
Batch APIs process large async request sets at a discount. Ideal for jobs that can tolerate delayed results.
The context window is the maximum tokens a model can see in one request. Bigger windows cost more per call when filled.
Embeddings turn text into vectors for search and retrieval. They're usually far cheaper per token than generation.
Fine-tuning trains a model on your data. It can cut prompt size and calls, but has upfront training cost to weigh.