Embeddings
Embeddings turn text into vectors for search and retrieval. They're usually far cheaper per token than generation.
Definition
Embeddings are numeric vectors that represent the meaning of text. They power semantic search and retrieval.
Why it matters for cost
Embedding models are typically much cheaper per token than chat or generation models. Using embeddings for retrieval—then generating only on the matched chunks—keeps generation token counts low.
Practical tip
Cache embeddings for stable documents so you don't re-embed the same content on every request.
Related reading
Batch APIs process large async request sets at a discount. Ideal for jobs that can tolerate delayed results.
The context window is the maximum tokens a model can see in one request. Bigger windows cost more per call when filled.
Fine-tuning trains a model on your data. It can cut prompt size and calls, but has upfront training cost to weigh.
Distillation trains a small model to mimic a large one, trading some quality for much lower per-token cost at scale.