ModelRefs / LLMOps — AI Glossary
LLMOps — AI Glossary
The discipline of deploying, monitoring, versioning, and iterating on LLM-powered applications in production. Also called LLM operations or AI ops.
Overview
LLMOps extends MLOps with LLM-specific concerns: prompt versioning, LLM-as-judge evals, retrieval pipeline monitoring, model drift detection, and cost optimization. Key tools: LangSmith, Weave (W&B), Braintrust, Arize Phoenix.
Reference details
| Topic | operations |
|---|---|
| Also known as | LLM operations, AI ops |
| Last reviewed | 2026-06-24 |
Related terms
Commonly confused with
The same discipline as MLOps applied where the model is called rather than trained, which moves the hard parts. Versioning covers prompts and retrieval corpora, not just weights; evaluation has no single accuracy number; and cost is per request rather than per training run. Observability is one component of it, not a synonym.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to LLMOps — AI Glossary.
Frequently asked questions
What is LLMOps?
The discipline of deploying, monitoring, versioning, and iterating on LLM-powered applications in production.
Is LLMOps the same as LLM operations?
Yes — LLM operations, AI ops are common aliases for LLMOps.
What concepts are related to LLMOps?
Closely related concepts include eval, tracing, prompt management, observability.