ModelRefs / LLM Observability — AI Glossary
LLM Observability — AI Glossary
Collecting traces, logs, and metrics across LLM calls to understand system behavior, diagnose failures, and measure output quality.
Overview
LLM observability instruments inputs, outputs, latency, token usage, tool calls, and retrieval quality for every request. Platforms: LangSmith, Langfuse, Weave, Arize Phoenix. Complements traditional APM tools which lack LLM-specific semantics.
Reference details
| Topic | operations |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Commonly confused with
Monitoring tells you a known metric moved; observability is being able to ask why afterwards, including questions you had not thought to ask in advance. Tracing is the mechanism that makes it possible for multi-step applications, by joining spans under one request. For LLM systems the spans carry prompts and outputs, which is why this is also a store of user data and needs the same handling.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to LLM Observability — AI Glossary.
Frequently asked questions
What is LLM Observability?
Collecting traces, logs, and metrics across LLM calls to understand system behavior, diagnose failures, and measure output quality.
What concepts are related to LLM Observability?
Closely related concepts include tracing, llmops, eval.