Distributed Tracing (LLM) — AI Glossary

Reference details

Related terms

• LLM Observability
• LLMOps

Example: Where the four seconds went

A response takes 4.2 seconds and the team starts optimising retrieval. The trace says otherwise: retrieval 0.3s, reranking 0.2s, generation 3.6s, serialisation 0.1s. Three quarters of the wall clock is one span, and no amount of index tuning touches it — the levers are a smaller model, a shorter output, or streaming so the user stops waiting for all of it. The value of tracing is not that it records the steps; it is that it attributes the time to one of them, which a log line per step cannot do because logs are not joined by a trace ID.

Commonly confused with

Traces, logs and metrics answer different questions. Metrics tell you latency rose. Logs tell you what one component printed. A trace tells you which span of which request caused it, across services. LLM tracing adds token counts, prompts and tool arguments to the span — which is also why traces need the same redaction as any other store of user text.

When to use it

Reach for it when:

• Multi-step pipelines — RAG, agent loops, anything with more than one model call
• Latency and cost attribution, where per-span token counts localise spend
• Debugging a bad answer after the fact, where you need the exact prompt that was sent

Reach for something else when:

• Single-call applications, where request logging is sufficient and cheaper
• As a store for unredacted user content — trace payloads are still personal data
• Full-fidelity capture at high volume without sampling, which becomes its own cost centre

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Distributed Tracing (LLM) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path