Sampling (LLM) — AI Glossary

Reference details

Related terms

• Temperature
• Top-P (Nucleus Sampling)
• Min-P Sampling

Example: The two knobs, and the order they apply

Temperature rescales the distribution before selection: low sharpens toward the top token, high flattens it. Top-p then truncates to the smallest set of tokens whose probabilities sum to p. Raising temperature widens the field and top-p trims the tail back — which is why changing one without the other rarely does what people expect.

Commonly confused with

Sampling is not the same as temperature. Temperature is one parameter that reshapes the distribution; sampling is the act of drawing from it, and top-k, top-p and min-p all constrain that draw independently. Greedy decoding is the limiting case where nothing is sampled at all.

When to use it

Reach for it when:

• You want variation across runs — brainstorming, drafting, synthetic data
• The task has many acceptable answers rather than one
• You need diversity for self-consistency voting

Reach for something else when:

• Extraction, classification or anything where reproducibility matters
• You are debugging: nondeterminism makes a failure much harder to isolate
• Sampling parameters are being tuned before the prompt has been fixed

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Sampling (LLM) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path