Tree of Thoughts (ToT) — AI Glossary

Reference details

Related terms

• Chain of Thought (CoT)
• Reasoning (LLM)
• Task Decomposition

Example: Count the calls before choosing to branch

Chain-of-thought explores one path. Branching three ways at each of three levels explores 3³ = 27 leaf paths, and each branch also needs an evaluation call to decide whether it is worth continuing — so the cost is tens of model calls where linear reasoning made one. Sometimes that is obviously right: puzzles where a wrong early commitment is fatal and backtracking is the whole game. Usually it is not, because most tasks do not have a dead end to back out of. Reach for it when you can name the branch point, and measure against plain chain-of-thought with the same total budget spent on sampling instead.

Commonly confused with

Tree of thoughts is a search procedure you implement around the model; extended thinking is a mode the model runs internally. Self-consistency sits between them — it samples several independent chains and votes, with no evaluation or backtracking between branches, which is far cheaper and often enough.

When to use it

Reach for it when:

• Problems with identifiable decision points where an early wrong turn cannot be recovered
• Tasks with a cheap way to score a partial solution, which is what makes pruning possible
• Where accuracy justifies an order-of-magnitude increase in calls

Reach for something else when:

• Anything a single chain-of-thought pass answers acceptably
• Without a usable evaluation signal — unpruned branching is just expensive sampling
• Latency-sensitive paths, where the call count lands directly on the user

Primary source

• Yao et al., Tree of Thoughts: Deliberate Problem Solving with Large Language Models (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Tree of Thoughts (ToT) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path