Long-Context Model — AI Glossary

Reference details

Related terms

• Context Window
• Sliding Window Attention
• RoPE (Rotary Positional Embedding)
• KV Cache

Example: The cost of just pasting everything in

A 200-page document is roughly 100K tokens. At $3 per million input tokens that is about $0.30 per question, every question, before any answer is generated. Retrieval over the same document sends perhaps 4K tokens for around $0.012. Long context is not free storage — you re-pay for the whole document on every single call.

100,000 tokens x $3 / 1,000,000 = $0.30 per question
  4,000 tokens x $3 / 1,000,000 = $0.012 per question  # ~25x cheaper

Commonly confused with

A large context window is not the same as reliable use of it. Advertised limits describe what the model accepts, not what it attends to evenly — recall commonly sags for material in the middle. A million-token window does not promise a million tokens of usable attention.

When to use it

Reach for it when:

• The task genuinely needs the whole document at once — global summarisation, cross-references
• Chunking would sever relationships the answer depends on
• The corpus is small and stable enough that re-sending it is affordable

Reach for something else when:

• The same corpus is queried repeatedly; you pay for it every time
• Only a few passages are relevant — retrieval is cheaper and usually more accurate
• The critical detail would land mid-context, where recall is weakest

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Long-Context Model — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path