Model Gateway (LLM Gateway) — AI Glossary

Reference details

Related terms

• Model Routing
• Fallback Chain
• Semantic Cache
• Rate Limiting

Example: Failover that does not need a deploy

The primary provider returns 529s. With provider calls spread across services, recovery means shipping code under pressure. Behind a gateway it is a configuration change in one place, and the applications never learn that anything moved.

Commonly confused with

Model gateway, LLM gateway and AI gateway name the same component; the vocabulary has not settled. What it is not is a router — routing is one capability a gateway may offer, alongside auth, caching, quotas and observability.

When to use it

Reach for it when:

• Several applications call providers and need consistent policy
• Failover, quotas or cost attribution must be centrally controlled
• You want provider-agnostic application code

Reach for something else when:

• A single application with one provider
• The extra hop breaks a tight latency budget
• It would become an unmonitored single point of failure

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Model Gateway (LLM Gateway) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path