ModelRefs / LiteLLM — AI Glossary

LiteLLM — AI Glossary

An open-source Python library and proxy server providing a unified OpenAI-compatible interface across 100+ LLM providers.

Overview

LiteLLM (BerriAI 2023) translates OpenAI API calls to any provider format (Anthropic, Cohere, Bedrock, Ollama, Hugging Face). As a proxy server it adds: virtual keys, cost tracking, rate limiting, fallback routing, and logging. The most-starred open-source AI gateway; used both as a library and as a self-hosted gateway.

Reference details

Topic	infrastructure
Last reviewed	2026-06-24

Commonly confused with

Two things ship under this name and the choice between them matters: a Python library you import to normalise provider calls in-process, and a proxy server you run so any client can reach many providers through one endpoint. The library adds no hop; the proxy adds one and gives you central keys, budgets and logging. Portkey and a hosted gateway occupy the same role as the proxy form.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to LiteLLM — AI Glossary.

Frequently asked questions

What is LiteLLM?