ModelRefs / llama.cpp — AI Glossary
llama.cpp — AI Glossary
A CPU/GPU inference engine for quantized LLMs in pure C/C++, enabling local inference without Python or ML frameworks.
Overview
llama.cpp (Gerganov 2023) implements efficient GGML tensor operations for quantized LLM inference. Supports GGUF model format with INT4/INT8/BF16 quantization, partial GPU offloading (layers on GPU, rest on CPU RAM), Metal (Apple Silicon), CUDA, and Vulkan backends. The foundation of LM Studio, Ollama, and Jan.
Reference details
| Topic | infrastructure |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to llama.cpp — AI Glossary.
Frequently asked questions
What is llama.cpp?
A CPU/GPU inference engine for quantized LLMs in pure C/C++, enabling local inference without Python or ML frameworks.
What concepts are related to llama.cpp?
Closely related concepts include gguf, ollama, mlc llm.