ModelRefs / GGUF — AI Glossary
GGUF — AI Glossary
A binary file format for storing quantized LLM weights optimized for CPU and edge inference with llama.cpp. GGUF replaced GGML in 2023. Also called GGML.
Overview
GGUF replaced GGML in 2023. It stores model weights, tokenizer, and metadata in a single self-contained file. Enables running 7B–70B models on consumer hardware. Ollama and LM Studio use GGUF internally.
Reference details
| Topic | infrastructure |
|---|---|
| Also known as | GGML |
| Last reviewed | 2026-06-24 |
Related terms
Example: One file, and why that is the point
A GGUF file carries weights, tokenizer and metadata together, so running a model is opening a file rather than assembling a directory of shards, configs and vocab files that must all match. That self-containment is why local runtimes standardised on it — the format removes an entire class of setup failure.
Commonly confused with
GGUF is a container format, not a quantization method. The quantization scheme (Q4_K_M, Q5_K_S and so on) describes how weights were compressed; GGUF describes how the result is stored. Two GGUF files of the same model can differ enormously in quality and size.
When to use it
Reach for it when:
- Running models locally through llama.cpp, Ollama or LM Studio
- CPU or mixed CPU/GPU inference on consumer hardware
- You want a single artifact to distribute
Reach for something else when:
- Server-class GPU serving, where vLLM-style formats are better suited
- You need to fine-tune — train first, convert afterwards
- Picking a quant level without checking quality on your own task
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to GGUF — AI Glossary.
Frequently asked questions
What is GGUF?
A binary file format for storing quantized LLM weights optimized for CPU and edge inference with llama.cpp.
Is GGUF the same as GGML?
Yes — GGML are common aliases for GGUF.
What concepts are related to GGUF?
Closely related concepts include quantization, ollama, open weights.