ModelRefs / QLoRA — AI Glossary
QLoRA — AI Glossary
LoRA performed on top of a 4-bit quantized base model, enabling fine-tuning of 70B-parameter models on a single 48 GB GPU.
Overview
QLoRA (Dettmers et al., 2023) combines NF4 quantization, double quantization, and paged optimizers with LoRA adapters. Near-full-precision quality at a fraction of the VRAM cost.
Reference details
| Topic | training |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Commonly confused with
LoRA applied on top of a base model held in 4-bit precision, which is what makes single-GPU fine-tuning of large models feasible. The consequence people miss is that quantization does not disappear afterwards: the adapters were trained against quantized weights, so merging them into a full-precision base is not guaranteed to behave like the model you trained. Evaluate the deployment configuration, not the training one.
Primary source
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to QLoRA — AI Glossary.
Frequently asked questions
What is QLoRA?
LoRA performed on top of a 4-bit quantized base model, enabling fine-tuning of 70B-parameter models on a single 48 GB GPU.
What concepts are related to QLoRA?
Closely related concepts include lora, quantization, peft.