ModelRefs / GPTQ — AI Glossary

GPTQ — AI Glossary

Q: What concepts are related to GPTQ?

Closely related concepts include awq, quantization, gguf.

A one-shot post-training quantization method for LLMs achieving near-lossless INT4 compression using second-order weight updates.

Overview

GPTQ (Frantar et al. 2022) quantizes LLMs to INT4 with <1% perplexity increase by using Hessian-based layer-wise reconstruction. It processes one layer at a time, compensating for quantization error. GPTQ models on HuggingFace are widely used for 4-bit inference on consumer GPUs via AutoGPTQ and ExLlamaV2.

Reference details

Topic	training
Last reviewed	2026-06-24

Commonly confused with

GPTQ and AWQ both produce near-lossless 4-bit weights after training, and differ in how they decide what to protect. GPTQ reconstructs layer outputs using second-order information about the weights; AWQ identifies the small fraction of weights the activations actually depend on and keeps those at higher precision. Neither is a format — GGUF is a container, and either method's output can end up in one.

Primary source

Frantar et al., GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to GPTQ — AI Glossary.

Frequently asked questions

What is GPTQ?