ModelRefs / VQA (Visual Question Answering) — AI Glossary

VQA (Visual Question Answering) — AI Glossary

A benchmark asking models to answer open-ended natural language questions about images, requiring joint vision-language understanding.

Overview

VQA v2 (Goyal et al. 2017) contains 1.1M questions on MS COCO images, balanced to reduce language prior exploitation. Accuracy on open-ended VQA v2 has progressed from 65% (early fusion models) to 85%+ for GPT-4V and Gemini. Foundation for evaluating multimodal reasoning capabilities.

Reference details

Topic	evaluation
Last reviewed	2026-06-24

Example: Balanced pairs, and the prior they were built to break

Early visual question answering could be gamed without looking at the image: “what colour is the banana” is answered correctly by a model that has learned bananas are yellow. The fix was structural — for each question, include a second image where the same question has a different answer. Now a model that ignores the image is right about half the time on that pair instead of nearly always. That construction is the benchmark's real contribution, and it generalises: whenever a multimodal eval is easy for a text-only model, the eval is measuring the language prior.

Commonly confused with

Visual question answering is a task, and the dataset of that name is one instance of it. Chart and document question answering are separate benchmarks with different failure modes — reading a value off an axis is not the same skill as recognising an object, and a model can be strong at one and weak at the other.

When to use it

Reach for it when:

Checking that a multimodal model actually conditions on the image
Regression testing a vision encoder or projection layer change
As a floor check, alongside document and chart benchmarks that probe different skills

Reach for something else when:

As a summary of multimodal ability — the question distribution is narrow
Judging OCR or dense-text reading, which it barely exercises
Comparing figures without the answer-normalisation rules, which move results materially

Primary source

Goyal et al., Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to VQA (Visual Question Answering) — AI Glossary.

Frequently asked questions

What is VQA (Visual Question Answering)?