ModelRefs / MMMU Methodology — Methodology

MMMU Methodology — Methodology

MMMU evaluates multimodal university-level reasoning across 30 subjects requiring image understanding.

Overview

What it measures: Multimodal academic reasoning that requires integrating text and images.

How it works

  • ~11.5K questions across 30 subjects and 6 disciplines.
  • Each question includes one or more images (diagrams, charts, photos).
  • Multiple choice or short-answer.

Strengths

  • Genuine multimodal requirement
  • Wide subject coverage

Limitations

  • OCR-heavy items can be solved without vision
  • Subject distribution uneven

Best use cases

Multimodal model evaluation

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to MMMU Methodology — Methodology.

Frequently asked questions

What does MMMU measure?

Multimodal academic reasoning that requires integrating text and images.

What are its main limitations?

OCR-heavy items can be solved without vision Subject distribution uneven

When should I use this benchmark?

Multimodal model evaluation