ModelRefs / Multimodal Assistant — Canonical Workflow
Multimodal Assistant — Canonical Workflow
Canonical Multimodal Assistant workflow: vision/audio/text models, guardrails and deployment.
Overview
A multimodal assistant accepts images, audio and text and produces grounded answers, edits or generations. The canonical stack uses a frontier multimodal model with modality-specific pre/post-processing and content safety guardrails.
Implementation profile
| Category | multimodal-models |
|---|---|
| Implementation maturity | production |
| Evidence status | incomplete |
| Primary use cases | agents, extraction, ocr |
| Deployment options | managed-api, hybrid |
| Architectures | serverless-api, managed-container |
Candidate models with published references
- GPT-5
- GPT-5 Mini
- Claude Opus 4
- Llama 4 Scout
- o3
- o4 Mini
- Claude Sonnet 4
- Claude 3.5 Sonnet
- Claude 3 Haiku
- Gemini 2.5 Pro
- Gemini 2.5 Flash
- Gemma 3 27B
Coverage means the model is a candidate worth evaluating for this workflow, not a ranking or a recommendation. Models whose reference pages are still in review are omitted.
Benchmarks relevant to this workflow
swe-bench, aider-polyglot, gpqa, aime-2025, tau-bench, browsecomp-long-context, longfact-concepts, terminal-bench, mmmu, mmlu-pro, livecodebench, mmmu-pro, mathvista, chartqa.
Relevance is a coverage signal from the canonical registry. Each benchmark only describes its own protocol and date, so confirm the harness matches your workload before treating a score as evidence.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Multimodal Assistant — Canonical Workflow.