ModelRefs / Modal — AI Glossary
Modal — AI Glossary
A serverless cloud platform for running Python workloads including ML training and inference with GPU/CPU autoscaling.
Overview
Modal (2022) lets developers run Python functions on cloud GPUs with @app.function() decorator—no Dockerfile, no infra setup. Features: persistent volumes, GPU cold start in <500ms (cached containers), secret management, and scheduled jobs. Widely used for LLM inference APIs, batch embedding jobs, and ML training workflows.
Reference details
| Topic | infrastructure |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Commonly confused with
General serverless compute for Python, not an LLM API — which is the distinction against Together, Fireworks or Bedrock. You bring the code and the model and it supplies autoscaling GPUs, so it is the right tool for custom inference, batch jobs and training, and the wrong comparison for a hosted endpoint you call with a model name. Compare it with RunPod and other GPU platforms, not with model providers.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Modal — AI Glossary.
Frequently asked questions
What is Modal?
A serverless cloud platform for running Python workloads including ML training and inference with GPU/CPU autoscaling.
What concepts are related to Modal?
Closely related concepts include replicate, runpod, serverless inference.