ModelRefs / Code Generation — AI Glossary

Code Generation — AI Glossary

LLM-driven production of complete code files, functions, or programs from natural language specifications or docstrings.

Overview

Code generation models (GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, DeepSeek-Coder-V3) generate complete implementations from natural language descriptions. Evaluated by Pass@k on HumanEval, MBPP, LiveCodeBench, and BigCodeBench. Production use cases: scaffolding new features, generating boilerplate, writing unit tests, and code translation.

Reference details

Topic	applications
Last reviewed	2026-06-24

Example: What the benchmark does not measure

A function that passes every provided test can still hard-code the test cases, ignore error handling, or import a package that is not in your lockfile. Pass@k measures whether generated code satisfies given tests — not whether it is safe, maintainable, or consistent with the codebase around it. Review effort is the real cost, and it does not appear in the score.

Commonly confused with

Code generation is not code understanding. A model can write a plausible implementation without having understood the surrounding system — which is why generation quality on a benchmark and usefulness in a large repository diverge sharply.

When to use it

Reach for it when:

Boilerplate, scaffolding, test stubs and translation between languages
The output will be reviewed before it runs
The task is well specified and locally scoped

Reach for something else when:

Nothing reviews the output before it reaches production
The change spans a system the model cannot see
Correctness is safety-critical and the tests are the model's own

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Code Generation — AI Glossary.

Frequently asked questions

What is Code Generation?