ModelRefs / LLM Security — AI Glossary

LLM Security — AI Glossary

The practice of securing LLM-based systems against adversarial inputs, data leakage, model abuse, and supply-chain vulnerabilities.

Overview

LLM security threats: prompt injection (hijacking agent actions via malicious instructions in retrieved content), indirect prompt injection (via web pages the agent reads), jailbreaking (bypassing safety training), data exfiltration (extracting training data or system prompts), and plugin/tool abuse. OWASP Top 10 for LLMs (2023) is the primary reference.

Reference details

Topic	safety
Last reviewed	2026-06-24

Example: Three properties that are only dangerous together

Prompt injection has no reliable fix, because a model cannot separate instructions from data when both arrive as text. What can be controlled is the combination that makes it exploitable: access to private data, exposure to untrusted content, and a way to send information outward. An agent that reads your email, browses arbitrary web pages, and can make outbound requests has all three, and a malicious instruction on a page it visits can move data out. Remove any one — no private data, or no untrusted input, or no exfiltration path — and the attack collapses. Design against that triad rather than trying to filter every hostile phrasing.

Commonly confused with

Prompt injection is not jailbreaking. Jailbreaking is a *user* trying to make the model misbehave for themselves; injection is a *third party* placing instructions in content the model reads, so the victim is the user. Guardrails and refusal training address the first and barely touch the second, which is why an aligned model is not a safe agent.

When to use it

Reach for it when:

Any agent that reads content it did not author — pages, documents, emails, tool output
Threat modelling before granting tools: what can this reach, and what can it send out
Reviewing tool permissions as a security boundary rather than a capability list

Reach for something else when:

Relying on prompt-level instructions to ignore injected commands — they are bypassable
Treating model alignment as a substitute for permissions and sandboxing
Granting broad tool access on the assumption that content the agent reads is trustworthy

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to LLM Security — AI Glossary.

Frequently asked questions

What is LLM Security?