Guardrails 101: what a guardrail is and what it checks

Companies adopt AI assistants and agents because they answer customers faster, read documents in seconds and take repetitive work off their teams. A guardrail is what lets them do it with confidence. It is a checkpoint that sits between people (or agents) and a language model, looks at what goes in and what comes out, and applies the company’s rules: this data stays private, this topic is out of scope, this instruction is not allowed to take control of the assistant.

A good guardrail is invisible to normal users. Most messages pass straight through. It acts on the few that carry a risk, and it acts in proportion: hiding a card number is better than refusing the whole question.

Two checkpoints: input and output

• Input checks look at what is sent to the model: the user’s message, but also the documents, web pages and tool results the application adds to the prompt. They catch attempts to manipulate the model, and personal data that should never reach it.
• Output checks look at what the model answers, ideally while the answer streams. They catch a secret, a customer’s data or off-policy content before the reader sees it.

The two work as a pair. An input check stops most problems early, when it is cheapest. An output check is the safety net for anything that was only visible in the answer, such as a model that was tricked into repeating its own instructions.

Three different problems

OWASP lists prompt injection as the first risk for applications built on large language models, and describes jailbreaking as a form of prompt injection that makes the model disregard its safety protocols ^[1]. Research on why safety training fails points to two causes: the model’s goals compete (be helpful, be safe), and its safety training does not generalise to every kind of input, such as encoded text ^[2]. Keeping the three problems apart matters because each needs a different kind of check, and because a guardrail that confuses them tends to block ordinary questions.

Direct and indirect

A direct attack is typed by the person talking to the assistant. An indirect attack is hidden in content the assistant reads on someone’s behalf: an email it summarises, a PDF uploaded to a knowledge base, a web page an agent visits, the result of a tool call, or a message from another agent. Greshake and colleagues showed that instructions planted in such content can steer an application that retrieves it, without the user typing anything ^[3].

• Documents and knowledge bases: white text, comments, metadata or invisible characters can carry instructions a human reader never sees.
• Tool outputs: an API result or a search result is data, but a model may read an instruction inside it as an order.
• Agents: an agent that can send email, run code or call other agents turns a successful injection into an action. Messages between agents deserve the same scrutiny as user input.

One well-studied defence is to mark untrusted content clearly as data, so the model can tell it apart from the instructions it should follow; Microsoft researchers call this family of techniques spotlighting ^[4]. A guardrail adds a second line: it scans that content for instructions before the model reads it.

What a guardrail is not

• It is not a replacement for good application design. Give an agent only the tools and data it needs, keep secrets out of prompts, and validate what it sends to other systems.
• It is one layer in a defence in depth. NIST’s taxonomy of attacks on AI systems treats these as open problems where defences reduce risk rather than remove it ^[5].
• It does not decide what is true. Checking facts in an answer is a different job.