The short answer
- An AI agent guardrail is a rule the agent must stay inside, checked by something outside the agent.
- Guardrails on words shape what a model says. Guardrails on actions check what an agent does, before it takes effect.
- Agents need guardrails in three places: what comes in, what the agent does, and what goes out.
- Prompt injection may never be fully solved, so limit what a tricked agent can do. Give each guardrail four answers: allow, fix, hold or block.
What are AI agent guardrails?
An AI agent guardrail is a rule the agent must stay inside, checked by something outside the agent. “Never delete more than a hundred rows without a person” is one. A rule written only into the agent's own instructions is a request, because the agent can be talked out of it.
Why outside? The UK National Cyber Security Centre warns that prompt injection may never be totally mitigated, and advises fixed safeguards, not built on a language model, that limit what the system can do. OWASP's prompt injection entry says it is unclear whether any method prevents it completely.
This guide is part of our AI agent security guide.
What are AI guardrails?
AI guardrails are the checks that keep an AI system inside the limits you set. Some shape what a model says, such as topics it avoids or formats it follows. Others check what an AI agent does, such as the commands it runs, the data it changes and the replies it sends.
What does “guardrails” mean in AI?
The word comes from the barrier at the side of a road. It doesn't steer the car. It stops the car leaving the road when something goes wrong. In AI it means limits that hold whatever the model decides. A barrier is no substitute for a careful driver, so give each agent only the access its job needs, then check what it does with it.
What are guardrails in an LLM application and what are their benefits?
They are checks placed around the model: on the prompt, on the answer, and on any tool the model may call. The model alone can't be trusted to follow its instructions. NIST's Generative AI Profile notes these systems are open to indirect prompt injection, where attackers plant prompts in data the application is likely to retrieve.
- Fewer harmful actions. A rule stops the change you can't undo.
- Fewer leaks. Card numbers, keys and personal data stay out of replies.
- A person where it counts. The few actions that matter wait for a yes.
- Evidence. Every decision leaves a record you can show a reviewer.
Guardrails on words, guardrails on actions
Most early guardrails worked on words. They checked a prompt for a known attack, or an answer for a banned topic. Agents also act. They run commands, change data and call other systems, and a check on words can miss a harmful action because the words that led to it looked harmless.
The OWASP Top 10 for Agentic Applications names risks such as agent goal hijack, tool misuse, identity and privilege abuse, and unexpected code execution. Each one ends in something the agent does. So agents need a guardrail that judges the action itself, before it takes effect. Some people call this an agent firewall.
LLM guardrails vs AI agent guardrails
| LLM guardrails | AI agent guardrails | |
|---|---|---|
| What they check | Text going into or out of the model | The action: a command, a change, a tool call, a reply |
| When | Before or after the model answers | Before the action takes effect |
| What a miss costs | A bad answer | A changed system, lost data or a leak |
| Usual answers | Allow or refuse | Allow, fix, hold for a person, or block |
Most teams need both. They catch different kinds of failure.
What are the different types of guardrails in an LLM?
| Type | What it checks | Example rule |
|---|---|---|
| Input | Prompts, documents and tool results | Treat instructions in outside content as data |
| Output | Answers and messages | Remove card numbers from a reply |
| Topic and format | What the model talks about, and how | Stay on the subject of your products |
| Action | Commands, data changes and tool calls | Hold any delete with no filter for a person |
The OWASP Top 10 for LLM Applications 2025 maps onto these. Prompt injection is an input problem, sensitive information disclosure an output problem, and excessive agency an action problem. The list also covers unbounded consumption, so limits on spending and volume are guardrails too.
Where guardrails belong
Think of an agent as a worker with an inbox, a set of tools and an outbox. Each needs its own check.
1. What comes in
Documents, web pages and tool results can carry instructions meant for the agent, not for you. OWASP's prompt injection entry advises separating and clearly marking untrusted content. Simon Willison calls one mix the lethal trifecta: private data, untrusted content, and a way to send data out. With all three, an attacker can trick the agent into sending your private data to them.
2. What the agent does
Every command, data change, system change and tool call. OWASP's Excessive Agency entry traces this harm to too much functionality, too many permissions and too much autonomy. It advises putting authorisation in the systems the agent calls, not leaving the model to decide what is allowed. The 2025 paper Defeating Prompt Injections by Design takes the same line, enforcing security policies at the moment a tool is called.
3. What goes out
Replies can carry a key, a card number or a link nobody approved, and a sent reply cannot be unsent. OWASP's entry on sensitive information disclosure names personal data, financial details, health records and security credentials among what can leak.
What guardrails must stop
- Destructive changes. A delete with no filter, a dropped table, or removed backups.
- Changes that open the door. A server opened to the internet, or more access than the task needs.
- Leaks. Keys, passwords, card numbers and customer data in a reply or a log.
- Orders from strangers. Instructions hidden in what the agent reads.
- Actions nobody approved. Anything your rules say a person must see first.
What are some examples of AI guardrails?
- “Hold any delete or update with no filter for a person.”
- “Keys never go into code, commit messages or logs.”
- “Nothing opens to the internet without a person.”
- “Card numbers never go to a customer. Remove them, and send the rest.”
Each names one action, one limit and one answer. For twelve you can copy, see our real guardrail examples.
Four answers, not two
A guardrail that can only say yes or no gets switched off. If it blocks a whole helpful reply over one bad line, people route around it.
- Allow. Safe work goes straight through.
- Fix. Remove the part that breaks the rule, and let the rest through.
- Hold. Wait for a person to say yes, with the exact action shown.
- Block. Stop it, and tell the agent why so it can try a safer way.
Use holds sparingly. For systems it classes as high risk, the EU AI Act's Article 14 asks that overseers can override the output, stop the system with a “stop” button or a similar procedure, and stay aware of automation bias. Too many requests, and people approve without reading. See human approval for AI agents.
How do I add guardrails to my AI agent?
List every action your agent can take. Mark the ones that could do real harm. Write a plain rule for each, with a named owner. Put the check outside the agent, where it sees every action. Then review held actions and rules every month, and remove the rules nobody needs.
How to create AI guardrails
Start with deletes, payments, access changes and anything sent outside the company. Write each rule so a machine can check it: “no filter” can be checked, “unusual” can't. Then give it an owner.
The NIST AI Risk Management Framework asks that roles and responsibilities for AI risk are documented and clear (GOVERN 2.1), and for mechanisms to disengage or deactivate AI systems that behave outside their intended use (MANAGE 2.4). In practice: an owner for every rule, and a way to stop every agent at once.
How to use LLM guardrails
Roll them out in stages. Run every rule in watch-only mode first. Switch on the blocks for things that should never happen, then the fixes, then the holds.
Test with attacks that adapt. In The Attacker Moves Second, researchers bypassed 12 recent defences against jailbreaks and prompt injection, with attack success above 90% for most. Simon Willison puts it plainly: in web application security, 95% is a failing grade. Check that the harmful action is stopped even when the words get through.
The UK NCSC's guidelines for secure AI system development add the basics: access controls on your APIs, models and data, secure defaults, and responders trained for AI incidents.
Where the checks run matters too. Read self-hosted guardrails before you choose, and see policy enforcement for AI agents for how rules are checked at the moment an agent acts.
Why do chatbots have guardrails?
Because a model left alone will sometimes say things nobody wanted. NIST's Generative AI Profile lists 12 risks that generative AI creates or makes worse, including confabulation (confident but false content), dangerous or hateful content, and data privacy. And a chatbot speaks in your company's name, so customers read its answers as yours.
Does ChatGPT have guardrails?
Yes. OpenAI's Model Spec sets out how its models should behave, and names ChatGPT among the products it covers. It sets rules that conversation cannot override, tells the model to refuse some requests, and gives instructions found in files, tool results and web pages no authority by default.
Those guardrails belong to the model maker. They don't know your rules: which tables matter, or who may approve a refund. Whatever model your agents use, your own rules need their own check, outside the model.
What is the 30% rule in AI?
There is no standard rule by that name. We know of no standards body that defines one, and none of the frameworks cited here uses the term. Online, it is used loosely, most often for a split where AI does about 70% of a task and people keep about 30%. Treat it as a rule of thumb. For guardrails, the useful question is which actions need a person before they run.
Where Cognilox fits
Cognilox, the verification and defense layer for AI agents, made by Aptibit Technologies, puts guardrails on actions. Before any agent action takes effect, it checks it against your company's rules. It tests risky database and infrastructure changes before they run. Then it allows, fixes, blocks, or holds the action for human approval.
When a reply contains a card number or a key, Cognilox can remove just that part and let the rest through. Start in watch-only mode: Cognilox shows what it would have done, without stopping anything. One switch stops every agent at once.
Run it inside your walls, fully self-hosted or air-gapped, or in Cognilox Cloud. When you host it yourself, none of your data comes to us. See the product, or book a live demo.
Cognilox runs in production on our own AI agents every day. We were its first customer.
See guardrails on real actions.
A live demo on our own AI agents, with your questions answered.
Questions
What are AI agent guardrails?
Rules an AI agent must stay inside, such as “never delete more than a hundred rows without a person”. Something outside the agent checks each rule every time.
Are guardrails the same as an AI agent firewall?
An agent firewall is one kind of guardrail. It checks actions before they take effect. Other guardrails shape what a model says.
Can I put guardrails in the agent's prompt?
You can, but the agent can ignore them or be tricked out of them. A guardrail that matters is checked outside the agent.
Are there AI models with no guardrails?
We don't rank or name models. Some openly released models can be run with little or no safety filtering, and any model's own guardrails can be talked around. That is why your rules should sit outside the model.
Sources
- OWASP Top 10 for LLM Applications 2025, OWASP Gen AI Security Project
- LLM01:2025 Prompt Injection, OWASP Gen AI Security Project
- LLM02:2025 Sensitive Information Disclosure, OWASP Gen AI Security Project
- LLM06:2025 Excessive Agency, OWASP Gen AI Security Project
- OWASP Top 10 for Agentic Applications, OWASP Gen AI Security Project
- Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, NIST
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, NIST
- Prompt injection is not SQL injection (it may be worse), UK National Cyber Security Centre
- Guidelines for secure AI system development: Secure deployment, UK National Cyber Security Centre
- Article 14: Human Oversight, EU Artificial Intelligence Act, artificialintelligenceact.eu (Future of Life Institute)
- The lethal trifecta for AI agents, Simon Willison
- Defeating Prompt Injections by Design, arXiv (Debenedetti et al.)
- The Attacker Moves Second (adaptive attacks against LLM jailbreak and prompt injection defences), arXiv (Nasr, Carlini, Tramèr et al.)
- Model Spec, OpenAI
Keep reading
In this series
This guide supportsVerify before it runs