Skip to content

Guardrails

AI agent guardrails: where they belong and what they must stop

Guardrails keep an AI agent inside the lines you set. Here is what they are, where they belong, what they must stop, and how to add them this week.

Updated , with the Cognilox team

The short answer

  • An AI agent guardrail is a rule the agent must stay inside, checked by something outside the agent.
  • Guardrails on words shape what a model says. Guardrails on actions check what an agent does, before it takes effect.
  • Agents need guardrails in three places: what comes in, what the agent does, and what goes out.
  • Prompt injection may never be fully solved, so limit what a tricked agent can do. Give each guardrail four answers: allow, fix, hold or block.
Cognilox is our product, so read our views on it with that in mind.

What are AI agent guardrails?

An AI agent guardrail is a rule the agent must stay inside, checked by something outside the agent. “Never delete more than a hundred rows without a person” is one. A rule written only into the agent's own instructions is a request, because the agent can be talked out of it.

Why outside? The UK National Cyber Security Centre warns that prompt injection may never be totally mitigated, and advises fixed safeguards, not built on a language model, that limit what the system can do. OWASP's prompt injection entry says it is unclear whether any method prevents it completely.

This guide is part of our AI agent security guide.

What are AI guardrails?

AI guardrails are the checks that keep an AI system inside the limits you set. Some shape what a model says, such as topics it avoids or formats it follows. Others check what an AI agent does, such as the commands it runs, the data it changes and the replies it sends.

What does “guardrails” mean in AI?

The word comes from the barrier at the side of a road. It doesn't steer the car. It stops the car leaving the road when something goes wrong. In AI it means limits that hold whatever the model decides. A barrier is no substitute for a careful driver, so give each agent only the access its job needs, then check what it does with it.

What are guardrails in an LLM application and what are their benefits?

They are checks placed around the model: on the prompt, on the answer, and on any tool the model may call. The model alone can't be trusted to follow its instructions. NIST's Generative AI Profile notes these systems are open to indirect prompt injection, where attackers plant prompts in data the application is likely to retrieve.

  • Fewer harmful actions. A rule stops the change you can't undo.
  • Fewer leaks. Card numbers, keys and personal data stay out of replies.
  • A person where it counts. The few actions that matter wait for a yes.
  • Evidence. Every decision leaves a record you can show a reviewer.

Guardrails on words, guardrails on actions

Most early guardrails worked on words. They checked a prompt for a known attack, or an answer for a banned topic. Agents also act. They run commands, change data and call other systems, and a check on words can miss a harmful action because the words that led to it looked harmless.

The OWASP Top 10 for Agentic Applications names risks such as agent goal hijack, tool misuse, identity and privilege abuse, and unexpected code execution. Each one ends in something the agent does. So agents need a guardrail that judges the action itself, before it takes effect. Some people call this an agent firewall.

LLM guardrails vs AI agent guardrails

LLM guardrails and AI agent guardrails compared
LLM guardrailsAI agent guardrails
What they checkText going into or out of the modelThe action: a command, a change, a tool call, a reply
WhenBefore or after the model answersBefore the action takes effect
What a miss costsA bad answerA changed system, lost data or a leak
Usual answersAllow or refuseAllow, fix, hold for a person, or block

Most teams need both. They catch different kinds of failure.

What are the different types of guardrails in an LLM?

Four types of guardrails
TypeWhat it checksExample rule
InputPrompts, documents and tool resultsTreat instructions in outside content as data
OutputAnswers and messagesRemove card numbers from a reply
Topic and formatWhat the model talks about, and howStay on the subject of your products
ActionCommands, data changes and tool callsHold any delete with no filter for a person

The OWASP Top 10 for LLM Applications 2025 maps onto these. Prompt injection is an input problem, sensitive information disclosure an output problem, and excessive agency an action problem. The list also covers unbounded consumption, so limits on spending and volume are guardrails too.

Where guardrails belong

Think of an agent as a worker with an inbox, a set of tools and an outbox. Each needs its own check.

1. What comes in

Documents, web pages and tool results can carry instructions meant for the agent, not for you. OWASP's prompt injection entry advises separating and clearly marking untrusted content. Simon Willison calls one mix the lethal trifecta: private data, untrusted content, and a way to send data out. With all three, an attacker can trick the agent into sending your private data to them.

2. What the agent does

Every command, data change, system change and tool call. OWASP's Excessive Agency entry traces this harm to too much functionality, too many permissions and too much autonomy. It advises putting authorisation in the systems the agent calls, not leaving the model to decide what is allowed. The 2025 paper Defeating Prompt Injections by Design takes the same line, enforcing security policies at the moment a tool is called.

3. What goes out

Replies can carry a key, a card number or a link nobody approved, and a sent reply cannot be unsent. OWASP's entry on sensitive information disclosure names personal data, financial details, health records and security credentials among what can leak.

What guardrails must stop

  • Destructive changes. A delete with no filter, a dropped table, or removed backups.
  • Changes that open the door. A server opened to the internet, or more access than the task needs.
  • Leaks. Keys, passwords, card numbers and customer data in a reply or a log.
  • Orders from strangers. Instructions hidden in what the agent reads.
  • Actions nobody approved. Anything your rules say a person must see first.

What are some examples of AI guardrails?

  • “Hold any delete or update with no filter for a person.”
  • “Keys never go into code, commit messages or logs.”
  • “Nothing opens to the internet without a person.”
  • “Card numbers never go to a customer. Remove them, and send the rest.”

Each names one action, one limit and one answer. For twelve you can copy, see our real guardrail examples.

Four answers, not two

A guardrail that can only say yes or no gets switched off. If it blocks a whole helpful reply over one bad line, people route around it.

  • Allow. Safe work goes straight through.
  • Fix. Remove the part that breaks the rule, and let the rest through.
  • Hold. Wait for a person to say yes, with the exact action shown.
  • Block. Stop it, and tell the agent why so it can try a safer way.

Use holds sparingly. For systems it classes as high risk, the EU AI Act's Article 14 asks that overseers can override the output, stop the system with a “stop” button or a similar procedure, and stay aware of automation bias. Too many requests, and people approve without reading. See human approval for AI agents.

How do I add guardrails to my AI agent?

List every action your agent can take. Mark the ones that could do real harm. Write a plain rule for each, with a named owner. Put the check outside the agent, where it sees every action. Then review held actions and rules every month, and remove the rules nobody needs.

How to create AI guardrails

Start with deletes, payments, access changes and anything sent outside the company. Write each rule so a machine can check it: “no filter” can be checked, “unusual” can't. Then give it an owner.

The NIST AI Risk Management Framework asks that roles and responsibilities for AI risk are documented and clear (GOVERN 2.1), and for mechanisms to disengage or deactivate AI systems that behave outside their intended use (MANAGE 2.4). In practice: an owner for every rule, and a way to stop every agent at once.

How to use LLM guardrails

Roll them out in stages. Run every rule in watch-only mode first. Switch on the blocks for things that should never happen, then the fixes, then the holds.

Test with attacks that adapt. In The Attacker Moves Second, researchers bypassed 12 recent defences against jailbreaks and prompt injection, with attack success above 90% for most. Simon Willison puts it plainly: in web application security, 95% is a failing grade. Check that the harmful action is stopped even when the words get through.

The UK NCSC's guidelines for secure AI system development add the basics: access controls on your APIs, models and data, secure defaults, and responders trained for AI incidents.

Where the checks run matters too. Read self-hosted guardrails before you choose, and see policy enforcement for AI agents for how rules are checked at the moment an agent acts.

Why do chatbots have guardrails?

Because a model left alone will sometimes say things nobody wanted. NIST's Generative AI Profile lists 12 risks that generative AI creates or makes worse, including confabulation (confident but false content), dangerous or hateful content, and data privacy. And a chatbot speaks in your company's name, so customers read its answers as yours.

Does ChatGPT have guardrails?

Yes. OpenAI's Model Spec sets out how its models should behave, and names ChatGPT among the products it covers. It sets rules that conversation cannot override, tells the model to refuse some requests, and gives instructions found in files, tool results and web pages no authority by default.

Those guardrails belong to the model maker. They don't know your rules: which tables matter, or who may approve a refund. Whatever model your agents use, your own rules need their own check, outside the model.

What is the 30% rule in AI?

There is no standard rule by that name. We know of no standards body that defines one, and none of the frameworks cited here uses the term. Online, it is used loosely, most often for a split where AI does about 70% of a task and people keep about 30%. Treat it as a rule of thumb. For guardrails, the useful question is which actions need a person before they run.

Where Cognilox fits

Cognilox, the verification and defense layer for AI agents, made by Aptibit Technologies, puts guardrails on actions. Before any agent action takes effect, it checks it against your company's rules. It tests risky database and infrastructure changes before they run. Then it allows, fixes, blocks, or holds the action for human approval.

When a reply contains a card number or a key, Cognilox can remove just that part and let the rest through. Start in watch-only mode: Cognilox shows what it would have done, without stopping anything. One switch stops every agent at once.

Run it inside your walls, fully self-hosted or air-gapped, or in Cognilox Cloud. When you host it yourself, none of your data comes to us. See the product, or book a live demo.

Cognilox runs in production on our own AI agents every day. We were its first customer.

See guardrails on real actions.

A live demo on our own AI agents, with your questions answered.

Questions

What are AI agent guardrails?

Rules an AI agent must stay inside, such as “never delete more than a hundred rows without a person”. Something outside the agent checks each rule every time.

Are guardrails the same as an AI agent firewall?

An agent firewall is one kind of guardrail. It checks actions before they take effect. Other guardrails shape what a model says.

Can I put guardrails in the agent's prompt?

You can, but the agent can ignore them or be tricked out of them. A guardrail that matters is checked outside the agent.

Are there AI models with no guardrails?

We don't rank or name models. Some openly released models can be run with little or no safety filtering, and any model's own guardrails can be talked around. That is why your rules should sit outside the model.

Sources

Keep reading

In this series

This guide supportsVerify before it runs