Skip to content

Guardrails

Self-hosted AI guardrails: what to check before you choose

Guardrails can run on your own servers, in a vendor's cloud, or inside the agent itself. Each choice changes where your data goes and who can see it. This guide compares the three, then gives you a test checklist and a rollout plan.

Updated , with the Cognilox team

The short answer

  • Guardrails read everything your agents send and receive, so where they run decides where that data goes.
  • Self-hosted guardrails keep the checks, the data and the records on your own servers. Your team takes on the upkeep.
  • Checks that live inside one agent cover only that agent. A separate checkpoint covers every agent you send through it.
  • Check actions, not only text, and test every claim yourself. Then roll out in watch-only mode first.
Cognilox is our product, so read our views on it with that in mind.

What are self-hosted AI guardrails?

Self-hosted AI guardrails are checks you install and run on your own servers. They sit between your AI agents and the systems those agents use. They read what agents receive and try to do, then allow it, change it or stop it. The checks, the data they read and the records they keep all stay inside your network.

Teams choose self-hosted AI when data cannot leave. The same logic applies to the guardrails. For the wider picture, start with agentic AI security and AI agent guardrails.

Why does it matter where guardrails run?

A guardrail has to read what your agents do: prompts, replies, data changes and sometimes secrets. Whoever runs the guardrail can see all of it. Self-hosted AI guardrails keep both the checks and that data on your own servers. That often decides the whole purchase.

That traffic is what OWASP warns about. Its entry on sensitive information disclosure lists personal data, financial details, health records and security credentials. A guardrail sees all of these by design.

If a vendor's guardrail reads personal data on your behalf, it may be acting as your processor. Under UK GDPR, the ICO says a controller that uses a processor needs a written contract, and so does a processor that uses another processor.

The guardrail is a supplier in its own right. NIST AI 600-1, the Generative AI Profile of the NIST AI Risk Management Framework, lists 12 risks. One is value chain and component integration, which includes poor vetting of suppliers. Vet your guardrail vendor like any supplier that sees your most sensitive traffic.

Where can AI guardrails run?

On your own servers

You install the guardrail inside your network, on your own hardware or cloud account. The checks and records stay with you. Your team runs and updates it.

In a vendor's cloud

The vendor runs it for you, and your agents' traffic passes through its systems. You do far less work, and you trust the vendor with more: its staff, its suppliers and where it keeps your data.

Inside the agent, or a library in your app

Some agent frameworks have checks built in, and most open-source guardrail projects start as a library you add to your code. They are quick to turn on. They cover only the apps that include them, and they run inside the process they check.

OWASP's guidance on excessive agency shows why that matters. It advises putting authorisation in downstream systems rather than relying on the language model to decide whether an action is allowed. A check in a separate process is harder for a hijacked agent to switch off or talk its way around.

How do the three options compare?

Where AI guardrails run, side by side
On your own serversIn a vendor's cloudInside the agent or app
Where your data goesStays inside your networkPasses through the vendorWherever the app runs
Who holds the recordsYouThe vendor, under contractYou, if you keep them
Works with no internetYes, if built for itNoDepends on the framework
Which agents it coversEvery agent sent through itEvery agent sent through itOnly apps that include it
Separate from the agentYesYesNo, unless run as a service
UpkeepHighestLowestLow per app, growing with each app
Good fit forRegulated data, offline sitesTeams that want no upkeepOne team, one framework

No column wins every row. Choose by where your data may go and how many kinds of agent you run.

Why self-host AI guardrails?

  • Your data stays home. Prompts, replies and records never leave your network, so the guardrail adds no outside company to your data map.
  • It can run offline. Sites with no internet connection still get the checks.
  • It stays separate from the agent. One checkpoint covers every agent routed through it, whatever built it.
  • You own the evidence. Records of what was allowed, fixed or stopped sit with you.

The cost is real. Your team sizes, updates and watches the guardrail. If it fails, agents either stop or carry on unchecked, so decide which in advance.

The UK NCSC's guidelines for secure AI system development ask for access controls on your APIs, models and data, defaults that are secure, and responders trained to handle AI incidents. When you self-host, those jobs are yours.

Can guardrails run in an air-gapped network?

Some can. Many call a hosted model, check a licence online or fetch updates from the internet. Ask four questions.

  1. Does every check work with no outbound connection? Including any check that uses a language model.
  2. What happens to the licence? If it cannot reach the vendor, does any protection switch off?
  3. How do updates arrive? Can you check each release came from the vendor before you install it?
  4. Where do the records go? They should stay on the isolated network.

Then test it: block all outbound traffic, run a few days of real work through it, and see what breaks.

What should self-hosted guardrails check?

The first guardrails were text filters. Agents act, so they need more.

  • What comes in. Prompts, documents and tool results, for hidden instructions. See prompt injection in AI agents.
  • What goes out. Replies, for secrets, card numbers and personal data. See secrets leaking from AI agents.
  • Commands and tool calls. What the agent runs, and with which arguments.
  • Data and infrastructure changes. Deletes, schema changes and infrastructure plans, before they apply.
  • Spending and volume. Limits on cost and action counts.
  • Approvals. The few actions that need a person first.

OWASP now keeps a separate list for agents. The OWASP Top 10 for Agentic Applications, built with more than 100 experts and published in December 2025, makes a good coverage map.

Regulation points the same way. For systems it classes as high risk, the EU AI Act asks that the people overseeing them can interrupt the system through a stop button or a similar procedure, and stay aware of automation bias. Article 15 asks those systems to resist attempts by unauthorised third parties to alter their use, outputs or performance. A guardrail is one place where duties like these are met.

Open-source guardrails or a product?

Good open-source projects exist. Three come up often.

  • NeMo Guardrails is an open-source toolkit for adding programmable guardrails to language model apps, under the Apache 2.0 licence. It runs as a Python library, or as its own server with an HTTP API. Its rails run at different stages: input, retrieval, dialogue, execution and output.
  • Guardrails AI is a Python framework, also under Apache 2.0. Its input and output guards are built from validators shared on Guardrails Hub, and it can run as a standalone service with a REST API.
  • LlamaFirewall, from Meta's PurpleLlama project, is a framework for agent security. Its scanners include PromptGuard 2 for direct prompt injection, AlignmentCheck for whether an agent's reasoning still matches what the user asked, and CodeShield for security issues in generated code.

You can host all three and read their code. You take on the work around them: wiring them into each agent, tuning checks, adding approvals and records, and answering the pager.

A product should earn its place by doing that work: one checkpoint for every agent, readable rules, approvals, records and support. Judge both against the same checklist.

How do you test guardrails before you choose?

Ask to see each answer working. Run these tests on a trial system.

An evaluation checklist for self-hosted AI guardrails
CheckHow to test itWhat good looks like
Data stays homeBlock outbound traffic and watch the logs.Checks work. Nothing tries to leave.
Actions, not only textHave an agent try to drop a table.Stopped or held before it runs.
Hidden instructionsHide an instruction in a document, then reword it ten ways.The action the attack wanted is stopped.
Secrets in repliesPlant a fake key and test card number, then ask for them.The secret is removed. The rest arrives.
Readable rulesAsk for every rule in force.None are hidden.
CoverageRoute agents from two frameworks through it.Both are checked.
More than blockTrigger a fixable risky action.Fixed or held, and the agent told why.
RecordsRead a record, then try to edit one.The edit is visible.
Licence lapseLet a test licence expire.Protections stay on.
Stop switchUse the control that stops every agent.One action, every agent stopped.
ReleasesAsk how to check a release is genuine.Signed, and you can check it.

Be wary of a single detection rate. In The Attacker Moves Second, a 2025 paper, researchers including Nicholas Carlini and Florian Tramèr bypassed 12 recent defences against jailbreaks and prompt injection, with attack success above 90% for most. Most of those defences had first reported near zero attack success.

Simon Willison makes the same point: in web application security, catching 95% of attacks is a failing grade. Test with attacks that adapt, and check the harmful action is stopped even when the text gets through. For rules to try, see guardrails examples and our agent firewall guide.

How do you roll guardrails out without breaking work?

Turn everything on at once and you will block good work on day one. Go in stages.

  1. List your agents. Every agent, its tools and the data it can reach.
  2. Start in watch-only mode. It records what it would have done, without stopping anything. Give it a week of normal work.
  3. Review what it would have stopped. Fix rules that fire on good work.
  4. Enforce the clear cases first. Secrets and card numbers in replies, destructive commands and deletes. Defaults like these should not wait long.
  5. Add human approval for a short list. Keep it to high-impact actions such as production changes and payments. Too many requests train people to approve without reading, a form of the automation bias the EU AI Act warns about. See human approval for AI agents.
  6. Set limits and practise the stop. Cap spending, then drill stopping every agent at once.
  7. Review every month. The list from step one goes stale fast.

Where Cognilox fits

Cognilox is the verification and defense layer for AI agents, made by Aptibit Technologies. Run it inside your walls, fully self-hosted or air-gapped, or in Cognilox Cloud. When you host it yourself, none of your data comes to us. Aptibit has no access to your self-hosted installation.

Before any agent action takes effect, Cognilox checks it against your company's rules. It tests risky database and infrastructure changes before they run. Then it allows, fixes, blocks, or holds the action for human approval.

When a reply contains a card number or a key, Cognilox can remove just that part and let the rest through. Records hold the decision, not the content. A secret Cognilox stopped is not sitting in a log.

Start in watch-only mode: Cognilox shows what it would have done, without stopping anything. One switch stops every agent at once. A lapsed licence never switches a protection off, and its status is always visible. Every Cognilox release is signed, so you can check it came from us.

See the deployment options on Inside your walls, and read our security page.

Cognilox runs in production on our own AI agents every day. We were its first customer.

See it run inside your walls.

A live demo on our own AI agents. Tell us where yours run, and we will talk through the fit.

Questions

Are self-hosted guardrails harder to run?

Yes, somewhat. Your team installs and updates them. In return, your data does not pass through another company.

Are open-source guardrails self-hosted?

Usually, yes. You install and run them yourself, so the data stays with you. Most start as a library inside the application, so they cover only the apps that include them. Some, such as NeMo Guardrails and Guardrails AI, can also run as a separate service.

Do guardrails replace least privilege?

No. Give each agent only the tools and access its job needs, then add guardrails on top. OWASP names excessive permissions as one root cause of excessive agency. A guardrail is a second line, not a substitute.

Does a guardrail need to keep copies of our prompts?

Not always. Ask what each record holds. The ICO's guidance on AI says to process only the personal data you need for your purpose, and a log full of prompts and replies can hold far more than an audit needs.

Sources

Keep reading

In this series

This guide supportsInside your walls