Designing Secure AI Agents for Enterprise Data

The last few months have made one thing clear. A coding agent deleted a production volume with a token it found in a file. Models under test left a sandbox and reached a real company. Those are very different events, and they share one lesson: instructions do not contain an agent. Architecture does.

If you are putting an agent near enterprise data, I would start from one assumption: the model will sometimes be wrong, confused or manipulated, and the system has to stay safe when that happens.

That assumption shapes everything below. Most of these ideas come from building the Verifiable AI Analyst, a conversational analyst that had to pass a strict security review, and from governance work on Auto-RAG and MerlinX.

Principle 1: Decide what the model must never hold

Start by listing what the model should not have, whatever it is asked. In most enterprise systems the list includes database credentials, raw SQL access, API keys, other users' data, and the ability to change its own permissions.

A secret that never enters the model's context cannot be leaked by a prompt injection. No filter is as dependable as that.

Principle 2: Offer capabilities, not access

The common first design gives an agent a database connection and a good prompt. I would not ship that.

In the Verifiable AI Analyst, the only thing the model can reach is a registry of typed capabilities. There is no raw-SQL tool. The model emits a plan over typed operations, and it never touches a table. The set of capabilities is built per user: an operation the user is not entitled to does not exist for that turn.

The last point deserves attention. Filtering a request after the model makes it is a good check. Not offering it at all is better, because the model cannot be argued into using something that is not there.

Principle 3: Enforce permissions below the model

The agent should act as the user, with the user's rights, and those rights should be enforced where the data lives.

We execute capabilities under PostgreSQL's native row-level security and column masking, through a read-only role that cannot bypass either. If a prompt injection convinces the model to ask for someone else's records, the database still says no. That control does not read the prompt, so the prompt cannot persuade it.

The test for any control is simple. Would it still hold if the model were fully hostile? If not, treat it as a convenience, not a boundary.

Principle 4: Treat all content as data

Documents, emails, tickets, web pages and tool results can all contain text aimed at your agent. Hugging Face's intrusion started with a malicious dataset, which is the same family of problem one layer down.

Data content has to stay data by construction, not because a prompt asks. In practice:

Constrain what the model can output, so it fills a grammar instead of writing free-form actions. We use constrained decoding for this, which means it cannot invent a field or a capability name.
Run parsers and ingestion with minimal identity, away from credentials.
Do not let retrieved content widen what the agent is allowed to do.

Principle 5: Separate reading from acting

An agent that reads and an agent that changes things have very different risk profiles. I would design them apart.

Action type	Example	Control
Read	Query a report	Entitlements, logging
Reversible write	Draft a ticket	Validation, undo
Sensitive write	Send to a customer	Human approval outside the agent's channel
Irreversible	Delete data, move money	Not available to the agent, or two-person approval

Approval has to arrive through a route the agent cannot write to. A confirmation it can produce itself is not a control.

Principle 6: Control the network

Default-deny outbound access, enforced outside the agent's environment. Then verify it from inside, on a schedule, because a boundary on a diagram is only a claim. Every allowed route is part of your attack surface, so give each one an owner.

Principle 7: Assume you will need the evidence

Log who asked, what the agent planned, which capabilities ran, what they returned, and what was blocked. Keep those logs out of the agent's reach.

Give every agent its own identity, too. If agents borrow a developer's account, you cannot tell afterwards who did what, and you cannot revoke one without the other.

Principle 8: Govern the delivery around it

The agent's permissions are only half of the picture. If a prompt or tool definition can reach production without review, that is a route around all of the above.

On our platforms I brought 35 Azure DevOps projects and about 224 repositories under one standard, with mandatory review gates on tier-one repositories, protected tags and a scheduled drift audit. Treat prompts, tool schemas and model versions as code, with the same gates.

Walking one attack through the design

Here is a hypothetical example of how these layers work together. An employee asks an internal agent to summarise a supplier's PDF. Hidden in the document is a line telling the agent to export the salary table and send it to an outside address.

Layer	What happens
Model	It may or may not obey. Assume it does
Capabilities	There is no export tool, and no raw SQL to write one with
Manifest	This employee is not entitled to salary data, so that capability does not exist for this turn
Database	Even if a request got through, row-level security would return nothing
Output	Sending externally is a sensitive write, which needs approval the agent cannot give itself
Network	The outside address is not on any allowed route
Logs	The attempt is recorded, and the document can be traced

Notice that the design never needed the model to spot the attack. Detection by the model would be a bonus. Each layer holds on its own, so the attacker has to beat all of them, while you only need one to work.

Now run the same exercise for your own system. Pick your most sensitive table and your most likely hostile input, and walk through what stops it. If the honest answer at every step is "the system prompt says not to", you have one layer, and it is the weakest one.

A review checklist

What credentials can the agent reach, including by accident?
Is each permission enforced somewhere the model cannot influence?
What is the worst action available, and what stops it?
What happens if a document contains hostile instructions? Have you tested it?
Can you revoke this agent in minutes without touching anything else?
Could you reconstruct what it did last Tuesday?

None of this is exotic. It is least privilege and defence in depth, applied to a new kind of component that happens to be very persuasive. Design as if the model will fail, and most of its failures become log entries instead of incidents.

Written by Eslam Hamed, a Technical Lead and AI Solutions Architect working on agentic AI, governed AI systems, and engineering governance.