Building AI agents that are safe to deploy inside an enterprise

An AI agent is a language model given tools and a loop: it decides, calls something, reads the result, and decides again. The moment you give it tools, it stops being a chatbot and becomes software with write access to your systems. Almost every failure we are asked to fix comes from teams that built the second thing while thinking about the first.

The demo-to-production gap

A demo succeeds when the model produces a plausible answer once, watched by the person who built it. Production means it runs unattended, thousands of times, against inputs nobody anticipated, including inputs written by people actively trying to break it.

The gap is not model quality. It is that a demo has no blast radius and a production agent does.

Scope tools, not prompts

The most common design error is trying to constrain behaviour with instructions — telling the model in its system prompt what it must not do. Prompts are not a security boundary. They are advisory text in the same channel as untrusted input, and prompt injection works precisely because the model cannot reliably tell the difference between your instructions and instructions embedded in a document it was asked to read.

Constrain the tools instead. If the agent should only refund orders under ₹5,000, the refund tool enforces that limit in code and rejects anything above it — regardless of how convincingly the model argues. If it should read from one table, its database credential grants access to one table. The model can then be wrong without being dangerous.

Make actions reversible, or gate them

Sort every action the agent can take into three buckets.

Reversible and low-cost — drafting text, retrieving records, running a read query. Let the agent do these freely.

Reversible but visible — sending an internal message, creating a ticket, updating a draft. Let it act, log everything, make undo trivial.

Irreversible or externally visible — issuing a payment, emailing a customer, deleting data, changing a production configuration. These need a human approving the specific action, not a general policy approved once at design time.

The design goal is that the worst plausible failure is embarrassing rather than expensive.

Evaluate before you deploy, not after

Teams routinely ship agents with no evaluation beyond manual spot-checks, then discover problems through customer complaints. Build an evaluation set early: fifty to two hundred real cases with known-good outcomes, including the awkward ones — ambiguous requests, missing data, adversarial phrasing, requests the agent should refuse.

Run it on every prompt change, every model version, every tool change. Model upgrades are not automatically improvements for your specific task, and without evaluation you will not know which direction you moved.

Track refusal behaviour explicitly. An agent that never refuses is not well-aligned, it is untested.

Log enough to reconstruct a decision

When an agent does something wrong, you need to answer: what did it see, what did it decide, what did it call, what came back. That means logging the retrieved context, the tool calls with arguments, and the results — not just the final output.

This is also where data governance bites. Those logs now contain whatever the agent retrieved, which may include personal or confidential data, and they fall under the same retention and access rules as any other store of it.

Where agents genuinely pay

The strongest cases we see are internal, high-volume and well-bounded: triaging and routing support tickets, extracting structured data from documents, reconciling records across systems, first-pass code review, drafting responses a human sends.

The weakest are open-ended agents pointed at production with broad permissions and a vague brief. Those demo well and fail quietly.

---

We build AI systems that are evaluated before they ship and scoped so failure is survivable. See our AI systems engineering work, or tell us what you are trying to automate.