the guardrails

The agent I fiddle with most runs on the CRM and the customer portal of an installation firm. Reading records, staging configurations, creating new ones, all nice and proper through the official API. I built an MCP server for it. And the question I’ve gotten most often since? It isn’t about the code. It’s something else. How do you actually do this safely?

Good question. Because turning an AI loose on a client’s live system, that can go wrong in ways that make you wince. Only most companies go looking for the answer in the wrong spot.

And it starts right here, honestly: not every action deserves the same suspicion. Let an agent create a task, or a customer, fine, I just let it get on with it. But the second it touches a money process, an invoice, an amount, a payment, a whole lot more lights and checks come on for me than they would for some little task. That same distinction, you build it into your tooling. The small slip gets to move fast. The expensive one has to clear a gate first.

hands for an advisor

MCP, that stands for Model Context Protocol. An open standard that puts tools in a model’s hands. Look, a model like Claude is good at exactly one thing on its own: making text. Inside your systems it can’t do a thing. And that’s where the MCP sits. The bridge from the model to a real application, with a set of tools the AI is genuinely allowed to run. Look up a record. Stage a change. That kind of work.

The picture that keeps holding up for me: without an MCP, that AI is the advisor sitting next to you telling you what you ought to do. That’s it. Give it an MCP, and that same advisor suddenly has hands.

And the second it can actually carry out actions, the whole risk shifts shape. A wrong answer in a chat window? Annoying, otherwise harmless. But a wrong action in a live system touches real data, real customers, real processes. Work you’d normally do by hand, click for click, becomes repeatable and fast all of a sudden. Huge gain, that. Only that exact same lever blows the mistake up too.

the default mistake

The market’s reflex is to put the safety on the model. Better prompts. Stricter system instructions. Some little sentence somewhere urging the agent to take care. Or it swings the other way entirely, and then the agent gets no access at all. Tucked away safe in a sandbox, where it harms no one and, well, helps no one either.

Both of them miss the point. An instruction to a model, that’s an agreement. Not a guarantee. And an agent that can’t touch anything? That’s an expensive chatbot, nothing more.

So flip it around. The safety doesn’t sit in how clever the model is. It sits in the discipline you build into the tooling itself. Every tool you hand an agent is a spot where you get to enforce a rule. One the model can’t talk its way out of. The tool just plain won’t allow anything else.

six guardrails

This is the pattern I stick to myself. Six principles, together roughly one safety belt, and deliberately vendor-neutral. They do their job on SAP just as well as on any CRM you like, or just about any other system with an API. I’ll set them down in the order I usually build them in.

I always start with preview-then-apply. Because that one principle catches by far the most accidents. The agent never just writes straight through. Every change falls neatly into two steps. First it shows exactly what’s about to happen. Then a human approves it. And only after that does it execute. And the heavier the action, the stricter that moment gets: creating a customer slips right through, anything with money in it gets the full gate.

Then I draw a hard line around production. A bit of poking around in a test or acceptance environment, sure, fine. But writing to a live production environment? That ought to cost a separate, explicit confirmation. Landing in production by accident, that just can’t be allowed to happen.

Least privilege is the third. Really it’s just common sense out of the world of authorization roles. The agent gets access to what the task needs and not a millimeter more. On the systems where I run this they’re called scopes: only the permissions the action truly asks for. Look, you don’t hand all your colleagues admin rights either, right? An agent that’s allowed to create categories has no business getting anywhere near deleting contracts.

Then the audit trail. Less exciting, that one, sure. But you can’t do without it. Every action goes into the logbook. What happened, when, with what result. Because something goes wrong and you’ve logged nothing? Then you’re standing there empty-handed afterward. You can’t find the mistake, and recovering turns into pure guessing.

The fifth is read-back verification. Never blindly trust a system that reports “done.” Just have the agent read back afterward what’s really sitting in the system. And have it flag the moment that system quietly adjusted or dropped something on you. Because what you thought happened and what actually happened, those two drift apart more often than you’d like.

And sixth, a kill switch. A fail-safe that steps in the moment things really head south. Stopping after a string of failed attempts, say, instead of blindly shoving on. Because without a brake like that, an agent blows a little problem up into a big one.

how you find out which ones you need

Now that list maybe sounds like you know precisely, up front, which gates to set. Wish it were so. For us it works the other way round, actually. Early on we love letting the agent make loads of mistakes. Not on the live system, obviously. In the test. You let it wander around on its own in there, hours on end, and you just watch what breaks.

And every time it walks into something, you write it down. We call it a gotcha-section, that’s the name it goes by: a little list of pitfalls the agent keeps itself. It hits that same wall again, and next time it’s already standing there warned. That’s how it makes those pitfalls its own in the end. So the guardrails you’re left with aren’t the rules I sat and dreamed up behind my desk. They’re the rules the agent first showed me it needed by getting them wrong.

and then the chatbot

Up to here it’s been about an agent I steer myself, inside my own walls. But put an agent like that out front as a chatbot, where customers and total strangers talk to it straight, and the game changes. Because a chatbot agent like that often has access to a big chunk of your system. And the person questioning it sits outside your organization. So that’s exactly where you want a whole lot of specific guardrails, ones that protect your organization.

Take prompt injection. An outside user acts like they’re on the inside, and hands the chatbot an instruction it then just goes along with. Or jailbreaks: ready-made prompts that crack a model open. Those things are simply sitting there on the internet, you know. People have tested themselves silly on which little phrases make which model do what they want, until the thing suddenly does all sorts for them. And that’s exactly why you don’t trust the model on its honest face. You build the recognition of tricks like these into the tooling around it. Not into a friendly request to the model to behave itself.

limits as the condition for freedom

Running through all of it is the same thread. In doubt? Ask, and keep a human in the loop. That AI really isn’t dumb, you know. The point is simply that a mistake in a real system is too expensive to hand over entirely to some automatism.

And honestly? This discipline isn’t free. Preview-then-apply means someone has to keep looking. And someone who taps approve a hundred times a day, at some point they stop really looking. I get it. So the guardrails don’t take the thinking off your plate. You’ve got to keep weighing which actions truly deserve that human moment, and which ones you cover just fine with the other principles. That’s exactly why you weigh on risk. A little task is a different thing from an amount.

But the core of it stands. This discipline doesn’t slow the value down. It’s what makes it possible in the first place. Because whoever frames their agents properly dares to give them more freedom. Simply because the damage stays manageable when something does go wrong. Chain your agent up out of fear, and you get nowhere. Let it run completely loose out of naivety? Same result. Set the limits sharp. And inside those limits, suddenly almost anything can run.

Responsible AI isn’t a marketing term. And it’s certainly not a policy document sitting in a drawer. It’s a handful of concrete choices you lock into your tooling, before you ever let an agent do a single thing on a real system. Trust in an agent, then, isn’t some feeling you let wash over you. It’s a choice. One you make in the architecture.