AI agents promise something more useful than a chatbot: software that can search, read documents, use applications, and complete multi-step tasks. An agent might compare travel options, sort support requests, update a spreadsheet, or help write and test code. The change is not merely that AI can produce language. It is that AI can increasingly connect language to actions.
That makes the central safety question surprisingly familiar. The important issue is not whether an agent can be persuaded to say something odd. It is whether it has the authority to do something consequential when it has been confused, misled, or given incomplete instructions. As agents gain access to email, files, calendars, browsers, databases, and payment tools, sound design depends on permission boundaries: clear limits on what the system may read, change, send, or spend.
A conventional chat assistant mainly returns text. It can be wrong, biased, or unhelpful, but its output usually remains a suggestion for a person to assess. An agent adds a loop: it interprets a goal, gathers information, selects a tool, takes a step, observes the result, and continues. This can save time because the user no longer has to perform every click.
But that loop changes the consequences of ordinary model errors. A mistaken answer in a chat window may waste a few minutes. A mistaken decision by an agent with access to a customer database or an email account can alter records, expose information, or send a message before anyone notices. Security therefore cannot be treated as a final review of the model’s wording. It has to be built into the tools and permissions surrounding the model.
NIST describes agent hijacking as a form of indirect prompt injection: malicious instructions placed in material an agent reads can steer it toward unintended actions. The affected material need not look technically suspicious. It could be a webpage, document, calendar invitation, support ticket, or email that the agent was legitimately asked to process.
People naturally distinguish between an instruction and a piece of content. A message from a manager might tell an employee what to do; a customer review is merely information to evaluate. Language models do not possess that distinction as a built-in security boundary. They process both kinds of text in a shared working context, then attempt to infer what matters.
That is the opening exploited by prompt injection. Imagine an agent asked to summarize vendor proposals. One proposal contains hidden or conspicuous text telling the agent to ignore the comparison task, search connected files for confidential pricing, and email it elsewhere. The instruction is not authorized simply because it appears in a document, but the model may still treat it as relevant language unless the system is designed to resist it.
This is closer to social engineering than to a simple bad command. A malicious page may impersonate urgency, claim to be an administrator, offer a plausible explanation for an unusual request, or ask the agent to complete a seemingly routine verification. OpenAI argues that effective real-world prompt injections increasingly rely on this sort of contextual manipulation, which makes perfect detection unrealistic. A system may not always be able to classify every sentence as safe or malicious before acting.
Because no filter can reliably recognize every deceptive instruction, the safer design principle is least privilege. Give an agent only the access needed for its defined job, for only as long as it needs it. This is a longstanding security idea, but it becomes especially important when software is interpreting open-ended natural language from many sources.
An agent that only needs to search a public website should not also have access to a personal inbox. An agent that drafts a reply should not automatically send it. An agent that can create a purchase order should face a strict spending cap and should not be able to add itself as an approved vendor. Narrow permissions do not eliminate errors, but they reduce the harm an error can cause.
OWASP identifies “excessive agency” as a risk when an AI system is granted tools or authority beyond what a task requires. The problem can be triggered by malicious prompting, but it can also arise from ambiguity, faulty reasoning, or a plain hallucination. That is an important distinction: permission boundaries are useful even when nobody is attacking.
Human approval is often described as keeping a person “in the loop,” but that phrase can hide an important design choice. A useful confirmation screen tells the person exactly what will happen: which account will be used, which recipients will receive information, what data will be included, and what money or records will change. A vague button labeled “Continue” does little to restore control.
The best moments for confirmation are not every minor step. Constant pop-ups encourage people to approve automatically. Instead, systems should pause when an action crosses a meaningful boundary: sending information outside an organization, making a purchase, deleting material, changing permissions, or committing an irreversible transaction.
OpenAI’s published safety guidance similarly emphasizes reviewing consequential actions and avoiding instructions that give an agent broad latitude, such as asking it to examine an inbox and take whatever action seems necessary. Specific assignments are easier for a user to inspect and easier for a system to constrain.
Permission design works better when users can see what an agent did and why. An activity record should show the sources consulted, tools called, actions proposed or completed, and any permissions used. For higher-risk work, it should be possible to inspect the exact external material that influenced a decision.
This is not merely a feature for incident investigations. Visibility improves normal use. If an agent recommends a supplier, a user should be able to distinguish a conclusion drawn from stated criteria from one that followed irrelevant text inside a webpage. If it changed a spreadsheet, a reviewer should be able to see which cells and formulas changed. Traceability turns an opaque automation into a process people can supervise.
Builders also need to test agents in hostile conditions rather than assuming ordinary benchmarks are enough. NIST’s work on agent hijacking evaluations and OWASP’s agent-security guidance both point toward testing untrusted inputs, tool access, data handling, and multi-step behavior together. A model that refuses an obvious malicious prompt may still fail when that prompt is embedded in a realistic workflow.
It is tempting to describe agent safety as a contest between smarter models and cleverer attacks. Better model behavior matters, and researchers continue to improve training, monitoring, and red-teaming. But an organization does not need to wait for a model that never makes a mistake before using automation productively.
The more practical path is graduated autonomy. Start an agent in an advisory role. Allow it to prepare drafts, collect options, or flag exceptions. Then permit low-risk, reversible actions under tight limits. Only after testing and review should it receive broader authority. Access should expand in response to demonstrated reliability, not because a tool happens to offer an integration.
That approach also clarifies responsibility. The person or organization deploying an agent decides what it may connect to, what it may do without approval, and what evidence it must leave behind. Better prompts can improve a task. Permission boundaries determine the blast radius when improvement is not enough.
AI agents will be most trustworthy not when they appear fully independent, but when their independence has well-marked edges. Treat emails, webpages, documents, and other outside material as potentially untrusted; give agents narrow, temporary authority; require meaningful approval for consequential actions; and preserve a record of what occurred.
Those practices may sound less dramatic than an all-purpose digital assistant. They are also what makes useful automation compatible with the ordinary expectation that software should not silently exceed the job it was asked to do.
Photo by Matheus Bertelli on Pexels.
A growing number of AI features no longer begin by sending your request to a distant data center. They begin inside the device in your hand: a phone, laptop, earbud case, camera,…
Passkeys remove one of the internet’s oldest security traps: the reusable secret. Instead of typing a password that a convincing fake site can capture and reuse, a person unlocks…
For a long time, computer security was organized around a simple picture: the organization owned a network, a firewall guarded its edge, and the systems inside that edge were…