How does an LLM take an action at all?
The model itself does not log into your systems. Through function or tool calling, it returns a structured request, such as create_ticket with a title and priority. OpenAI's function calling guide describes the flow: your application sends the available tools, receives the model's tool call, executes code on the application side and returns the result. Every action therefore passes through code you control, and the approval step, the permission check and the audit log all belong in that code. A design that hands the model broad credentials and lets it call any API removes those control points.
What can go wrong when an assistant has too much reach?
OWASP's Top 10 for LLM applications calls this risk excessive agency: damaging actions that happen because of unexpected or manipulated model output. It names three root causes, and each one has a plain business example, shown below. Check every proposed tool against all three before it goes live, and repeat the check whenever the tool, its permissions or the underlying system changes. The examples are illustrative and are not drawn from a real incident.
Prompt injection raises the stakes. OWASP describes indirect prompt injection, where content in an email, document or web page the assistant reads alters its behaviour. If the assistant can act, an instruction hidden in that content can become an action.
- Excessive functionality: a tool built for reading a mailbox can also send and delete messages.
- Excessive permissions: the tool connects with an administrator account when the user only needs their own records.
- Excessive autonomy: a refund or other high-impact action runs without a person checking it.
Which actions need a person to approve them?
Match oversight to impact. The Guidance for AI Adoption says human oversight should match how much autonomy the system has and how high the stakes are, ranging from automated monitoring for low-stakes uses to mandatory human review for high-stakes decisions. In practice, sort each proposed action by whether it can be undone, who it affects and what it costs if wrong. Drafting is low impact because nothing leaves the business. Sending, paying, deleting, changing a customer's record or deciding something about a person should wait for a named person to approve the exact action, shown in full, before it runs.
How should permissions work for each tool?
Give each tool the narrowest access that does its job. OWASP's mitigations include limiting extensions to the minimum necessary, preferring granular tools over open-ended ones, applying least privilege, executing actions in the user's own context and enforcing authorisation in the downstream system. In practice that means a tool called update_ticket_status instead of a general API caller, a connection that acts as the signed-in user instead of a shared service account, and a target system that rejects any request the user could not have made themselves.
- List each tool with its system, allowed operations and the data it can touch.
- Remove any operation the use case does not need, such as delete.
- Set spending or volume limits in the downstream system where it supports them.
- Review tool permissions when staff roles change.
What should be logged for each action?
For every proposed action, record the user, the request that led to it, the tool and parameters the model proposed, whether a person approved or rejected it, the result from the target system and the time. Store the record outside the assistant so it survives a reset or rebuild. These logs answer the questions that follow any mistake: who asked, what the assistant proposed, who approved it and what changed. OWASP notes that monitoring, logging and rate limits do not prevent excessive agency on their own, but they reduce the damage by making detection possible.
How do you introduce actions gradually?
Start with the assistant proposing actions that a person completes by hand, and measure how often the proposal was right. Move a single action to one-click approval once its proposals are reliable, then consider letting low-impact, reversible actions run without approval. Hypothetical example: an IT service desk first lets the assistant suggest ticket categories to agents, then create tickets after an agent approves, and only later close duplicate tickets automatically with a daily review. Each step is a separate decision with its own test. Some actions may never be automated, and that is a reasonable outcome.