State of AI Agent Security 2026 Commentary: Why the account/authority/suspension loop must be designed first before the model to increase agents

Publication date: 2026-06-30 | Category: ai News

1. One-line problem definition

Key line: The number of AI agents in enterprises is growing rapidly, but the scope of actual control by security teams is not keeping pace.

The State of AI Agent Security Report, released by Gravitee on June 15, 2026, surveyed 750 senior technology leaders in the UK and the US as of April 2026. The report notes that the enterprise AI agent set has doubled in about four months since December 2025, and that an average of 48% of agents in production environments are running without security or governance.

The target audience of this article is CTO, platform team, security officer, and senior developer who want to add agents to in-house work automation, development automation, customer service, and data analysis. The scope is not a comparison of model performance, but rather how the organization manages agents with accounts, permissions, tool calls, logs, and outage procedures.

Scope of application: Situations where an individual is experimenting with one or two chatbots locally, simple document summaries that do not call external APIs at all, and unregulated sample projects are not within the direct scope of this article.

2. First, conclusion

Key one-liner: The first question when introducing an agent should not be “Which model is smart?” but “When can we stop this agent?”

• Teams that can be expanded now:Teams with separate accounts for each agent and already in place with tool call logs and privilege revocation procedures
• Teams that still need to be observed: Teams that use shared API keys and cannot reconstruct what data the agent accessed
• My judgment: The core risk of 2026 is not a problem caused by agents being stupid. The problem is getting agents to move fast enough, with enough authority, and quietly enough.

The recommendation order is conservative. First, we give every agent a unique identity. Second, break down tools and data permissions into business units. Third, it leaves runtime logs. Fourth, document procedures for immediately retrieving tokens and stopping work when an incident is suspected.

3. Decomposition of core structure

Key one-liners: Agent Security is not a single security product, but rather an operational structure with linked identities, model calls, tool calls, runtime actions, and audit logs.

1. Agent Identity: Agents, like human accounts, must be tracked as to who did what. Running multiple agents with one shared token makes it impossible to isolate the cause when an accident occurs.
2. Model call boundary: This is the layer that checks which models can be called, what the budget and rate limits are, and whether sensitive input goes to external models.
3. Tool call boundary: This is the layer that accesses actual business tools such as database, CRM, GitHub, payment system, and file system. Agent incidents usually occur in this tool call, not in the reply statement.
4. Runtime observation:The agent detects a strange order when executing several steps one after another. For example, if a large file read is followed by an external upload, you should question whether this is normal.
5. Audit and Stop Loop:This is the layer that leaves who approved, what exceptions were made, and which tokens need to be retrieved. This is the starting point for incident response.

To put it simply, for a novice developer, an AI agent is not a smart assistant, but an in-house account that automatically presses buttons. So, authority tags, access records, and emergency stop buttons are needed before speaking.

4. Description of design intent

Key line: The agent security structure should be designed with the intention of “let’s make the damage range small even if the agent makes a mistake” rather than “let’s trust the agent”.

Traditional security tools are largely built around a single human click or request. However, the agent calls multiple tools in succession, reads untrusted input such as web pages or documents, and uses the results of the previous step to determine the next step.

So simply protecting API keys is not enough. Even if the API key is secure, the agent may have excessive privileges, may follow instructions from a malicious document to read internal files, or may send incorrect data to an external SaaS.

There are three alternatives. First, it bans agents. Reduces risk but misses opportunities to automate tasks. Second, it allows agents to be free. Although the speed is fast, it is difficult to trace the cause of the accident. Third, agents operate within unique identities and limited tool privileges. The third option has initial design costs, but is scalable over the long term.

5. Evidence and Comparison

Key line: The risks seen in the latest survey are not abstract future risks, but control gaps already observed in production environments.

There are three numbers that are particularly important in the Gravitee report. First, the corporate agent set roughly doubled in four months. Second, an average of 48% of AI agents in production environments are reported to be running without security or governance. Third, 54% of responding organizations said they had experienced or suspected an AI agent security or privacy incident in the past 12 months.

NIST's AI page explains AI governance with a focus on risk-based approaches, measurement, evaluation, standards, and AI Risk Management Framework. This perspective also applies to agent security. Agents are not something that ends with “there is a policy document,” but rather something that needs to be managed with operational indicators on how to measure and reduce risks.

6. Actual operation flow / step-by-step execution method

Key line: The first implementation should start with creating an agent register, permission table, logs, and suspension procedures, not a feature list.

1. Create an agent register. Record name, purpose, owner, model used, access tool, access data, deployment location, termination criteria line by line.
2. Separate accounts per agent. If possible, use OIDC, service accounts, scoped tokens, and remove shared API keys.
3. Divide permissions into work units. Divide risk levels into read-only, comment writing, ticket creation, DB inquiry, DB modification, and external transfer, and leave the default value as read-only.
4. Model calls are collected by Gateway.Records model, token cost, latency, requester, project, and error rate. You should be able to see budget crashes and abnormal calls first.
5. Collects tool calls as well into a Gateway or central wrapper. Leaves which agent called which tool with which payload.
6. Specifies the stop condition like a code. Sets the condition to stop immediately, such as secret exposure, external transfer of personal information, mass inquiry within 5 minutes, and write operation without approval.
7. View metrics with 2-week pilot. Logs number of blocks, false positives, approval delays, increase in permission requests, and likelihood of reproducing an incident.

{
  "agent": "sales-summary-agent",
  "owner": "revenue-ops",
  "allowedTools": ["crm.read", "docs.write"],
  "blockedTools": ["crm.export", "billing.refund"],
  "stopRules": [
    "external_upload_after_bulk_read",
    "pii_detected_in_model_prompt",
    "write_action_without_human_approval"
  ],
"definitionOfDone": "All tool calls are reconstructed with agent_id, user_id, tool, decision, timestamp"
}

This example is intended to illustrate what an operating agreement looks like, not the actual product code. The key is to leave the “what you can” and “what you should never do” for your agent in a format that can be read by the system, not in your head.

7. Pitfalls

Key line: Agent security failures often start with overprivileges, missing logs, and missing owners rather than model illusions.

• Mistake 1: Pilot account is still used for operation
  Prevention: Set pilot end date and permission expiration date together
  Recovery: Reclaim pilot agent token and make last 30 days tool call. Thank you.
• Mistake 2: Leaving agent owner as team name only
  Prevention: Register personal responsible person, backup responsible person, and security contact
  Recovery: Agents whose approvers are not found during incident response are temporarily Stop.
• Mistake 3: Treating read and export permissions as the same level
  Prevention: Separate viewing, download, external transfer, deletion, payment and distribution into separate permissions
  Recovery: Use external transfer enabled tools Change to allowlist method and remove export permission without reason.
• Mistake 4: Seeing prompt injection as just a chatbot issue
  Prevention: Distrust any external input read by the agent, such as web pages, emails, PDFs, ticket bodies.
  Recovery: External Track which tools were called by the session that read the input, separating sensitive operations into new sessions and human acknowledgments.
• Mistake 5: Log but no stop button
  Prevention: Create a runbook that executes token reclaim by agent_id, block tool, stop queue, and discard session all at once.
  Recovery: Accident Afterwards, all active credentials of the agent_id are stopped before log analysis.

8. Strengths and Limitations

Key one-liners: A central control structure increases incident response, but if designed poorly, it can hinder development speed and a culture of experimentation.

• Strengths: With accounts and logs for each agent, you can narrow down the cause of the incident and quickly retrieve authority.
• Strengths: Gateway allows you to manage model costs, rate limits, provider failure response, and request tracking in one place.
• Strengths: Tool invocation policy directly controls the points where real damage occurs, i.e. DB modification, external transfer, and file deletion.
• Limitations: Centralizing all agent calls can make the initial development experience heavy. Low-risk internal experiments require a lightweight sandbox.
• Limitations: Even with Gateway, it cannot prevent all business logic errors, incorrect approvals, and inaccurate model judgments.
• Counterexample: It is expensive to attach large corporate-style controls to a one-off document summarization tool with no external connections. It is better to divide into tiers according to risk.

9. Points to study more deeply

Key one-liner: Next training should move away from model prompts and toward non-human accounting, policy-as-code, observability, and AI RMF.

• Non-human identity management: Learn service accounts, workload identity, token rotation, least privilege.
• AI Gateway: Check the options for model routing, cost limiting, request log, provider key hiding, and zero data retention.
• MCP/Tool Gateway: Study tool allowlist, tool-level RBAC, payload logging, pre-execution guardrail.
• Runtime detection: Learn how to view the sequence and combination of multiple tool calls rather than a single request.
• NIST AI RMF: Connect agent risk to documents and indicators with a Govern, Map, Measure, Manage perspective.

10. Reference

• Gravitee - State of AI Agent Security Report (Published 2026-06-15, Last updated 2026-04, Checked date 2026-06-30)
• NIST - Artificial intelligence (Confirmation date 2026-06-30)
• NIST AI Risk Management Framework Knowledge Base (Confirmation date 2026-06-30)
• TrueFoundry - Enterprise AI Agent Security Solutions: The Complete Buyer's Guide (2026) (Confirmation date 2026-06-30)
• Gravitee - A2A Summit Hub (Confirmation date 2026-06-30)

11. Action Checklist + Author's Perspective

Key line: The criteria for completion of introduction is not “the agent answers well” but “the agent can track, block, and reverse what it did”.

• Every production environment agent has a unique agent_id and owner
• Use agent-specific scoped credential instead of shared API key
• Read, write, delete, external transfer, payment, and distribution permissions are separated
• Model calls and tool calls are each reconstructed as logs
• There is a standard for immediate suspension of secret exposure, external transmission of personal information, mass inquiry, and unauthorized writing
• When an accident is suspected, token recovery and tool blocking based on agent_id can be performed within 10 minutes
• Pilot agent has an expiration date and operational transition review
• Agent privilege exceptions leave a reason, approver, and expiration date

Definition of Done: When one production environment agent is randomly selected, if the owner, authority, model call, tool call, recent operation, and stop method can be checked within 10 minutes, it is considered ready for the first operation.

My recommendation is “Register before expansion”. Before increasing the number of agents, start by listing the agents that currently exist. It is better to prevent agents not in the registry from accessing the operating environment, and to start with read-only registered agents by default. More important than rapid automation is the ability to stop incidents when they are small.