The Rise of Autonomous Agents and the Security of an Agent-Driven World
A supplier sends an updated invoice. One agent reads it, another prepares the payment, and a third updates the warehouse schedule. If the invoice contains a malicious instruction, how far can it travel before any system checks who actually authorized it?
I think agents could eventually initiate more digital interactions than people do in some services. That doesn’t mean agents will outnumber people. One person can delegate a task that produces dozens of requests while they’re offline: comparing suppliers, asking another agent to review a contract, or preparing an order.
The volume matters less to me than what those requests can do. A request might deploy code, move money, or send a robot across a warehouse. We still need to protect the person behind it, but we also need to know who gave the agent permission, what it can pass on to other agents, and which system can stop it.
The measurements below come from the cited studies. The scenarios and predictions are mine, not results from live experiments.
The rise of autonomous agents
A conversational assistant can suggest a supplier. An agent with tools could compare suppliers, check a budget, prepare an order, and request approval. Give that agent payment authority and the workflow can end with money moving rather than text appearing.
For this discussion, an agent is a system that uses a model to choose actions across multiple steps, using tools and feedback to pursue a goal. Autonomy is a matter of degree: preparing a transaction, asking for confirmation, and executing without another human interaction are different permission levels.
The distinction matters because useful autonomy is not unlimited autonomy. A system can be highly effective within a narrow scope without being allowed to change its own goals, recipients, or spending limits.
My expectation is that many successful deployments will look less like independent digital people and more like constrained specialists embedded in ordinary workflows.
What the measurements actually show
The evidence is stronger for growing delegation than for a fully autonomous economy. Three different kinds of research help separate those claims.
Financial adoption is not financial independence
The Bank of England and FCA’s October 2026 survey found that 80% of full-survey respondents reported deployed agents. The autonomy breakdown was 80% decision support, 46% constrained execution, 22% conditional autonomy, and 2% broad autonomy. These categories overlap: a firm can deploy several kinds of agent. BoE/FCA survey, sections 3.1–3.2
The full questionnaire had 105 respondents; percentages use valid responses to each question. This is participating-firm evidence, not a census of global finance.
My reading: the gap between deploying an agent and granting it independent authority is the important story. More agents do not automatically mean more autonomous payments.
Longer runs are real—but the tail is not the median
Anthropic’s February 2026 analysis of Claude Code found a median turn of around 45 seconds. At the 99.9th percentile, turn duration grew from under 25 minutes to over 45 minutes between October 2025 and January 2026. A turn ends when the agent finishes, asks a question, or is interrupted. These are runtime measurements, not task-success rates or human-equivalent working hours. Anthropic’s autonomy study
Figure 1. Published figure by Anthropic, reproduced unchanged. This tracks unusually long turns, not the median or task-success rates.
That supports a narrower conclusion than “agents can handle everything”: some users are allowing much longer uninterrupted work, while the typical interaction remains short.
More capability does not remove the organization
Microsoft’s May 2026 Work Trend Index reports that 58% of surveyed AI users said they could produce work they could not have a year earlier, while only 26% reported clear, consistent leadership alignment on AI. Its survey covered 20,000 AI-using knowledge workers in 10 markets. These are self-reports, not a controlled productivity experiment or a representative sample of all workers. Microsoft’s report and methodology
Together, these sources suggest opportunity with uneven readiness. They measure different populations and outcomes; combining them into a single “agent adoption score” would be misleading.
When agents use the web on our behalf
One person looking for a supplier might visit three sites. An agent doing the same job could query a catalog API, inspect documentation in a browser, and ask a specialist agent to compare the offers. A service would see many requests, all traceable to one delegated task. That’s one way agent traffic could grow faster than the number of people or agents making it.
Browser pages and API responses are useful inputs, but neither gets to redefine the job. A page can report a price. It can’t grant access to a private account or tell the agent to send the customer’s records somewhere else. The same rule applies to the output of another agent, even if its identity is verified.
MCP: connecting to tools and context
Model Context Protocol provides a shared interface for connecting AI applications to tools and resources. A2A’s documentation distinguishes this agent-to-tool role from agent-to-agent communication.
In a procurement workflow, an MCP tool might retrieve an invoice or an approved vendor record. Standardizing the connection does not make the returned content authoritative. The MCP authorization specification describes access to protected servers over HTTP when authorization is implemented. That is separate from allowing a specific transfer.
MCP’s security guidance discusses implementation risks including confused-deputy behavior, token passthrough, and session hijacking. A model-friendly interface still needs the same care around identity, consent, and isolation as any other interface.
A2A: communicating with other agents
Agent2Agent gives agentic applications a common way to discover peers, delegate work, and exchange results. It complements MCP rather than replacing it. The A2A specification leaves request authorization to the receiving server’s policies.
This is where I’d keep task delegation separate from access delegation. A contract-checking agent needs the contract, not the payment account. If it asks another agent to help, that next agent shouldn’t inherit permissions simply because it was brought into the same task.
I’d also keep the source of each claim attached when agents hand work off. “The supplier says this account is approved” is different from “finance approved this account.” A signed message identifies the agent that sent it; the payment service still needs evidence of the underlying approval. Otherwise, an untrusted claim can look more authoritative each time another agent summarizes it.
Finance is where delegation becomes concrete
Suppose I ask a procurement agent to renew our monitoring service for up to 500 USD. It retrieves the invoice, asks another agent to check the contract, and prepares a payment for 480 USD. The vendor name matches, but the bank details have changed. A note on the invoice says finance already approved the new account.
I authorized a renewal within a budget. The other agent checked the contract. Neither action authorized a new beneficiary. That’s the moment I’d want the payment system to stop and verify the account change independently.
The valuable opportunities extend beyond trading: reconciliation, expense review, procurement, invoice handling, and paid access to data or services. I expect these bounded tasks to be important because their rules and outcomes can often be checked independently of the model’s explanation.
Financial autonomy should therefore start with precise questions:
- Which account may the agent use, and for whose benefit?
- Which recipients, currencies, and amounts are permitted?
- Does the limit apply per transaction, per day, or across every delegated agent?
- What must happen before an irreversible action?
- How do we distinguish a failed payment from a successful payment whose response was lost?
A model saying “this looks correct” is not an authorization mechanism. The policy that permits a payment needs to exist outside the model’s reasoning.
Beyond one wallet: system-wide effects
There is also a coordination problem. If many financial agents respond to similar signals, their individually reasonable decisions could interact badly: delaying payments at the same time, concentrating liquidity demand, or depending on the same unavailable service. These are scenarios to investigate, not outcomes established by the adoption statistics above.
The BIS Innovation Hub’s Project Agenti, updated in October 2026, is exploring agentic simulations of wholesale payments and intraday liquidity with the Bank of Canada and Swiss National Bank. Its simulator is intended to study payment timing, liquidity, and gridlock. It is research in a simulated environment—not evidence that agents already run central-bank payment systems. BIS project description
What the payment protocols actually authorize
Getting an agent to the checkout is one problem. Proving it can spend the buyer’s money is another. The following protocols address different parts of that problem, not a mandatory sequence.
AP2: evidence of payment authorization
AP2 uses linked checkout and payment mandates. A non-agentic Trusted Surface obtains consent: the user either approves a specific checkout or sets constraints in advance that verifiers check against the transaction.
What if the agent reads the poisoned invoice before it prepares the mandate? By the time anyone signs, the proposal may already contain the wrong recipient. If an approval screen shows only “Renew monitoring — 480 USD,” a signature alone doesn’t tell me whether the approver saw the account change. Likewise, if an autonomous mandate constrains price but leaves out merchant identity, a wrong merchant can still fit the rules. That would be a gap in the workflow’s policy, not a broken signature.
x402: paying for a resource
With x402, a server can state the price and recipient and the client can retry with a payment payload. That makes it possible to buy API access in the middle of a task, but cheap requests can add up when retries, loops, or delegated tasks multiply them.
Checkout and identity at the merchant boundary
Agentic Commerce Protocol (ACP) connects an agent to merchant checkout. The merchant still validates the order and handles payment processing. That integration doesn’t set the buyer’s cross-merchant spending limit.
Visa’s Trusted Agent Protocol describes signed requests that help merchants recognize Visa-approved agents, with additional mechanisms for consumer recognition and payments. Visa’s documentation
Identity is useful evidence, not a blanket guarantee of good behavior. An authentic agent can still act on manipulated input or attempt an action outside the user’s intended scope.
My practical test for any integration is to name the exact guarantee: “this request has a valid signature,” “this credential may access this resource,” or “this transaction matches this approval.” None of those statements means “the entire workflow is safe.”
Physical AI: when a bad instruction moves something
Autonomy already exists outside the browser. Waymo’s rider-only driving studies document a deployed automated driving system in a defined operating environment. That’s not evidence that a general-purpose language agent can safely drive. It shows why we need to be precise about what kind of system is acting and where it was tested.
Language-driven robots raise a related but different problem. Google DeepMind’s robotics model card distinguishes high-level checks on an instruction from lower-level controls for motion, balance, and force. And in RoboPAIR, researchers tested whether adversarial prompts could elicit unsafe actions from particular LLM-controlled robotic systems. Those are experiments on specific setups, not evidence of attacks against production vehicle fleets.
Imagine a warehouse robot whose planner reads work orders. An attacker slips an instruction into one telling it to enter a restricted aisle. The planner may produce a perfectly plausible route. The meaningful security question is whether the motion system can reject that route independently, using the current map, safety zones, and sensor state. A signed work order doesn’t make an occupied aisle safe.
Photo: NIST’s mobile manipulator testbed, used to study how accurately a moving robot can locate itself relative to a workpiece. This is a robotics research setup, not the warehouse scenario or a RoboPAIR experiment.
I’d apply the same split to an industrial agent: let it summarize alarms or propose a change without giving it a direct path to override a controller or a safety interlock. NIST’s operational technology guide puts safety, reliability, and availability alongside security. In a physical system, stopping abruptly isn’t always safe either. The equipment needs a defined way to pause or reach a safe state, not just a button that stops the AI.
The dangerous step is turning information into permission
Take a hypothetical supplier document that says two things: the new bank account is already approved, and the shipment should go through a restricted loading bay. A procurement agent reads it, summarizes both changes as “verified,” and forwards them to the payment and warehouse agents.
If the downstream systems accept that summary as approval, the payment agent could send funds to the wrong account and the warehouse agent could dispatch a robot into an area it shouldn’t enter. The document didn’t need access to the payment service or the robot. It only needed the procurement agent to carry its claims across both boundaries. This is a threat scenario, not an observed exploit. The payment service should verify the beneficiary against trusted records, and the warehouse controller should reject an unauthorized route independently.
This is the trust boundary that interests me most: when does something the agent reads become something the system is allowed to do?
That’s what caught my attention in my Kiro investigation. I approved a webpage fetch, but the page’s instructions ended up in an executable workspace hook. Kiro was a single coding agent, not a multi-agent system. The supplier example asks what happens when that same confusion over permission reaches two other systems with different privileges.
A real identity can still make a bad request
I want to know which workload made a request, on whose behalf, and under what limits. Those are three different questions. A system like SPIFFE can provide verifiable workload identity. It doesn’t establish that the user delegated this task or that the action falls within that delegation. If an attacker compromises the workload, its requests can still arrive with a genuine identity.
In the supplier example, a genuine identity on the forwarded message would tell me which agent sent it. It still wouldn’t tell me who approved the bank change or whether the loading bay was safe. NIST’s design tenets for agent identity discuss limiting authority as it passes to downstream agents and keeping logs independent of the agent. OWASP’s agentic security guidance distinguishes the spread of a failure from the flaw that starts it.
I’d test that chain with one compromised or mistaken agent, not just one malicious prompt against a chatbot. Can it pass its own approval off as a user’s? Can its delegates spend from the same account at once? Does stopping the parent job also stop queued child actions? These are questions about the whole workflow, not just the model that first read the message.
Why a reassuring benchmark can be misleading
In a January 2025 agent-hijacking evaluation, NIST reported 11% attack success for its strongest baseline attack versus 81% for its strongest newly developed attack on held-out Workspace tasks. The tested agent used the October 2024 release of Claude 3.5 Sonnet. This is a historical, specific evaluation—not a current vulnerability rate for all agents. NIST’s evaluation, Insight 2
Figure 2. Original published chart by NIST, reproduced unchanged. It compares attack methods across four test environments, not two model generations.
The lesson I take from this is methodological: test adaptive attackers, repeated attempts, and the actual tools and permissions of a deployment. Passing a fixed collection of known attacks is not the same as preserving a trust boundary.
What I’d enforce outside the agent
My preferred design would separate proposal from execution. The model could extract invoice fields and recommend an action. A separate policy service would validate the recipient, amount, currency, approval state, and budget against authoritative records before permitting execution.
Figure 3. Agent-hijacking overview by NIST, reproduced unchanged. This illustrates the attack path, not an implementation of the policy gate proposed above.
For higher-impact actions, I’d use short-lived credentials scoped to the task and confirm the exact recipient, currency, and amount. A change after approval should invalidate that approval. If a payment times out, I’d reconcile it before retrying and use the same idempotency key for the same payment. If several agents draw from one budget, the limit has to be enforced against their combined spending.
I’d want an audit trail that joins the original request, the transaction shown for approval, and the final outcome. Where a payment cannot be reversed, recovery means limiting damage and escalating—not pretending every action has an undo button.
The same approach works beyond payments. Give each worker the narrowest permission it needs. Ask a human to review the exact action when it falls outside what they already delegated. Make sure stopping a workflow also prevents pending tasks from acting later. In a physical system, keep the safeguards that constrain motion or bring equipment to a safe state independent of the planner.
These controls don’t eliminate prompt injection. They limit what can happen after an agent gets something wrong.
Will agents become more present than humans?
My prediction is about request volume, not headcount. Agents could eventually initiate more interactions than humans in some digital services without outnumbering them as distinct entities. None of the studies above measures that crossover, and none shows it will happen in every service.
One person could delegate work to several agents. Each agent could make many requests, coordinate with other services, and continue monitoring a task while its owner is offline. Under that model, machine-initiated activity could grow faster than the number of human users.
But request volume is a weak measure of value. A thousand unnecessary tool calls are not better than one correct decision. Adoption will also depend on reliability, cost, liability, privacy, and whether people actually want to delegate a particular task.
I expect the more interesting change to be architectural: services increasingly designed for both human interfaces and constrained machine clients. Clear APIs, explicit permission models, verifiable approvals, and useful audit trails could matter as much as the quality of the conversational interface.
Humans would still need ways to set goals, contest outcomes, revoke authority, and assign responsibility. Being less visible in the execution loop should not mean losing control over it.
The future I want to help secure
I am interested in a future where agents can do meaningful work without being trusted with everything. More capable models and better interoperability are only part of that future. The harder question is how much authority a system should receive, how that authority is enforced, and what happens when it is misused.
For me, the opportunity in AI security is to examine those boundaries before they become invisible infrastructure: the tool response that changes a decision, the delegated task that expands access, or the valid payment request that no longer reflects what a person intended. I want to know which component can still say no, whether the last step is an API call, a transfer, or a machine moving through a shared space.
I don’t think the hard part will be making every agent behave correctly on its own. It will be keeping one agent’s mistake from becoming another agent’s permission. That’s what I want to test: whether authority stays narrow, traceable, and revocable as work passes from a webpage to an agent, a payment service, or a machine.
Sources and reading notes
The statistics come from the original publications, not market-size forecasts. The three original research figures are credited in place, as is the NIST robotics testbed photo.
- BoE/FCA: AI in UK financial services, October 2026 — survey definitions, respondents, and overlapping autonomy categories.
- Anthropic: Measuring AI agent autonomy, February 2026 — observed runtime, not a universal measure of competence.
- Microsoft: Work Trend Index, May 2026 — self-reported workplace outcomes and methodology.
- BIS: Project Agenti, October 2026 — ongoing simulation research, not production deployment.
- NIST: Agent-hijacking evaluations, January 2025 — historical security experiment, included for its evaluation lesson.
- A2A documentation, MCP security guidance, AP2 specification, x402 documentation, and Visa Trusted Agent Protocol — implementation references, not guarantees of adoption or safety.