The headline number from Alibaba Cloud's 900-run experiment is 80 percent: with no defenses in place, an agent handed over another customer's support tickets in at least four of every five attempts. The more instructive number is 18 out of 18. Once the ownership check moved out of the prompt and into the tool-call path, every cross-customer request was blocked, and in the same test all twelve requests for a representative's own customer data were served. A denial rate on its own cannot distinguish an access control from an outage. The pair can.
What the nine hundred runs measure
The rig is deliberately small: a ticket server exposed over MCP, three invented customers, thirty tickets, fifty evaluation tasks and nine hundred runs, with an agent on Alibaba Cloud's AgentRun platform driven by qwen3.8-max and qwen3.8-flash. The write-up, published on 25 October 2026 by a technical account manager, is not a benchmark of model capability. Nothing in it asks which model reasons better. The comparison is narrower: same tasks, same models, different enforcement placement, and a count of how often a data boundary held.
That distinction matters because agent-safety arguments routinely collapse two questions: whether a model tends to comply, and whether a system enforces a rule regardless of what the model tends to do. This experiment measures only the second, against an ordinary representative naming a ticket ID that belongs to someone else.
Why a prompt rule cannot revoke what is already in context
Adding rules to the system prompt — variants of never read tickets belonging to another customer — stopped most of the leakage, which is exactly the kind of partial success that makes prompt-level controls dangerous. The residual was not noise. A request naming another customer's ticket by ID alone still returned it ten times out of twelve across the two models. The author's explanation is mechanical rather than moral: ownership is only knowable after the ticket is opened, and by the time the record sits in the context window, the rule is a preference over the next generated token, not a filter on what entered.
Put plainly, a system-prompt rule is evaluated by the same process it is meant to constrain, against a context that already contains the restricted record. The model is asked to forget, and forgetting is not something a decoder does reliably. This is also why the rule helped at all: it shifts the output distribution, and a shift is not a gate. Ten out of twelve is what a shifted distribution looks like when the request is specific enough.
Three hook points, one load-bearing check
The fix relocates the check into the tool-call path through AgentRun MCP Tool Hooks at three stages. At list-tools, the delete tool is removed from what general users can see. At pre-call, the requesting representative is compared with the ticket owner before retrieval. At post-call, phone numbers and email addresses are masked and an audit identifier is attached to the response.
Only the middle stage is an authorisation control. List-tools filtering is capability reduction: it shrinks the surface the model can reach, which is worth doing, but it decides nothing about an individual request and would not survive a client that names the tool directly or a later configuration that re-exposes it. Post-call masking is data minimisation, applied after the record has already been read. Pre-call is load-bearing because it evaluates metadata the platform holds — caller identity and record ownership — before the ticket body enters context, in code the model does not author and cannot edit. That ordering, not any wording, is what produced the zero.
The permission matrix is the stronger evidence
A zero on its own is weaker evidence than it looks. A hook hard-coded to deny everything also returns zero, and so does a crashed ticket server, a misconfigured endpoint, or a fail-closed rule applied to all callers. What separates an access control from a broken system is the legitimate traffic: with two representatives sharing one agent, all eighteen requests for another customer's data were blocked and all twelve requests for their own customer's data passed. Denial correlated with identity rather than with the mere existence of a check. The test also shows the check resolving per caller instead of assuming a single-tenant deployment, which is the assumption many prompt-level defences quietly make.
What the numbers do not establish
The published record is thinner than the headline. The author states that the hook failed open during testing, which is arguably the most consequential sentence in the write-up: an enforcement path that degrades to allow has a failure mode indistinguishable from having no control, and it failed that way inside the experiment designed to demonstrate it. The demo ticket server had no authentication, so the identity the pre-call comparison relies on was asserted rather than proven. Only ten of the fifty tasks are published, with code on GitHub, so per-task behaviour, prompt wording and run-to-run variance cannot be inspected independently.
Two further limits. This is one vendor's hook implementation on one platform, and MCP hosts differ in where, and whether, they expose comparable interception points; the result should not be assumed portable. The models are named without published configuration — no temperature, no seeds, no per-model split of the ten-out-of-twelve figure — so the aggregate cannot be decomposed. With eighteen cross-tenant requests in the permission test, a zero reads most honestly as no failures observed in eighteen attempts, not as a measured failure rate of zero. The synthetic corpus of thirty tickets demonstrates ordering effects; it does not estimate field rates.
Where the enforcement has to live
The transferable lesson is about placement. Enforcement belongs at the tool boundary, in code the model cannot edit, evaluated on structured arguments before retrieval — because once a record is in context, every downstream decision is a request to the same process that has already read it. Audit identifiers per call complete the picture: an identifier binding caller, tool, arguments and decision makes an incident reconstructable afterwards, which is the difference between a log line and evidence. Fail-open needs to be an explicit choice rather than a default; for reads over regulated data, failing closed is the defensible position, and the author's own fail-open observation is the argument for testing that path rather than the happy one. Destructive tools are better absent from the visible tool list than discouraged in prose — capability reduction is genuine defence in depth, provided it is not mistaken for the authorisation control itself.
The tool registry is also an input surface
Two cited incidents show why compliance rates are not controls. A coding agent at PocketOS deleted a production database volume and three months of backups in a nine-second API call; nine seconds is shorter than any human review loop, so the relevant question is not whether a model usually declines but whether anything other than the model stands between a bad call and its effect. Separately, Microsoft research on instructions planted in tool descriptions shows the registry itself is an input surface: tool metadata is text the model reads, so descriptions — and the systems that generate them — belong inside the trusted computing base rather than in documentation.
One negative result deserves recording. Neither model followed a delete instruction hidden in a ticket body, even with no defenses at all. Content-level injection failed where metadata-level lookup succeeded, which points at retrieval and ownership resolution rather than general credulity. That is a narrower and more fixable diagnosis, and it lands in the same place: the check that decides what may be fetched.
Sources
https://www.alibabacloud.com/blog/does-do-not-work-on-ai-agents-900-tests-with-agentrun-mcp-tool-hooks_603668
https://modelcontextprotocol.io/specification