A support agent picks up a ticket: a player paid 648 CNY for an in-game top-up and the package never arrived. The order shows the payment succeeded. The payment callback carries a valid signature. The delivery log holds one line, deliver_status = failed, and the system log adds BAG_FULL — the inventory was full, delivery retried three times, and every retry failed. Alibaba Cloud's cloud-native community published a walkthrough of this demo on 29 October, and its real subject is not the agent that produces the answer but the dictionary that makes the answer checkable.
The system has three parts. A Source Model maps one-to-one to a project: an LLM reads dashboards, alerts, Scheduled SQL and data samples, then extracts field semantics, metric definitions and glossary terms. Each entry lands in one of three states — auto-adopted where evidence is sufficient, not adopted where evidence is thin, manually adopted where a person has checked it — and the DataAgent uses only adopted entries, so a field whose meaning is still in doubt is never treated as a business fact. A Context Model sits at account level: it selects which LogStores a business actually needs, syncs adopted entries with a stated maximum delay of 60 seconds, appends without ever removing, and keeps manual additions untouched by sync. A DataAgent is then an instance bound to at least one Context Model; it maps a question to an explicit metric definition and data source, builds queries, chains evidence across stores, and returns a conclusion, the supporting evidence and a draft reply.
The design claim is narrower than it first looks. NL2SQL and MCP servers already let a model query logs. The article's argument is that query capability and a trustworthy answer are separate problems. 'Revenue' can mean order face value, the amount actually paid, or the delivered amount; a model asked cold has to guess, and may guess differently on the next ticket. Player vocabulary does not exist in the logs — 'Skysplitter Blade' is WPN_LEG_001 — so full-text search on the player's words returns nothing and the agent can conclude the item was never lost. And a one-shot query tends to stop at the first plausible cause: it finds deliver_status = failed, blames delivery, and never asks why delivery failed.
The three published cases show what an evidence chain means in practice. For a missing item, the agent joins the inventory transaction log to trade records through item_uuid, finds change_reason = trade_sell at 64,663 gold, checks logon devices, and reports that the item was sold on the auction house from the current account. For a rating drop after a win, it shows battle_result = win, a rating move from 1259 to 1237, rank_change_reason = disconnect_penalty, and 130–200 seconds offline against a 120-second threshold. For a cheating-ban appeal, it lines up a detection value of 8.85 against an 8.5 threshold — 4.1% over — an ml_score of 0.62 below the 0.7 high-confidence bar, a clean memory scan, and ping of 180–250 ms.
The most defensible part of the write-up is the stopping rule. The missing-item conclusion does not say the item was not stolen; it says the available logs show no evidence of unusual logon and recommends the player check account security. The ban appeal does not overturn the ban; it says the evidence better supports a high-latency deviation and recommends manual review. Refunds, redelivery, bans and unbans stay with a person. That is a policy expressed in the shape of the output rather than in a prompt instruction — a stronger place to put it, because output shape survives model changes and prompt edits.
The adoption gate is the second good idea. A definition extracted with thin evidence is marked and withheld, so uncertainty is handled once, at definition time, instead of being rediscovered per answer. It is the semantic equivalent of a type check, and it moves review to the cheapest point in the pipeline. The third is that the permission boundary is a RAM role — aliyunslsdataagentrole, or a custom role passed with ram:PassRole — which restricts reachable resources. A prompt is not an authorization system; a role is.
What the article does not publish is the result of its blind test. It says the test runs on reproducible synthetic logs, covers 13 categories of complaint, and correctly isolates the ground truth in a separate LogStore that the agent is not granted. That isolation detail is right, and plenty of vendor write-ups get it wrong. But there are no numbers: no pass rate, no evidence-recall figure, no clarification rate, no count of conclusions that were confidently wrong. 'Blind test' is described as a procedure rather than reported as a measurement, so a reader cannot tell whether the dictionary fixed the ambiguity or whether 13 scenarios were written around the dictionary.
The second gap is staleness. Synchronisation is append-only and entries are never removed. If a producer renames a field or adds an enum value, the old definition stays in the model and the new value arrives unadopted — the safe direction, but only if someone notices. The article's own advice, that wrong definitions get worse as coverage grows, is an argument for versioning and drift detection. Neither is described. On a schema that changes weekly, the business dictionary becomes an artifact nobody owns.
The third gap is that synthetic logs are the easy case. The schema is fully enumerable; no field is missing, no log arrives late, no retry produces duplicate trace_ids, no two tenants collide on an identifier. The article names trace_id and business ids such as order_id, battle_id and item_uuid as the strings that tie evidence together, but says nothing about a join key that is absent, reused or ambiguous — which is the failure mode that makes production log analysis hard in the first place.
Cost and latency are also unquantified. Source Model generation 'usually finishes within a few minutes' depending on how many LogStores a project holds and how much data they contain; per-answer cost depends on how many hops the agent takes. Without either number, the economics of replacing the three to six manual queries a ticket currently needs cannot be evaluated. Deployment limits are stated plainly: Context Model and DataAgent are available only in cn-beijing, Source Models in five regions, and 50 of each per account and per user by default.
If you build something like this, the measurements that matter are unglamorous. Definition coverage: what fraction of fields and enum values on the queried path have been adopted. Clarification rate: how often the agent stops to ask which definition of a metric is meant. Evidence recall: whether the chain includes the causal log line, not just the symptom. Stale-definition rate: how many adopted entries no longer match the live schema. Escalation precision: of the cases handed to a human, how many actually needed one.
The contribution worth taking from this demo is not the agent. It is the ordering — define the metric, then query; gate the definition by evidence; keep irreversible actions outside the tool surface. Those are checkable engineering choices, and they hold whether the model behind the agent is Qwen or anything else. The missing piece is the measurement that would show the dictionary is right, and that is the work still outstanding after the demo.
Sources
Alibaba Cloud Community, "The Player Says 'My Top-Up Never Arrived.' The Log Only Says BAG_FULL.", 29 October 2026 — alibabacloud.com