On 30 October, Alibaba Cloud's community blog published Tarmac: an airline irregular-operations system in which five Qwen agents trade revocable claims over a seat ledger, with a sixth, larger model ruling on the deadlocks. The headline is a comparison against a single greedy planner — 17 protected passengers stranded versus 3, zero crew-duty violations, every special-needs service level met. The question worth asking is not whether the gap is large, but which layer of the system produced it.
The Gap Is a Distribution, Not a Total
The scenario is fixed and small: 180 displaced passengers, 139 legal seats. Capacity is fixed, so 41 people fail to fly in both arms of the comparison, and the published numbers are about which 41. The single planner stranded 17 passengers the scenario marks as protected; the society stranded 3.
The repository's ablation table supplies a figure the post leaves out: overnight strandings of 52 [50–53] for the planner against 50 [49–52] for the society, medians with inter-quartile ranges across ten seeds. Two passengers, overlapping intervals. The society did not move more people; it moved different people, displacing roughly fourteen unprotected passengers to seat fourteen protected ones. The result is a distributional claim, though "stranded just 3" invites a capacity reading.
Protected is defined by example only: a medical courier, an unaccompanied minor, a wheelchair user, and tight connections. No weighting among those categories is stated and no per-category result is broken out, so the gap could come from one category or all four.
The Benchmark Does Not Contain the Model
The post attributes the result to Qwen: five agents on qwen3.7-plus, a duty-manager mediator on qwen3.7-max, roughly sixty turns per run, cheap enough to benchmark over many seeds. The repository's benchmark document says something narrower. Its header reads "10 seeds · offline deterministic policy agents · fixed temperature (live mode 0.2)", and its figures are medians across seeds.
The committed live run — 324 log entries, Qwen roles and a Qwen mediator — exists to show that the wire path works and replays exactly. It is not one arm of the comparison, and neither the post nor the repository reports a live-mode ablation.
What the measurement supports is a claim about protocol: a set of claim-placing roles with an adjudicator allocates this fixture differently from a single planner. That Qwen's judgement is responsible is untested, because no language model sat in the measured loop. The experiment that would settle it is the same ten seeds in live mode against policy agents on identical rows, which separates the mechanism from the model driving it.
Zero Percent Is a Signature of the Objective
An arm that meets none of the special-needs service levels is an arm in which the constraint is absent, not one where it is weighted low. A planner that knew about the wheelchair and under-served it would land somewhere in between. The post gives no objective function for the baseline; the benchmark documentation describes a "single greedy planner" and adds that it "presses the duty-illegal ferry into service" — an option it did not know was excluded rather than a trade-off it lost.
If the planner's constraint set omits protection and duty legality, the comparison establishes that constraints have to be encoded somewhere, which nobody disputes, rather than that negotiation discovers them. The version of this test worth running keeps the constraint set identical in both arms and asks whether a negotiated allocation beats a solved one on secondary metrics: connections saved, rounds to quiescence, run cost.
The post also omits the planner's crew-duty figure. The repository gives it one violation against zero in both society conditions. That row is the most robust difference in the table, because crew duty is a legality invariant rather than an objective: binary, mechanically checked, and immune to weighting.
The Mediator Carries the Throughput
The third arm is the one the post does not mention. Remove the mediator and the society strands 14 protected passengers, meets none of the service levels, and leaves 96 people overnight [81–109], against 52 for the planner. Contested claims are quarantined at the round cap with nobody to rule on them, so seats go unfilled.
That reframes the architecture. The post presents adjudication as an edge case — rare deadlock rulings for the exceptional tie. The measurement says the adjudicator is where throughput comes from: strip it out and the society is worse than the system it was built to beat on total strandings, while still marginally better on protected ones. What the five agents contribute is a contested queue; what decides who flies is a single privileged rule-applier reading position papers. When the system misallocates, the place to look is the rule set the mediator applies, not the emergence of the society.
Sealed Bids Buy Non-Adaptation
Commit-reveal here is concrete: a bid is a digest over the claim and a nonce, and the ledger rejects any reveal that fails to re-derive it. What that removes is a specific failure mode — an agent revising its bid after watching a rival's. What it does not remove is strategic bidding. With a credibility currency, contesting costs points, winning refunds them plus a premium and losing burns them, so each agent is bidding in a first-price contest under a budget, and the budget-optimal bid is not the sincere one. The post reports no bid distribution, no contest frequency and no count of stakes that hit the round cap, so it is not possible to tell whether agents were expressing preferences or managing a balance.
The ledger itself holds up. Claims are typed mutations against SQLite with row-level locking, so two threads racing for the last seat resolve to exactly one winner — a property with a test rather than a prompt. An agent society needs a transaction boundary; a transcript does not have one.
Replay Is the Stronger Claim
The hash-chained log, the invariant suite and the byte-for-byte replay are evidence of a kind the score is not: falsifiable in one command, by someone who does not trust the authors. The suite is specific — capacity and exclusivity, crew duty, cited rulings, sealed-bid reveals, replay identity — and it runs server-side against the committed live run.
Two limits. The fixture is built so that a deadlock always occurs, which makes it a test case engineered to exercise the mediator rather than a sample of real disruptions; a second scenario with different contention would show whether the gap generalises. And ten seeds of a stochastic process is a small sample: ranges are reported for overnight strandings only, so the headline gap arrives without an error bar. Nor is it clear what the seed varies — with a frozen scenario and deterministic agents, ten seeds would be ten identical runs, so something changes between them and the documentation does not say what.
The signed ruling has a narrower signature than it looks. The invariant checks that a ruling cites a regulation, not that the citation settles the question. Rules supplied by the system and unambiguous by construction leave the hard case untested: two rules that conflict, or a situation no rule covers.
What survives scrutiny is the substrate — a row-locked ledger, a mechanical deadlock detector, signed rulings that replay exactly — plus the finding that removing the adjudicator costs most of the throughput. What does not yet survive is the headline. Five Qwen agents negotiating better than a planner is a claim about model judgement, and the measurement that produced the gap did not have a model in it. The post gives no token counts, no wall clock and no cost per run; the only cost figure in the materials is relative, roughly ten times token spend without a shared context prefix. A society of five agents negotiating in rounds has a bill, and the argument for paying it would be easier to assess with the bill attached.
Sources
- Alibaba Cloud Community, "Tarmac: Five AI Agents Negotiate the Storm, and One Mediator Signs the Ruling" (30 October 2026): https://www.alibabacloud.com/blog/tarmac-five-ai-agents-negotiate-the-storm-and-one-mediator-signs-the-ruling_603706
- Project repository, README: https://github.com/edycutjong/tarmac
- Project repository, ablation benchmark (docs/BENCH.md): https://github.com/edycutjong/tarmac/blob/main/docs/BENCH.md