Two constraints, nine reversals
Nine of the RoboDojo tasks in SenseTime's study failed five times out of five. The obvious reading is that the model cannot do them. The company's result is that after lifting two constraints on the test harness, with no retraining and no weight changes, the same model succeeded five times out of five. Whatever those nine tasks were measuring, it was not a fixed property of the checkpoint.
The study covers 12 simulated benchmark suites and eight robot types: tabletop single and dual arms, wheeled-arm platforms, humanoids, parallel grippers and dexterous hands. The two constraints were, first, permission to read the training demonstrations, which enables in-context learning, and second, a per-episode step limit raised to as much as ten times each benchmark's official setting. Under those conditions GPT-6-Astra scored 81.9 percent on RoboDojo, and SenseTime reports that its results on LIBERO and SIMPLE are saturated.
What in-context learning actually buys
The paper's description of the in-context learning step is the most useful engineering detail in it. Before acting, the model examines demonstration images, robot states and action trajectories, and compiles what it finds into task notes, reference actions and small code tools. That preparation happens once per task and is reused across trials.
This is worth naming precisely: it is per-task context engineering, not learning in the sense of gradient updates. The gain comes from converting demonstrations into a retrievable scaffold the model can consult while acting. It also explains the transfer limitation SenseTime reports, since a skill pack assembled for one task carries the specifics of one embodiment and one scene, and its value degrades when the task changes. Six distinct capabilities are said to emerge during this process, though the categorization is the authors' own.
The cost column
SenseTime reports cost alongside scores, using a rough conversion of one million tokens to one dollar. Stacking bowls averaged about 131 minutes and roughly 19 dollars of inference. Putting a pen into a holder averaged about 263 minutes and 55 dollars. A person does the second task in a few seconds.
Those figures make an argument that score tables usually hide. Much of what looks like model capability here is purchased with test-time compute: thousands of environment steps, repeated attempts, and a slow model reasoning at every step. A benchmark that reports only a success rate cannot distinguish a policy that gets it right immediately from one that gets it right after two hundred tries. The second is a legitimate system, but it is a different product.
Fast and slow, and the price of checking
The natural next question is whether a slow System 2 model can delegate to a fast System 1 policy. SenseTime paired GPT-6-Astra with the pi-0.5 policy and tried several arrangements. Open-loop delegation, where the planner issues instructions and does not check what happened, reduces cost but damages performance badly. Closed-loop control, where results are inspected before continuing, recovers performance but gives up the cost advantage. The best combination found, GPT-6-Astra with in-context learning driving pi-0.5 in a closed loop, reached 54 percent success across 42 RoboDojo tasks, still at meaningful expense.
The pattern is consistent: verification is the expensive part, and skipping it is what breaks the system. SenseTime's own failure taxonomy points the same way. The model sometimes cannot confirm that a task is finished, handles physical state changes poorly, and loses track of where it is across subtasks. These are closed-loop problems, not perception problems.
Real robots, and a mock apartment
On eight real platforms the model could operate every robot tested and was strongest at tabletop grasping. The gaps were deformable materials such as folding clothes, fine control such as inserting a plug, and long-horizon navigation. One positive result is worth recording: with an obstructed view of a bottle mouth, the model moved its free arm so that the camera in its other hand could see the target, then completed the insertion. That is active perception used as a tool rather than a scripted recovery.
The real-world constraints differ from simulation in ways that matter. Robots stop after collisions or motor overheating, time-sensitive tasks cannot simply be given more steps, and demonstrations are often unavailable. SenseTime also built a mock apartment where water runs and heating continues while the model deliberates. There the model knocked other objects off a desk while moving a book, left a refrigerator door open for minutes, and turned off a stove too late, having decided to act on a view of the egg that was already stale by the time the action executed. That last case is a clean illustration of staleness: correct inference, wrong moment.
What the open-source release changes
The substantive contribution is the harness. SenseNova-RoboRSI packages each benchmark and its dependencies as a Docker image and exposes environment interaction and native scoring through a unified HTTP API, decoupling the environment from the agent and the policy. A researcher supplies a configuration file and runs an end-to-end evaluation, which makes replication of these findings possible by parties who do not trust the report.
The accompanying 27B model is a different claim. SenseNova-RoboRSI-27B-Preview scores 26.13 points and 20.83 percent success on RoboDojo, exceeds 90 on LIBERO-Pro under both direct System 2 control and fast-slow coupling, and improves RoboCasa365 success from 26.0 to 46.5 percent while cutting token cost by 73 percent. The headline that it costs one-fiftieth of GPT-6-Astra needs care: a 27B specialist that has internalized verified experience is being compared with a frontier general model on the same harness. That measures the value of distillation and scaffolding, not parity between model sizes. Its RoboDojo success rate also sits far below the 81.9 percent the larger model reached with a generous budget, so the comparison cuts both ways.
Three numbers hiding in one score
SenseTime proposes that future embodied benchmarks require three things: tasks derived from real human needs including their side effects and unstated responsibilities, experience acquired through autonomous interaction when demonstrations are absent, and accounting in world time, so that deliberation counts as elapsed time alongside action. The mock apartment is an argument for the third point rather than a benchmark.
The result worth keeping from this study is not 81.9 percent. It is that a single success rate conflates three separable quantities: what the policy can do, how much search budget the evaluator is willing to spend, and whether the system knows when it is done. Raise the step ceiling and the first quantity looks larger, remove demonstration access and it looks smaller, change the verification loop and the cost moves by an order of magnitude. Until harness settings, verification design and dollar cost are published next to the score, an embodied benchmark number measures a configuration. The harness release is what makes that comparison possible, and it is the part of this work most likely to still matter a year from now.
Sources
SenseTime, "SenseTime tests GPT-6-Astra and open-sources SenseNova-RoboRSI: where does embodied AI go next?" (official newsroom, 19 October 2026). Technical report and SenseNova-RoboRSI codebase, OpenSenseNova on GitHub.