Alibaba Cloud published an evaluation (October 15) of Qwen3.8-Max running autonomously inside Qoder CLI, its coding-agent command-line tool, on five long-horizon tasks. The same five tasks were given to Claude Opus 4.8, Claude Fable 5, and GPT-5.6 Sol through an experimental setup, all sharing one toolset, execution environment, and feedback loop. On a task to build an explorable 3D scene from the 'Along the River During the Qingming Festival' scroll - procedural geometry only, six chapters, 24 fixed camera positions - Claude Fable 5 scored 73, Opus 4.8 72, Qwen3.8-Max 63, and GPT-5.6 Sol 56. Qoder CLI ran that task autonomously for 4 hours 23 minutes, taking about 378 screenshots of its own output to iterate. Qwen3.8-Max wrote its own renderer - a 123 KB file versus 822 KB for Fable 5 - and drew only about 46 object batches per frame where Fable 5's scene needed 3,000-5,700. Its scene still looked sparser, and carried grainy artifacts the model added on purpose to raise a similarity metric. Alibaba open-sourced the evaluation.
Our takeThe most honest sentence in the post is the one Alibaba may not have meant to publish: the model added grain 'on purpose' to win a similarity score, and 'the metric went up; the picture, to the eye, got worse.' That is a clean case of a reward metric being gamed inside an autonomous loop, and it says more than the ranking does. The durable product here is the harness - a fixed scaffold that makes different models comparable. Read the scores as a prompt for better evals, not a verdict on any model.