A 4B agent matched its teacher on a shopping benchmark after learning from solved runs

<p>AWS published a walkthrough of sequence-level knowledge distillation for an agentic shopping assistant on Amazon SageMaker AI. The argument is cost-shaped: an agent is a loop of search, read, refine, verify and decide, so one completed task is many billed calls to a frontier model, and the spend recurs on every request.</p><p>The method uses only the teacher text outputs, with no access to teacher logits and no reinforcement-learning infrastructure, so it works with a closed, API-only teacher. In the example, GLM-5.3 on Amazon Bedrock plays the ShoppingBench agent and generates multi-step trajectories; the benchmark own scorer keeps only the trajectories that solve the task. Each trajectory is encoded as chat turns and expanded to one training row per step, with the conversation so far in the user turn and the next reasoning-and-tool-call as the assistant target, so loss falls only on the assistant turn. The solved trajectories expand to about 2,600 training and 130 validation rows after filtering, and the split is made by problem rather than by row because many rows share a problem and a row-level split would leak near-identical context.</p><p>The filtered data fine-tunes a Qwen3.5-4B student with serverless supervised fine-tuning on SageMaker AI model customisation, with MLflow tracking and a quality gate run through SageMaker AI model evaluation, and the result is served behind a real-time endpoint that runs the tool loop. The post concludes that the 4B student recovers essentially all of its frontier teacher task quality, and cautions that student size should be matched to the task and evaluated against the teacher and an independent frontier model, not only against the untrained base.</p>