An AWS Machine Learning Blog post published on October 23 walks through Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI. The method differs from supervised fine-tuning and single-turn RL in what it optimises: the training job sends a prompt to an agent hosted on Amazon Bedrock AgentCore (or on EKS, EC2 or Fargate) through a Lambda forwarder, the agent calls the policy model being trained, acts over as many turns as the task needs, and reports a single reward when the episode ends. No correct trajectory is specified, only how to score the result, which addresses the case where a decision made three steps before the outcome would otherwise get no credit. The worked example is Wordle, chosen because each guess depends on the previous feedback and the outcome is scored objectively; AWS notes frontier models already play the game well and that the real use is making a small, low-cost model good at a multi-step internal workflow. In the reported run, gpt-oss-20b went from 61% to 84% solve rate on held-out words at its default reasoning effort, and from 15% to 69% at the low effort it trained at, for about $339 in training tokens. The post also covers MLflow monitoring, evaluation at serving settings, reward-design mistakes and training failures, with code in an aws-samples repository; AWS says customer data and trained models stay in the customer's account and are not used to train base models.
Our takeThe 61% to 84% jump needs the caveat AWS supplies itself: Wordle is a teaching task that frontier models already solve, and the low-effort figure moving 15% to 69% shows part of the gain is matching the reasoning effort the model was trained at rather than new capability. The reusable part is the reward plumbing - one scalar per episode, no step labels - which is where most agent RL projects actually stall. The documented reward-design failures are more useful to practitioners than the headline score.