The standard way to teach a robot a new manipulation task is to collect dozens of demonstrations and train a policy, baking the skill into the network's weights. A new preprint argues that a frozen language model can do better by doing what coding agents already do: try things, watch what happens and write down what it learned, with a simulator playing the role a terminal plays for software.
The authors call the idea world in-context learning and build a platform, RoboScript, to run it. From a single demonstration, the agent rehearses each task in simulation, distils what works into a plain-text Task Memory, then freezes that memory and is tested on scenes it never rehearsed. The key claim is that what the agent stores is semantic, such as which object plays which role and which steps went wrong, rather than how the scene looks, so it carries over to new layouts. Acting agents see only camera images, depth and joint readings, the same inputs a real robot has, and a linter rejects memories that leak simulator coordinates or other privileged data.
On all 50 tasks of the RoboTwin 2.0 benchmark, RoboScript reached 83.4% mean success across clean and randomised settings with one demonstration per task and no gradient updates. The strongest trained action policy, given 50 demonstrations, reached 61.7%. The contrast under visual randomisation is starker: FastWAM, a widely used world action model, scores 77.8% on clean scenes but 1.9% once lighting and textures change. Results scale with the model plugged in: GPT-6.1-Sol scored 83.6% and 83.2% on clean and randomised scenes, GPT-5.6-Luna 79.8% and 76.0%, and even the lightweight DeepSeek-V4.1-Flash beat the best trained policy. The authors also report transferring memories to a physical two-armed robot through a small adapter.
The interesting shift is where the work happens. Instead of more data and more training, the cost moves to inference: many model calls during rehearsal and reflection. Teams that already run coding agents such as those built on GPT or Claude will recognise the loop, and the same failure modes, from wasted calls to confident mistakes, are likely to follow.
The caveats are significant. Results come almost entirely from one simulated benchmark, and the physical-robot section is a demonstration rather than a broad test. Rehearsal assumes a simulator faithful enough to the task, which is easy for RoboTwin scenes and hard for a real kitchen. The paper does not give the cost or time of rehearsal per task in the sections we read, and it is a preprint without peer review.