A classroom AI coding loop (a model writes a program, the robot runs it while an overhead camera films, the model sees the logs and frames and tries again), recorded in simulation. Every trial below ran the model's exact program through the robot's own Wi-Fi client library against a simulated AIM robot, and was scored with MotionBench.
Check against the real robot
Does the sim reproduce what the robot did?
Four programs from a recorded real Game 1 session, each replayed in simulation over many seeds. The real outcome is read from the recorded overhead frames (left three), the sim result is the share of seeds that end the same way.
The loop
Did feedback improve the code?
Goal score (MotionBench: share of the game's goal achieved) for each attempt, mean of 3 trials; dots are single trials. Attempt N is written after seeing attempt N−1's code, logs, telemetry and frames. In Game 3 the goal counts tags knocked down by the ball; a barrel hit is a rule flag that zeroes the game's own points, so Claude Opus attempt 2 reaches 100% goal with 0 game points (see the board). The loop never tells the model its score, so it can't learn that from the feedback.
MotionBench boards
Scores by task
One agent per program (model × attempt). Goal is the mean share of the goal over its trials, with a 95% bootstrap interval; MBI combines goal, path efficiency, consistency and safety (rule flags such as ramming a sign). Game points apply the game's own rules, including Game 3's "a barrel hit fails the run". Click a row to replay its first trial.
Run explorer
Replay any trial
0.0 s
Program
Speed
Where the time goes
How to read this
What was simulated, and what wasn't
Programs ran unchanged through the VEX AIM WebSocket client the loop uses. The simulated robot answers the client's commands and sends it status packets; time, sleep and threads run on simulated time.
The model calls were made by fresh Claude agents (Opus, Sonnet, Haiku), each shown only that attempt's prompt folder: the same system prompt, API reference, game description, first image and feedback the loop sends a model's API. With API keys, the loop calls the models' APIs directly. Agents decide how many of the up to 50 feedback images to look at; an API call always receives all of them.
Field layouts were measured from each game's start photo. The robot's size, camera geometry, held-ball detection, command latency and start-up time were calibrated against the recorded real Game 1 runs. Which corner sign is tag 0 and which is tag 4 in Game 4 is assumed.
"Knocked down by the ball" counts; a sign knocked over by the robot or by pushing the held ball into it is flagged as ramming. In Game 3 a ball hitting a barrel before bouncing off a wall, or running into a barrel while carrying the ball, fails the run.
Simulated runs are evidence about the code, not about a robot. Use them to choose which version to run on the real field.