Student-led · open source

Who really gets the job done?

A student driving. The team’s own code. A program an AI wrote. A driver with the robot helping. MotionBench gives every task a clear goal and measures every attempt against it the same way, so the answer is a number anyone can check.

Free and open source. A robot and a phone are enough to take part.

12345
RunWaypointsTimePath
Student 08 (One Stick controller)0 / 50.0 s…
LLM-written Python, round 3 (uses AI Vision)0 / 50.0 s…
Simulated VEX AIM runs: a student driving and AI-written Python, both scored against the same goal of five waypoints on a 6 × 4 ft table. Playback 1.5× real time.

Every task has its own board

Who’s fastest to the goal?

Like a lap-time board on a car show: one board per task, and the quickest to get the whole job done goes on top. Only agents that achieved the whole goal on every run get a time. The rest sit below the line, by how much of the goal they managed.

The coloured tab says who was in control, so a student driver, the team’s code and AI-written code sit on the same board for the same job.

  • Human
  • Team-written code
  • AI-written
  • Human + assist
  • Commercial robot

These are simulated runs, made to show how a result reads and to test the scorer in public. Real runs replace them as teams send them in. How scoring works

See every board →

MotionBench

The Board

Simulated

Precision Waypoint navigation on a 12 ft field. Drive through six waypoints in order, within 5 cm and 5°, in 20 seconds.

Fastest to the whole goal, on every run

  1. 1LLM-written auton, round 10AI-written10.98swhole goal, 10 of 10 runs
  2. 2LLM-written auton, round 3AI-written11.71swhole goal, 10 of 10 runs
  3. 3Team auton A (odometry + GPS correction)Team-written code12.75swhole goal, 10 of 10 runs
  4. 4Driver 05 with auto-alignHuman + assist13.19swhole goal, 10 of 10 runs
  5. 5Driver 12Human14.79swhole goal, 10 of 10 runs
  6. 6Driver 06Human15.13swhole goal, 10 of 10 runs
  7. 7Driver 10Human15.70swhole goal, 10 of 10 runs

Didn’t finish every run

  1. 8Driver 11Human97%of the goal · finished 8 of 10 runs
  2. 9Driver 02Human95%of the goal · finished 7 of 10 runs
  3. 10Driver 09Human95%of the goal · finished 7 of 10 runs

7 of 19 agents finished every run. A robot wandering at random gets 0% of the goal here. Full board, every agent →

It started with a fifteen-second argument

I’m Abhay, an eighth grader in the San Francisco Bay Area who competes in VEX robotics. Every match opens with fifteen seconds where the robot drives itself, running code my team wrote. Then we grab the controllers and drive.

After almost every match we argued about the same thing: was our code actually better than us? Then AI tools started writing robot code too, and the argument got bigger. I went looking for a way to settle it with a number someone else could check. There wasn’t one, so I started building it.

Read the whole story

Why it’s harder than it sounds

A match score mixes in the other team, the field and luck. A product page quotes its own best case. And a raw number like “six targets in eleven seconds” means little without a clear goal to measure it against.

So a fair test needs things most robot tests skip: one task with the same rules for every attempt, a goal that says exactly what “done” means, a robot moving at random to show what luck alone achieves, several tries each, and the worst run reported alongside the best. Comparing a robot with a person only makes sense when both use the same means, like drivers and code on the same robot.

Questions we want to answer

These are open questions, not results. We’ll publish the answers whoever wins.

  • VEX V5

    Is our code better than our driving?

    Student drivers and the same team’s autonomous routine, on the same 12 ft field with the same targets, all scored against the same goal. The argument that started all this. Then AI-written routines on the same robot.

  • VEX AIM

    Does AI write better robot code than a class?

    Python an AI wrote, code the class wrote, and students driving with the One Stick controller: waypoints and barrel sorting on the same 6 × 4 ft table, scored by the same rules.

  • And then

    • How much of a room does a robot vacuum really cover, and what does it miss?
    • Which strips of lawn does a robot mower leave behind?
    • Does an AI-written routine still reach the goal on real hardware, or only in simulation?
    • How much does an auto-align assist help a rookie driver?
    • How many picks an hour can a warehouse robot keep up without a near-miss?
    • How accurately does a classroom arm sort, run after run?
    • Do better drivers also write better code?
    • How many tries does it take before a robot’s average can be trusted?

What can be scored

If a job has a clear goal in a space with clear walls, any attempt at it can be scored: a person, a robot running its code, an AI-written program or a driver with robot assistance. Each job gets its own test, so a vacuum is never ranked against a VEX robot. A person is compared directly only when they use the same means, like students driving the robot their code drives.

WhereWho gets scoredAgainst the goal of
VEX V5 and STEM leaguesStudent drivers, team-written autons, AI-written code, assisted driversWaypoints reached, objects delivered, in the time limit
VEX matches, 2 v 2All four robots in a match, filmed from one fixed cameraEach robot’s path, speed and contacts, auton and driver separately (in development)
VEX AIM classroomsStudents with the One Stick controller, class code, AI-written PythonWaypoints and colour sorting on a 6 × 4 ft table
Robot vacuums and mowersCommercial robots, coverage planners, AI-written plannersAll the floor or lawn covered, with little overlap and few bumps
Warehouse and lab cellsPicking robots and their control softwareEvery pick delivered, short travel, no near-misses
Classroom armsArms running class code or AI-written codeEvery object in the right place, small placement error
Simulated and AI agentsAny program, first in simulation, then on hardwareThe same goal as the real task, so the two can be compared

Your robot is invited to check its score

Teams and classes to measure their drivers and their code against the same goals, researchers to poke holes in the method, and robot makers willing to be measured in the open. Every real run makes the answer a little clearer.