Results by task

330 scored runs across 3 tasks, scorer v0.1.0.

Every result on this page is synthetic. The agents are simulated with hand-tuned behaviour (drivers who correct visually, routines that drift, AI routines that improve with feedback) so the pipeline can be tested in public. Real field results replace this board once pilot teams have run the tasks.

Waypoint navigation on a 12 ft field

MB-VEX-NAV@1.0 · 200 runs · median of 12 simulated people: 5.5 waypoints · floor 0.0 waypoints (random walk)

Human
Human + assist
AI-written
Team-written code
Random-walk floor

Replay these runs and check them in the explorer

Show the table

Mean, worst and best in waypoints over 10+ runs per agent. Agents share a rank when their 95% intervals overlap.
Rank	Agent	Runs	Mean	Worst	Best	Path eff.	Violations	HNS	MBI
1	Driver 06 · Human	10	6.0	6.0	6.0	0.80	0.0	1.09	1.07
1	Driver 10 · Human	10	6.0	6.0	6.0	0.80	0.0	1.09	1.07
1	Driver 12 · Human	10	6.0	6.0	6.0	0.81	0.0	1.09	1.07
1	Driver 05 with auto-align · Human + assist	10	6.0	6.0	6.0	0.87	0.0	1.09	1.08
1	LLM-written auton, round 10 · AI-written	10	6.0	6.0	6.0	0.94	0.0	1.09	1.11
1	LLM-written auton, round 3 · AI-written	10	6.0	6.0	6.0	0.93	0.0	1.09	1.10
1	Team auton A (odometry + GPS correction) · Team-written code	10	6.0	6.0	6.0	0.94	0.0	1.09	1.10
1	Driver 11 · Human	10	5.8	5.0	6.0	0.75	0.0	1.05	1.02
9	Driver 02 · Human	10	5.7	5.0	6.0	0.74	0.0	1.04	1.01
9	Driver 09 · Human	10	5.7	5.0	6.0	0.75	0.0	1.04	1.01
9	Driver 05 · Human	10	5.3	5.0	6.0	0.71	0.0	0.96	0.96
9	Driver 04 · Human	10	5.1	4.0	6.0	0.70	0.1	0.93	0.93
9	Driver 07 · Human	10	5.0	4.0	6.0	0.68	0.0	0.91	0.92
9	Team auton B (odometry only) · Team-written code	10	4.9	1.0	6.0	0.95	0.0	0.89	0.95
15	Driver 01 · Human	10	4.2	3.0	5.0	0.64	0.0	0.76	0.84
15	Driver 08 · Human	10	3.7	3.0	4.0	0.61	0.1	0.67	0.77
17	Driver 03 · Human	10	3.3	3.0	4.0	0.57	0.1	0.60	0.72
17	LLM-written auton, round 1 · AI-written	10	2.1	0.0	6.0	0.66	0.3	0.38	0.49
19	Team auton C (timed, open loop) · Team-written code	10	1.2	0.0	4.0	0.66	0.2	0.22	0.42
20	Random walk (floor) · Random-walk floor	10	0.0	0.0	0.0	0.00	0.5	0.00	0.10

Collect and deliver on a 12 ft field

MB-VEX-PICK@1.0 · 100 runs · median of 12 simulated people: 5.7 objects · floor 0.0 objects (random walk)

Human
Human + assist
AI-written
Team-written code
Random-walk floor

Replay these runs and check them in the explorer

Show the table

Mean, worst and best in objects over 5+ runs per agent. Agents share a rank when their 95% intervals overlap.
Rank	Agent	Runs	Mean	Worst	Best	Path eff.	Violations	HNS	MBI
1	Driver 02 · Human	5	6.0	6.0	6.0	0.81	0.2	1.05	1.00
1	Driver 10 · Human	5	6.0	6.0	6.0	0.88	0.0	1.05	1.05
1	Driver 12 · Human	5	6.0	6.0	6.0	0.89	0.0	1.05	1.05
1	Driver 05 with auto-align · Human + assist	5	6.0	6.0	6.0	0.88	0.0	1.05	1.05
1	LLM-written auton, round 10 · AI-written	5	6.0	6.0	6.0	0.94	0.0	1.05	1.06
1	LLM-written auton, round 3 · AI-written	5	6.0	6.0	6.0	0.93	0.0	1.05	1.06
1	Team auton A (odometry + GPS correction) · Team-written code	5	6.0	6.0	6.0	0.92	0.0	1.05	1.06
1	Driver 04 · Human	5	5.8	5.0	6.0	0.80	0.4	1.02	0.96
1	Driver 06 · Human	5	5.8	5.0	6.0	0.84	0.0	1.02	1.01
1	Driver 09 · Human	5	5.8	5.0	6.0	0.80	0.4	1.02	0.96
1	Driver 01 · Human	5	5.6	5.0	6.0	0.75	0.0	0.98	0.96
1	Driver 05 · Human	5	5.6	5.0	6.0	0.78	0.0	0.98	0.97
1	Driver 11 · Human	5	5.6	4.0	6.0	0.79	0.0	0.98	0.97
14	Driver 07 · Human	5	5.2	4.0	6.0	0.70	0.0	0.91	0.91
14	Driver 03 · Human	5	5.0	4.0	6.0	0.66	0.0	0.88	0.88
14	Driver 08 · Human	5	5.0	4.0	6.0	0.62	0.2	0.88	0.84
14	Team auton B (odometry only) · Team-written code	5	4.4	3.0	6.0	0.92	0.0	0.77	0.88
18	Team auton C (timed, open loop) · Team-written code	5	2.2	2.0	3.0	0.80	0.0	0.39	0.66
18	LLM-written auton, round 1 · AI-written	5	1.8	0.0	3.0	0.49	0.2	0.32	0.47
20	Random walk (floor) · Random-walk floor	5	0.0	0.0	0.0	0.00	1.0	0.00	0.07

Room floor coverage

MB-COVER@0.1 · 30 runs · median of 6 simulated people: 86.9% · floor 49.2% (random walk)

Commercial robot
Human
AI-written
Random-walk floor

Replay these runs and check them in the explorer

Show the table

Mean, worst and best in % covered over 3+ runs per agent. Agents share a rank when their 95% intervals overlap.
Rank	Agent	Runs	Mean	Worst	Best	Path eff.	Violations	HNS	MBI
1	Robot vacuum, systematic (lane-planning) · Commercial robot	3	94.7	94.7	94.8	0.64	0.3	1.21	1.03
2	Person 03 with stick vacuum · Human	3	91.0	89.8	92.5	0.78	2.0	1.11	0.95
2	LLM-written coverage planner, round 3 · AI-written	3	89.7	89.6	89.8	0.76	0.0	1.08	1.04
4	Person 02 with stick vacuum · Human	3	88.1	87.4	89.2	0.81	2.7	1.03	0.91
4	Person 01 with stick vacuum · Human	3	87.2	85.9	89.3	0.74	1.7	1.01	0.90
4	Person 05 with stick vacuum · Human	3	86.6	83.9	89.7	0.80	5.3	0.99	0.87
7	Person 06 with stick vacuum · Human	3	83.2	80.6	85.0	0.77	2.0	0.90	0.84
7	Person 04 with stick vacuum · Human	3	81.1	79.6	82.6	0.77	2.0	0.84	0.82
9	Robot vacuum, bump-and-turn · Commercial robot	3	59.0	57.7	60.6	0.44	32.3	0.26	0.40
10	Random walk (floor) · Random-walk floor	3	49.2	45.3	53.8	0.31	7.7	0.00	0.23

Reproduce these numbers

pip install motionbench
motionbench synth --out data/synthetic-v0.1 --seed 2026
motionbench aggregate data/synthetic-v0.1 -o leaderboard.json

The dataset regenerates byte-for-byte and the Rust and Python scorers produce identical leaderboards. Read the dataset card.