CrowdMAPS: Crowdsourcing Multiple Actions per State for Real-World Robot Imitation Learning

A single visited robot state often admits several valid next actions: a grasp can be approached from slightly different poses, and a multi-step task can be completed in different orders. Standard imitation learning datasets record only the one action that happened to be executed. CrowdMAPS decouples hardware-bound state acquisition from software-scalable action annotation: real robots provide physically grounded states, and remote annotators add alternative valid actions for those states from a web browser, without access to robot hardware.

A standard trajectory pairs each visited observation o_t with one executed action a_t. A MAPS dataset keeps exactly the same visited observations but pairs each of them with a set of valid actions A_t: the executed action plus valid, unexecuted alternatives A_t^¬exec. The number of actions per state, K, is the dataset’s “width.” Under independent sampling, MAPS data scales the action-dependent variance of the imitation-learning gradient estimator by 1/K, without introducing bias.

In our implementation, every action is a goal pose (end-effector position, orientation, and a gripper open/close command) that a motion planner executes. Annotation is keyframe-centric: keyframes, which occur at the start of a trajectory and whenever a goal is reached, store the executed goal pose plus crowdsourced alternatives, while intermediate frames are paired with the goal currently being pursued.

CrowdMAPS collects alternative goal poses in two stages:

1. State acquisition. An expert collects standard real-robot trajectories using the same goal-pose representation.
2. Action densification. Recorded keyframes are distributed to remote annotators. For each keyframe, LangSAM segments the task objects, Any6D estimates their 6-DoF poses, and the corresponding CAD meshes are placed in Isaac Lab to reconstruct the scene. Annotators manipulate a rendered robot in this reconstruction, alongside the raw camera views, to specify an alternative goal pose.

A researcher validates each submission for consistency with the manipulation strategy used during physical data collection, semantic alignment with the keyframe instruction, and physical coherence. Because annotation needs no robot, many people can contribute at the same time, avoiding the one-user-one-robot bottleneck of synchronous web teleoperation.

• Keypose policies predict the next goal pose directly and are supervised on every label in each action set. We adapt Diffusion Policy for this setting.
• Trajectory policies are diffusion policies that predict action chunks. Each MAPS goal pose is converted into a synthetic chunk by interpolating from the current end-effector pose, and the policy is co-trained on these chunks and on continuous expert demonstrations.

Because alternative actions are never executed, the states they would lead to are absent from the data. We call this MAPS shift and mitigate it in two ways: validation keeps crowdsourced actions within the chosen manipulation strategy, and the loss weights executed actions above unexecuted alternatives, which in turn are weighted above intermediate-frame goals.

We recruited 47 participants through university mailing lists and social media; 78.7% had no prior experience with robot arms. Across the four tasks with timing logs, the median submission took 7 to 12 seconds, so annotators can contribute in short sessions rather than through sustained teleoperation. Submission acceptance rates were 51% for Drawer, 98% for Insertion, 66% for Pouring, 83% for Sorting, and 94% for Switches, and annotators generally became faster and more accurate with practice.

Averaged across all task and dataset-size settings, increasing width from K = 1 to K = 10 raises success from 38.7% to 74.2%, a gain of 35.6 percentage points. The effect is strongest on longer-horizon and multimodal tasks: on Sorting and Switches⁻, it averages 42.5 points, and K = 10 is the only width that exceeds 85% success.

The K = 10 (noise) row replaces every crowdsourced alternative with a Gaussian perturbation of the executed action, with covariance estimated from the crowd annotations themselves. Despite matching the action density of K = 10, it reaches only 43.3% average success, compared with 74.2% for crowdsourced labels. It also gets worse with more data in several settings: from 88% to 52% on Insertion and from 60% to 24% on Sorting.

We sample 100 goal-pose predictions from keypose policies trained with K = 1, 2, 5, 10 at the same held-out keyframe. On the unimodal Insertion task, wider MAPS data shrinks both the offset and the spread of the predictions. On the multimodal Sorting task, low-width policies smear predictions between the two cubes, whereas at K = 10 the policy represents both valid choices as distinct, sharp modes.

MAPS adds action labels only at states the robot already visited, so the gains cannot come from broader state coverage. We attribute them to two compounding effects: within-mode densification (denser sampling of valid actions around one solution) and multimodal coverage (distinct valid solutions represented at the same state). Crowd labels show both: the median within-mode spread ranges from 2.8 mm (Insertion) to 17.0 mm (Drawer), while distinct modes on Sorting and Switches are separated by a median of 122.8 mm and 201.5 mm. Consistently, the improvement from K = 1 to K = 10 grows with multimodality: +22, +32, +39, and +46 percentage points on Insertion, Pouring, Sorting, and Switches⁻.

Starting from a base set of expert demonstrations, we compare adding more expert teleoperated trajectories with adding CrowdMAPS data. At K = 1, MAPS data often underperforms extra teleoperation, since single keyframe labels carry less temporal detail. As K grows, denser supervision makes up for this: increasing width from 1 to 10 improves average success by 33.0 percentage points, and at K = 10 MAPS data gives the best result on every task.

The comparison approximately matches robot time (2.2 hours for MAPS data vs. 2.3 hours for the added expert trajectories) but not total human effort: at matched performance, MAPS data required about 1.5× more human time.