GraspXL synthetic data generation

Updated 16 Oct 2026, 10:40.

Each sample turns one released GraspXL grasp of an Objaverse object into a labeled pick-and-place on a random table. The MANO hand and the object follow the dataset motion until the grasp holds. Then the hand carries the object to a target, turns the wrist, and lets go. MuJoCo simulates the object after the release.

50samples on this page, 50 object categories

15,813usable objects in 1,018 LVIS categories

56 %of sources become a kept scene (56 of 100)

~26 scompute per kept scene on one RTX 5080, render included

What one sample contains

Hand: MANO right hand at 100 Hz: global rotation, translation and 45 pose values per frame.
Object: Mesh from the GraspXL object set and its 6-DoF path, with an estimated mass (material class times filled volume).
Scene: Table shape, size and photo texture; 0-3 free objects (ball, box, cylinder) and 0-2 raised surfaces (platform, stand, shelf, step) with their poses.
Label: One sentence that names the object and the target, for example "drop the strawberry onto the orange ball".
Forces: Contact forces, finger twists and joint torques per frame, and a flag for holds the contacts cannot carry ("glued").
Video: EEVEE render with a tracking camera, 1280x720 at 33 fps. This page shows 640x360 copies.

50 samples

Relations on this page: next to 8, drop on 11, place on 9, place on surface 11, drop on surface 11. Target colors rotate over the slots: red, green, yellow, purple, orange and white for free objects; wooden, white, gray and black for surfaces. Masses range from 18–1527 g. 17 samples have a glued hold: the hand moves the object, but the measured contacts cannot carry its weight.

Hide glued holds

How a sample is made

Source graspOne released GraspXL tabletop sequence (100 frames) of an LVIS-labeled object.

Approach0.5-1.0 s added before the grasp, along the approach direction.

SceneRandom table, texture, free objects and surfaces; a relation and a target.

Carry and releaseLift, carry with a wrist turn, release on the reversed approach path; MuJoCo after the release.

Checks, render, forcesFailed checks drop the scene. Kept scenes get a video and force estimates.

Quality gates

A source passes the gate when its grasp opposes the thumb to a finger or the palm and the added approach joins the dataset motion without a jump. A scene is kept when its outcome matches the label, the hand stays above the table, the release adds no deep penetration, and the object comes to rest. Glued holds stay, with a flag.

Job	Sources	Pass gate	Scenes	Kept	Glued	Minutes
`examples_local/job_0000`	20	15	15	12	4	5.5
`examples_local/job_0001`	20	18	18	14	4	5.3
`examples_local/job_0002`	20	19	19	14	5	5.4
`examples_local/job_0003`	20	17	15	8	3	4.0
`examples_local_extra/job_0000`	20	17	17	8	3	4.4
All	100	86	84	56	19	24.5

Scale-up on the cluster

One HTCondor job runs the whole chain for 20 sources on one GPU node. The first run is a pilot of 200 sources in 10 jobs, which should give about 110 kept scenes at the local rate.

running Test job (pilot job 0, 20 sources) runs again on an A100 node. Its first run passed every CPU stage and rendered all frames, but the video encoding failed.

Passed on the cluster: extension 131 s, grasp check 30 s, object geometry 58 s, scene build 367 s (the first run also builds the uv environments).
EEVEE renders correctly on the A100: 8 kept scenes, 164-244 frames each.
The first run failed at the video encoding: the node's system ffmpeg crashes on a library conflict with the NVIDIA OpenCL driver.
Fixed: the jobs now use a static ffmpeg build. The test job resumes at the render stage.
Next: the other 9 pilot jobs, about 110 kept scenes at the local rate of 56 %.

Frame 120 of scene 000 (futon, red box target), rendered with EEVEE on cluster node g170.

Coming soon

Full-body motion coming soon

An SMPL-X body fitted to every clip with the FUSION motion prior. The MANO hand stays exactly as generated, so the grasp contacts do not change.

Status (v8, 20 clips): 5 good, 15 with minor issues, 0 bad.
The body stands at the table edge, leans and bends its knees, and keeps its feet on the floor. No body part enters an object.
4-7 min per clip on one RTX 5080.
Open: the arm stays held forward at the clip end, the hip touches the table edge in 2 clips, feet glide in 5 clips.
FUSION is licensed for non-commercial research only.

SMPL-X bodies fitted to the 20 v3 clips (v8), 5 x 4 grid.

Room environments coming soon

Each clip placed in a MolmoSpaces iTHOR room as a rendered backdrop. The simulation, the labels and the checks stay the same.

Status: the 20 v4 clips rendered in 20 rooms, 10 kitchens and 10 living rooms.
Review: 5 OK, 15 with minor issues, 0 unusable. No room geometry passes through the table, the hand or the objects.
Open: in 11 clips the hovering hand seems to touch furniture behind the table (the real gap is at least 0.47 m), lens choice, table margins near furniture, brightness.
Next: a room stage in the batch job.

The 20 v4 clips in iTHOR kitchens and living rooms.

Known limits

Objects are hand-sized replicas: GraspXL scales every object to the hand, so a "chair" is a toy chair.
"Drop on" a free object needs only a touch. A dropped object often rolls off a ball or a box.
Masses come from a material class per LVIS category and the mesh volume. Some are high, for example solid replicas of hollow things.
The hand moves rigidly with the object during the carry. About a third of the kept scenes have a glued hold.
MANO right hand only. No other hand or gripper yet.

Reproduce

python3 build/render_tools/plan_scale_run.py --out build/scale/my_run --n_sources 80 --job_size 20 --seed 1
python3 build/render_tools/scene_job.py --job_dir build/scale/my_run/job_0000 --job_id 0 --workers 8
python3 build/render_tools/scene_samples_page.py

Built by build/render_tools/scene_samples_page.py from the job folders in build/scale/.