Egocentric EMG-Force#

First-person household task recordings pairing RGB-D video with wrist EMG and per-finger contact force, as native .mcap episodes.

Someone wearing a depth camera and an eight-channel EMG band on each wrist works through eight household tasks: sweeping and mopping, tidying a bedroom, a dining room and a living room, loading a washing machine, washing hands at a sink, wiping hands on paper towel, and applying hand cream. Alongside the video each episode carries the muscle signal from both forearms at roughly 550 Hz, a 21-point hand skeleton per frame, an estimate of how hard each finger is pressing, and wrist IMU.

Every episode is cut into subtasks a reviewer described in English, so the 122 segments read as instructions rather than indices: pick up dustpan and broom, fit the trash bag into the basin, restock paper rolls.

38.2 minutes of recording: 67,185 camera and depth frames, 2,539,493 EMG samples, 60,254 hand skeletons and 133,721 per-finger force readings across 121 distinct phases.

Contact force is an estimate derived from hand pose and depth rather than a reading from an instrumented glove, and it saturates at 45 N. The wrist EMG leads the camera by 80 ms, and the corrected clock the release publishes is the one used here.

Details

Example usage

 1import fiftyone as fo
 2import fiftyone.zoo as foz
 3
 4dataset = foz.load_zoo_dataset("egocentric-emg-force")
 5
 6# The tasks with the firmest contact
 7view = dataset.match({"peak_finger_force": {"$gt": 40}})
 8
 9# Episodes that involve folding
10view = dataset.match(
11    {"phases": {"$elemMatch": {"$regex": "fold"}}}
12)
13
14session = fo.launch_app(dataset)
egocentric-emg-force