#### NOTE This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs. ![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-yellow) # Dataset Card for SharpaDex Assemble Gears (one season, 410 episodes) ![SharpaDex Assemble Gears preview](https://huggingface.co/datasets/Voxel51/sharpa-assemble-gears-410ep/resolve/main/sharpa_assemble_gears.gif) This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 410 samples. Each sample is one teleoperated bimanual episode of the task “Gear Assembly on Base”. ## Installation ```bash pip install -U fiftyone ``` ## Usage ### Load from the Hub ```python import fiftyone as fo from fiftyone.utils.huggingface import load_from_hub dataset = load_from_hub("Voxel51/sharpa-assemble-gears-410ep") session = fo.launch_app(dataset) ``` ### Download a snapshot and load it locally ```python import fiftyone as fo from huggingface_hub import snapshot_download snapshot_download( repo_id="Voxel51/sharpa-assemble-gears-410ep", repo_type="dataset", local_dir="sharpa-assemble-gears-410ep", ) dataset = fo.Dataset.from_dir( dataset_dir="sharpa-assemble-gears-410ep", dataset_type=fo.types.LeRobotDataset, name="sharpa_assemble_gears", ) session = fo.launch_app(dataset) ``` ## Dataset Details ### Dataset Description One collection season (`season_POC22027_2026_04_30_15_04_50_train`) of the `assemble_gears_on_base` task from SharpaDex v1.0, taken from its `lerobot_v3.0` export: 410 episodes, 302,020 frames at 30 fps. The full SharpaDex v1.0 release has 28,993 episodes across 59 tasks and 495 seasons (about 300 hours); this FiftyOne dataset covers only this one season of one task. - **Curated by:** Sharpa - **Shared by:** `Sharpa-Robotics` on Hugging Face - **Language(s):** en - **License:** CC-BY-4.0 ### Dataset Sources - **Repository:** https://huggingface.co/datasets/Sharpa-Robotics/SharpaDex-v1.0 (path: `assemble_gears_on_base/season_POC22027_2026_04_30_15_04_50_train/lerobot_v3.0`) - **Demo:** https://www.sharpa.com/ ## Uses ### Direct Use Per the source README: research on imitation learning, visuomotor control, vision-language-action models, visual-tactile learning, and hierarchical policy learning. ## Dataset Structure **Topology.** `media_type: multimodal`, 410 samples, one per episode, 30 fps, 523-1,101 frames (17.4-36.7 s), 302,020 frames total. Samples hold a `media_reference` to the per-frame parquet data and 6 video streams. | Field | FiftyOne type | Description | |-------------------|---------------------------|------------------------------------------------------------------| | `media_reference` | `LeRobotEpisodeReference` | Pointers into `data/` parquet and the 6 video streams | | `episode_index` | `IntField` | Episode index (0-409) | | `task` | `StringField` | Structured task text (`Task`, `Instruction`, `Scene`, `Success`) | | `tasks` | `ListField(StringField)` | All task strings of the episode | | `length` | `IntField` | Frames in the episode | | `duration` | `FloatField` | Episode duration in seconds | | `robot_type` | `StringField` | `robot` (the value declared in the source `info.json`) | | `fps` | `FloatField` | 30 | **Data behind `media_reference`** (per-frame, from `meta/info.json` and the source README): | Feature | Shape | Description | |---------------------------------------------------------------------------|------------|--------------------------------------------------------------------------------------------------------| | `observation.state` | (65,) | Joint state: left arm 0-6, left hand 7-28, right arm 29-35, right hand 36-57, torso/motor joints 58-64 | | `action` | (65,) | Joint-space action target, same layout | | `observation.state.joint_torque` | (65,) | Joint-torque signal, same layout | | `observation.state.tcp` | (24,) | Left pose 0:6, left force/torque 6:12, right pose 12:18, right force/torque 18:24 | | `observation.state.tcp_pose` | (12,) | Observed TCP poses, left 0:6 and right 6:12 | | `action.tcp_pose` | (12,) | Commanded TCP poses, left 0:6 and right 6:12 | | `observation.tactile` | (60,) | 10 fingertips x 6-axis force/torque signal | | `subtask_index` | (1,) int64 | Index into the source subtask table | | `observation.images.head_left`, `head_right`, `wrist_left`, `wrist_right` | 480x480x3 | RGB cameras (H.264 yuv420p) | | `observation.images.tactile_deform` | 480x1200x3 | Tactile deformation video (H.264 yuv420p) | | `observation.images.tactile_raw` | 480x1600x3 | Raw tactile video (H.264 yuv420p) | **Label types and why.** The importer produces episode-level samples with a `media_reference`; numeric streams (state, action, torque, tactile force/torque, TCP) are read through the viewer and are not stored as FiftyOne label fields. The source’s temporally grounded language segments (start/end step, text, skill) are not imported as FiftyOne fields. No custom fields were added, so the FiftyOne dataset matches what the LeRobot export contains. **`dataset.info`.** `lerobot` entry with `format`, `format_major`, `episode_count`, `imported_episode_count`, `skipped_episodes` (empty). **Parsing decisions.** - One season was chosen as a bounded, self-contained slice (19.6 GB) of a 3.4 TB repo. It is a single contiguous season, not a random sample of the task or the release. - All 6 video streams were imported; no modality was excluded. No source file was modified. - The source README states the export does not fully declare TCP units, reference frames or rotation convention (`rotation_type` is null). Do not infer Euler angles or quaternions from the vector width alone. - `robot_type` is the literal string `robot` in the source `info.json`. - Sidecar metadata not carried by the export: the FiftyOne LeRobot exporter writes only `info.json`, `stats.json`, `tasks.parquet`, the episodes table, `data/` and `videos/`. The source `meta/subtasks.parquet`, `meta/annotations.jsonl` and `meta/modality.json` are therefore not in the pushed repo. The `subtask_index` column is kept in the per-frame parquet, but the subtask text and skill labels it points to are available only in the source repo. ## Dataset Creation ### Curation Rationale Per the source README: a real-world teleoperation dataset for bimanual dexterous manipulation, covering assembly, tool use, deformable-object manipulation, cleaning, material transfer, and long-horizon tasks. ### Source Data #### Data Collection and Processing Per the source README: human teleoperation of a real bimanual robotic system with two 7-DoF arms, two 22-DoF dexterous hands, four RGB cameras, and fingertip tactile sensors. Each episode provides synchronized joint state, joint torque, tool-center-point state, observed and commanded TCP poses, action, numeric tactile measurements, RGB video, tactile video, and temporally aligned language annotations. #### Who are the source data producers? Sharpa. ### Annotations #### Annotation process Per the source README: every episode has a global structured task description (`Task`, `Instruction`, `Scene`, `Success`) and temporally grounded subtask segments with a skill label. ## Citation **BibTeX:** ```bibtex @misc{sharpadex_v1_2026, title = {SharpaDex v1.0: Large-Scale Bimanual Dexterous Manipulation Demonstrations}, author = {{Sharpa}}, howpublished = {\url{https://huggingface.co/datasets/Sharpa-Robotics/SharpaDex-v1.0}}, year = {2026} } ``` **APA:** Sharpa. (2026). SharpaDex v1.0: Large-Scale Bimanual Dexterous Manipulation Demonstrations. https://huggingface.co/datasets/Sharpa-Robotics/SharpaDex-v1.0