#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# APAC Egocentric Stereo → FiftyOne (Native Multimodal MCAP)
The labeled stereo release of
[humyn-labs/APAC-Egocentric-Stereo-Labeled](https://huggingface.co/datasets/humyn-labs/APAC-Egocentric-Stereo-Labeled),
converted to native multimodal MCAP episodes.
Twelve people were filmed doing their jobs while wearing a head-mounted
stereo rig. Each sequence runs about a minute and carries the rectified
video from both eyes, a depth render, a hand and head tracking render, and a
caption describing what the wearer is doing at every moment. The work spans
industrial, hospitality, logistics and retail settings: an automotive
garage, a construction site, an electronics factory, a restaurant, a
kitchen, a bar, a shipment hub, a laundromat, a shopping mart, a clothes
store and a book store.
## Installation
```bash
pip install fiftyone
```
## Usage
```python
import fiftyone as fo
import fiftyone.utils.huggingface as fouh
dataset = fouh.load_from_hub(
"Voxel51/APAC-Egocentric-Stereo",
name="APAC-Egocentric-Stereo",
persistent=True,
)
fo.launch_app(dataset)
```
The industrial workplaces:
```python
view = dataset.match({"environment": "Industrial"})
```
## What you get
Twelve episodes and 12.0 minutes of recording at 30 fps. Each episode
carries:
- `/stereo-camera`, the rectified pair at 3840x1080 with 1920x1080 per eye,
as `foxglove.CompressedVideo`
- `/depth-map` and `/hand-tracking`, both 1920x1080, as
`foxglove.CompressedVideo`, on the same frame clock as the cameras
- `/action` and `/action-caption`, the verb, the object and the full caption
for each segment, stamped where the segment opens
- `/action.plot`, the index of the segment currently running
- `/instruction`, the task description for the sequence
Across the whole set that comes to 248 captioned segments and 62 distinct
verbs.
Episodes carry the fields `sample_id`, `environment`, `scene`,
`task_description`, `verbs`, `num_segments`, `num_frames`, `duration`,
`fps`, `sbs_width`, `sbs_height`, `per_eye_width`, `depth_width` and
`depth_height`.
## Notes on the conversion
The depth stream is a false-colour 8-bit render distributed as H.264 video.
Its pixel values are a colormap. Nothing in the source carries per-pixel
distance.
Hand and head tracking is an overlay drawn onto the camera frame, not a
stream of joint coordinates.
Both eyes ride in one side-by-side stream. The source publishes the pair
pre-rectified in a single file, and the halves are cut at `per_eye_width`.
All three streams are Annex-B H.264 without B-frames. The source stereo video
is MPEG-4 Part 2, and the source depth and tracking videos are H.264 with
B-frames, so each is re-encoded rather than carried through.
Segments tile each sequence end to end with no gaps. They run three seconds
each, except the last in a sequence, which is whatever time remains. One
sequence carries 17 segments rather than 21 because its clip runs 50.8
seconds rather than about 60.
The source’s verb vocabulary is inconsistently cased, listing both
`Applying` and `applying`. The `verbs` field is deduplicated
case-insensitively. The `/action` messages carry the verb as written.
## License & attribution
The source dataset is released by
[humyn labs](https://huggingface.co/humyn-labs) under
[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), and this
conversion is distributed under the same license.
Changes from the source: conversion to the FiftyOne MCAP flavor, encoding of
the action segments as message streams, and re-encoding of the video.
A monocular counterpart is published at
[humyn-labs/APAC-Egocentric-Monocular-Labeled](https://huggingface.co/datasets/humyn-labs/APAC-Egocentric-Monocular-Labeled).