#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the <a href="https://docs.voxel51.com/integrations/huggingface.html#loading-datasets-from-the-hub" target="_blank">Hugging Face integration docs</a>.

<a href="https://huggingface.co/datasets/Voxel51/SEW-Multimodal-AMR" target="_blank">![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-yellow)</a>

# SEW Multimodal AMR → FiftyOne (Native Multimodal MCAP)

![preview](https://huggingface.co/datasets/Voxel51/SEW-Multimodal-AMR/resolve/main/preview.gif)

The labeled test set of the
[SEW-EURODRIVE Multimodal AMR dataset](https://github.com/SEW-Eurodrive-Open-Source/Multimodal_AMR_dataset),
converted to native multimodal MCAP episodes. Six sensing modalities ride one
autonomous mobile robot: RGB, thermal, time-of-flight, 4D radar, two 2D laser
scanners, and an ultrasonic array. The 3,151 labeled frames are split into 55
episodes, one per source recording session, spanning three seasons, six
weather conditions, and day, dawn and night.

## Installation

```bash
pip install fiftyone
```

## Usage

```python
import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/SEW-Multimodal-AMR",
    name="SEW-Multimodal-AMR",
    persistent=True,
)
fo.launch_app(dataset)
```

## What you get

- 55 `.mcap` episodes of 6 to 317 frames, every modality sharing one frame clock
- Cameras: `/rgb-camera` (1224x1024) and `/thermal-camera` (640x512), each
  with a `-calibrated` counterpart registered to the thermal frame
- Time of flight: `/tof-depth` and `/tof-amplitude` as 16-bit PNG, plus
  `/tof-points` as `foxglove.PointCloud`
- Radar: `/radar-points` as `foxglove.PointCloud`, and six heatmap views on
  `/radar-azimuth-abs`, `/radar-azimuth-phase`, `/radar-doppler-abs`,
  `/radar-doppler-phase`, `/radar-elevation-abs`, `/radar-elevation-phase`
- `/laserscan-left-front` and `/laserscan-right-back` as `foxglove.LaserScan`
- `/ultrasonic.plot` carrying the four-sensor range array
- `/annotations-3d` as `foxglove.SceneUpdate`: 9,813 cuboids across person,
  bicycle, doll, slidecar, curb and vegetation
- `/rgb-annotations` and `/thermal-annotations` as
  `foxglove.ImageAnnotations`
- `/conditions` naming the season, weather and time of day
- Per-episode fields: `session`, `season`, `weather`, `daytime`,
  `num_frames`, `duration`, `num_boxes_3d`

## Notes on the conversion

Depth and amplitude are carried as 16-bit PNG at their native scale, so depth
values are millimetres over a 0 to roughly 12,500 range. Viewers that map the
full 16-bit range will render them dark; the preview above is contrast
stretched for display only.

The raw radar cubes shipped as MATLAB `.mat` files are not included. They are
two thirds of the source payload and no viewer renders them; the derived
heatmaps and point clouds are.

Cuboids are carried in the source’s own left-handed sensor frame with its
`h w l` and `x y z` fields unchanged rather than re-based into a right-handed
convention.

The source ships no class-name file for the YOLO labels. The four YOLO
indices are mapped to person, bicycle, slidecar and doll by matching
per-class box counts against the KITTI labels, which name their classes
directly. Curb and vegetation are annotated in 3D only.

Outside one dense 910-frame block, the test set samples every tenth source
frame, so most episodes are a low-rate timeline rather than continuous video.

## License & attribution

The source dataset is released by SEW-EURODRIVE GmbH & Co. KG under
[CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/); this
conversion is distributed under the same license.

## Citation

```bibtex
@misc{sew_dataset_2025,
  author = {{SEW-Eurodrive GmbH \& Co. KG}},
  title  = {{SEW-Dataset}},
  year   = {2025},
  url    = {https://share.sew-eurodrive.de/sew-dataset}
}
```