Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Hugging Face

SEW Multimodal AMR → FiftyOne (Native Multimodal MCAP)#

preview

The labeled test set of the SEW-EURODRIVE Multimodal AMR dataset, converted to native multimodal MCAP episodes. Six sensing modalities ride one autonomous mobile robot: RGB, thermal, time-of-flight, 4D radar, two 2D laser scanners, and an ultrasonic array. The 3,151 labeled frames are split into 55 episodes, one per source recording session, spanning three seasons, six weather conditions, and day, dawn and night.

Installation#

pip install fiftyone

Usage#

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/SEW-Multimodal-AMR",
    name="SEW-Multimodal-AMR",
    persistent=True,
)
fo.launch_app(dataset)

What you get#

  • 55 .mcap episodes of 6 to 317 frames, every modality sharing one frame clock

  • Cameras: /rgb-camera (1224x1024) and /thermal-camera (640x512), each with a -calibrated counterpart registered to the thermal frame

  • Time of flight: /tof-depth and /tof-amplitude as 16-bit PNG, plus /tof-points as foxglove.PointCloud

  • Radar: /radar-points as foxglove.PointCloud, and six heatmap views on /radar-azimuth-abs, /radar-azimuth-phase, /radar-doppler-abs, /radar-doppler-phase, /radar-elevation-abs, /radar-elevation-phase

  • /laserscan-left-front and /laserscan-right-back as foxglove.LaserScan

  • /ultrasonic.plot carrying the four-sensor range array

  • /annotations-3d as foxglove.SceneUpdate: 9,813 cuboids across person, bicycle, doll, slidecar, curb and vegetation

  • /rgb-annotations and /thermal-annotations as foxglove.ImageAnnotations

  • /conditions naming the season, weather and time of day

  • Per-episode fields: session, season, weather, daytime, num_frames, duration, num_boxes_3d

Notes on the conversion#

Depth and amplitude are carried as 16-bit PNG at their native scale, so depth values are millimetres over a 0 to roughly 12,500 range. Viewers that map the full 16-bit range will render them dark; the preview above is contrast stretched for display only.

The raw radar cubes shipped as MATLAB .mat files are not included. They are two thirds of the source payload and no viewer renders them; the derived heatmaps and point clouds are.

Cuboids are carried in the source’s own left-handed sensor frame with its h w l and x y z fields unchanged rather than re-based into a right-handed convention.

The source ships no class-name file for the YOLO labels. The four YOLO indices are mapped to person, bicycle, slidecar and doll by matching per-class box counts against the KITTI labels, which name their classes directly. Curb and vegetation are annotated in 3D only.

Outside one dense 910-frame block, the test set samples every tenth source frame, so most episodes are a low-rate timeline rather than continuous video.

License & attribution#

The source dataset is released by SEW-EURODRIVE GmbH & Co. KG under CC-BY-SA-4.0; this conversion is distributed under the same license.

Citation#

@misc{sew_dataset_2025,
  author = {{SEW-Eurodrive GmbH \& Co. KG}},
  title  = {{SEW-Dataset}},
  year   = {2025},
  url    = {https://share.sew-eurodrive.de/sew-dataset}
}