Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Construction-Site Traversability → FiftyOne (Native Multimodal MCAP)#

preview

Construction-site ROS 2 recordings from an autonomous mobile robot, rewritten from ROS 2 MCAP into native multimodal MCAP episodes.

A tracked mobile robot drives closed loops through two active construction sites carrying an OAK-D RGB-D camera, a Livox 3D LiDAR, two inertial units and a u-blox GNSS receiver, with a LiDAR-inertial odometry estimate and the wheel encoders recorded alongside them. The runs revisit the same ground, which is what makes them useful for loop closure and for traversability work.

4 sessions, 1h 45m of driving over 9760 m of ground, across 2 at site1 and 2 at site2.

The recordings were made on working sites and contain site personnel and vehicles in the camera streams. They are republished here unchanged from a public release.

Installation#

pip install fiftyone

Usage#

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/Construction-Site-Traversability",
    name="Construction-Site-Traversability",
    persistent=True,
)
fo.launch_app(dataset)

The longest run:

view = dataset.sort_by("duration", reverse=True)
fo.launch_app(dataset, view=view)

What you get#

Each episode carries:

/camera, the OAK-D colour camera at 640x480, as foxglove.CompressedImage
/depth, the OAK-D range image at 640x400, as foxglove.CompressedImage
/lidar-points, the Livox sweeps, as foxglove.PointCloud with x, y, z, intensity, tag, line and time_offset
/imu.plot and /livox-imu.plot, the body and LiDAR inertial units at 100 Hz and 200 Hz
/odometry, the LiDAR-inertial pose estimate, as foxglove.PoseInFrame, with its position and velocity also on /odometry.plot
/wheel-odometry.plot, the wheel encoder estimate
/gnss, the receiver’s position fixes, as foxglove.LocationFix
/camera-calibration, the colour camera’s intrinsics, as foxglove.CameraCalibration
/tf, where the camera and the LiDAR sit on the robot, as foxglove.FrameTransform
/session, naming the run

Across the whole set that comes to 62,995 colour frames, 62,931 range frames, 62,995 LiDAR sweeps holding 0.82 billion points, 1,888,884 inertial samples, 556,459 pose estimates and 4,986 satellite fixes.

Session	Site	Colour	Range	Sweeps	Fixes	Duration	Path
`site1_session01`	site1	28,219	28,181	28,219	1,850	47m 02s	3166 m
`site1_session02`	site1	8,854	8,854	8,855	884	14m 45s	1203 m
`site2_session01`	site2	17,226	17,201	17,225	1,382	28m 43s	4852 m
`site2_session02`	site2	8,696	8,695	8,696	870	14m 30s	538 m

Episodes carry the fields sequence, site, session, recorded, scene, num_camera_frames, num_depth_frames, num_lidar_scans, num_lidar_points, num_imu_samples, num_livox_imu_samples, num_odometry_poses, num_gnss_fixes, has_gnss, odometry_path_m and duration.

Notes on the conversion#

The source is already MCAP, but it is ROS 2 MCAP: native ROS message schemas on their original topic names. The episodes here carry foxglove schemas the App reads directly, with the scalar tracks on .plot channels that drive its timeline.

The colour camera and the LiDAR both run at 10 Hz. The range camera runs at 20 Hz and every second frame is kept, so every visual and 3D stream in an episode shares one clock.

The range camera publishes an 8-bit single-channel image rather than metric depth, and the two cameras differ in resolution, so the frames are not pixel-aligned. Range frames are PNG whatever their bit depth, since a range image quantized by a lossy codec stops being one. Colour frames are JPEG at quality 92.

The LiDAR’s per-point time is published as time_offset, in seconds relative to the sweep. The source records it as an absolute timestamp, and the point cloud format packs fields as 32-bit floats, which quantize a value that size into steps of minutes.

odometry_path_m is measured from the pose stream decimated to 10 Hz. The estimate is published at 100 Hz and its per-tick noise is the same order as the robot’s travel between ticks, so summing it at full rate reads several kilometres of travel across a site tens of metres across.

The receiver publishes raw NMEA text. Only GGA sentences are carried, since they are the ones with an altitude and a fix quality, and sentences reporting no fix are dropped. That is why the fixes arrive at about 1 Hz against a much busier source topic. The fixes place both sites near Bangalore, India.

The LiDAR fires whether or not anything comes back, and the driver writes a miss as a point at the origin. Those are dropped, so a sweep holds returns only. What is left runs from 9,313 to 15,449 points per sweep depending on how open the ground was.

The camera and the LiDAR each publish in their own frame, so each episode carries /tf placing both on the robot, and /camera-calibration carrying the colour camera’s intrinsics. Both come from the calibration the authors publish alongside their annotated frames, republished at the repo root. The range camera has no published extrinsic and stays in its own frame.

The odometry estimate stays on /odometry rather than being published as a moving transform, since the calibration does not say which body it tracks and its height over a run matches neither the robot’s base nor the scanner.

The TF and static transform streams of the source recording and the receiver’s raw correction input are not reproduced here.

License & attribution#

The source dataset is released under CC BY-NC 4.0, and this conversion is distributed under the same license. Use is limited to non-commercial purposes and attribution is required.

Changes from the source: conversion from ROS 2 MCAP to the FiftyOne MCAP flavor, JPEG encoding of the colour frames, PNG encoding of the range frames, the range camera halved onto the 10 Hz clock, the LiDAR per-point timestamp rebased to a per-sweep offset, no-return points dropped, the NMEA text parsed into position fixes, and the authors’ calibration carried as the camera intrinsics and the rig transforms.

The curated annotated frames, camera calibration and the trained segmentation model that accompany these recordings are published separately by the authors at manojkarnekar/construction-traversability-dataset.