Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Hilti SLAM Challenge 2023 → FiftyOne (Native Multimodal MCAP)#

preview

The Hilti SLAM Challenge 2023 recordings, converted from ROS 1 bags to native multimodal MCAP episodes.

The 2023 challenge extends the benchmark to multi-session and multi-platform mapping. Every run was walked or driven through an active construction site, and the runs at a site overlap, so a sequence can be solved on its own or several can be combined into one map.

Two rigs recorded it. The handheld platform is the Phasma-style suite carried over from the 2022 challenge: five synchronized global-shutter cameras at 720x540, a Hesai PandarXT-32 LiDAR and an inertial unit, with the cameras locked together by an FPGA and the cameras and LiDAR by PTP to within about a millisecond. The robot is a tracked drilling platform carrying four OAK-D stereo pairs, so eight camera streams at 1280x800, a RoboSense BPearl hemispherical LiDAR and an Xsens MTi-670.

A surveyor measured reference positions along every run.

Photosensitivity warning. site2_handheld_4, site2_handheld_5, site2_handheld_6 carry rapidly flashing lights in their camera streams. Every episode carries has_flashing_lights so they can be excluded before the App is opened.

Installation#

pip install fiftyone

Usage#

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/Hilti-SLAM-Challenge-2023",
    name="Hilti-SLAM-Challenge-2023",
    persistent=True,
)
fo.launch_app(dataset)

The runs recorded from the robot rather than by hand:

view = dataset.match({"platform": "robot"})
fo.launch_app(dataset, view=view)

Everything except the runs with flashing lights:

view = dataset.match({"has_flashing_lights": False})

What you get#

15 runs, 55m 40s of recording. 12 from the handheld rig and 3 from the robot, across 5 at site1, 6 at site2, 4 at site3.

Each episode carries:

/cam0 onward, the cameras, as foxglove.CompressedImage. Five on the handheld rig, eight on the robot
/cam0-calibration onward, the intrinsics for each camera, as foxglove.CameraCalibration
/lidar-points, the LiDAR at its native rate, as foxglove.PointCloud
/imu.plot, three-axis acceleration and angular rate
/ground-truth.plot, the surveyed reference position at each instant the release measured one
/ground-truth-control-points, those positions drawn together, as foxglove.SceneUpdate
/tf, the rig’s frame tree, as foxglove.FrameTransform
/sequence, naming the run

Across the whole set that comes to 207,825 camera frames, 33,401 LiDAR sweeps holding 1.98 billion points, 1,061,139 inertial samples and 63 surveyed reference positions.

Sequence	Site	Platform	Cameras	Frames	Sweeps	Duration	Marks
`site1_handheld_1`	site1	handheld	5	10,240	2,047	3m 25s	4
`site1_handheld_2`	site1	handheld	5	8,360	1,672	2m 47s	4
`site1_handheld_3`	site1	handheld	5	8,536	1,707	2m 51s	4
`site1_handheld_4`	site1	handheld	5	14,775	2,955	4m 55s	3
`site1_handheld_5`	site1	handheld	5	7,970	1,594	2m 39s	3
`site2_handheld_4`	site2	handheld	5	4,665	932	1m 33s	2
`site2_handheld_5`	site2	handheld	5	10,280	2,056	3m 26s	4
`site2_handheld_6`	site2	handheld	5	6,870	1,373	2m 17s	2
`site2_robot_1`	site2	robot	8	55,954	6,996	11m 40s	7
`site2_robot_2`	site2	robot	8	24,472	3,059	5m 06s	3
`site2_robot_3`	site2	robot	8	28,730	3,591	5m 59s	5
`site3_handheld_1`	site3	handheld	5	4,864	972	1m 37s	4
`site3_handheld_2`	site3	handheld	5	7,360	1,482	2m 28s	6
`site3_handheld_3`	site3	handheld	5	9,399	1,896	3m 10s	8
`site3_handheld_4`	site3	handheld	5	5,350	1,069	1m 47s	4

Episodes carry the fields sequence, site, platform, recorded, scene, cameras, camera_sources, lidar, imu, is_challenge_sequence, has_flashing_lights, num_camera_frames, num_lidar_scans, num_lidar_points, num_imu_samples, num_control_points, control_point_names and duration.

Notes on the conversion#

Camera frames are published at 10 Hz on both rigs, so they sit on the LiDAR’s own clock. The handheld cameras record at 40 Hz and every fourth frame is kept; the robot’s cameras already record at 10 Hz and every frame is kept.

Camera frames are JPEG at quality 92. The source frames are uncompressed 8-bit greyscale.

The LiDAR’s per-point time is published as time_offset, in seconds relative to the sweep. The source records it as an absolute float64 timestamp, and the point cloud format packs fields as 32-bit floats, which quantize a value near 1.67e9 into steps of about two minutes.

The two rigs stamp a sweep at opposite ends of it, so the handheld PandarXT-32 gives time_offset in [0, +0.1] and the robot’s BPearl in [-0.1, 0]. Both are the firing time relative to that sweep’s own stamp; the sign is the sensor’s convention, not a defect.

The cameras are fisheye. Their four distortion coefficients are published under the equidistant model name they were calibrated with.

The robot names its cameras by mounting position rather than by number. They are published as /cam0 through /cam7 in the order the release’s own calibration lists them, and each episode carries camera_sources giving the source topic for each.

Reference positions are the sparse surveyed marks the release ships. There is no continuous reference trajectory in the 2023 data. The marks are timed, not just placed: each one is a position at a recorded instant during the run, which is what an estimated trajectory is scored against, so they are carried on /ground-truth.plot at those instants as well as drawn together in one scene. They measure position only, and sit in their own ground-truth frame, since the surveying instrument has no measured relation to the rig.

Every sensor publishes in its own frame, so each episode carries the rig’s frame tree on /tf, taken from the release’s own calibration: each camera and the LiDAR against the inertial frame, for whichever of the two rigs recorded the run. The rest of the source calibration directory, which is mostly the calibration reports, is not reproduced here.

License & attribution#

The source dataset is released under CC BY-NC-SA 3.0, and this conversion is distributed under the same license. Use is limited to non-commercial purposes, attribution is required, and adaptations must be distributed under the same or a compatible license.

Changes from the source: conversion from ROS 1 bags to the FiftyOne MCAP flavor, JPEG encoding of the camera frames, camera frame rate reduced to the LiDAR rate, the LiDAR per-point timestamp rebased to a per-sweep offset, and the reference positions and the calibrated frame tree carried as streams alongside the episodes.

Citation#

@misc{hilti2023challenge,
  title  = {Hilti SLAM Challenge 2023},
  author = {{Hilti Research}},
  year   = {2023},
  note   = {https://hilti-challenge.com/dataset-2023}
}