#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Hilti SLAM Challenge 2023 → FiftyOne (Native Multimodal MCAP)
The [Hilti SLAM Challenge 2023](https://hilti-challenge.com/dataset-2023) recordings, converted from ROS 1 bags to native multimodal MCAP episodes.
The 2023 challenge extends the benchmark to multi-session and multi-platform mapping. Every run was walked or driven through an active construction site, and the runs at a site overlap, so a sequence can be solved on its own or several can be combined into one map.
Two rigs recorded it. The handheld platform is the Phasma-style suite carried over from the 2022 challenge: five synchronized global-shutter cameras at 720x540, a Hesai PandarXT-32 LiDAR and an inertial unit, with the cameras locked together by an FPGA and the cameras and LiDAR by PTP to within about a millisecond. The robot is a tracked drilling platform carrying four OAK-D stereo pairs, so eight camera streams at 1280x800, a RoboSense BPearl hemispherical LiDAR and an Xsens MTi-670.
A surveyor measured reference positions along every run.
> **Photosensitivity warning.** `site2_handheld_4`, `site2_handheld_5`, `site2_handheld_6` carry rapidly flashing lights in their camera streams. Every episode carries `has_flashing_lights` so they can be excluded before the App is opened.
## Installation
```bash
pip install fiftyone
```
## Usage
```python
import fiftyone as fo
import fiftyone.utils.huggingface as fouh
dataset = fouh.load_from_hub(
"Voxel51/Hilti-SLAM-Challenge-2023",
name="Hilti-SLAM-Challenge-2023",
persistent=True,
)
fo.launch_app(dataset)
```
The runs recorded from the robot rather than by hand:
```python
view = dataset.match({"platform": "robot"})
fo.launch_app(dataset, view=view)
```
Everything except the runs with flashing lights:
```python
view = dataset.match({"has_flashing_lights": False})
```
## What you get
15 runs, 55m 40s of recording. 12 from the handheld rig and 3 from the robot, across 5 at site1, 6 at site2, 4 at site3.
Each episode carries:
- `/cam0` onward, the cameras, as `foxglove.CompressedImage`. Five
on the handheld rig, eight on the robot
- `/cam0-calibration` onward, the intrinsics for each camera, as
`foxglove.CameraCalibration`
- `/lidar-points`, the LiDAR at its native rate, as
`foxglove.PointCloud`
- `/imu.plot`, three-axis acceleration and angular rate
- `/ground-truth.plot`, the surveyed reference position at each
instant the release measured one
- `/ground-truth-control-points`, those positions drawn together,
as `foxglove.SceneUpdate`
- `/tf`, the rig’s frame tree, as `foxglove.FrameTransform`
- `/sequence`, naming the run
Across the whole set that comes to 207,825 camera frames, 33,401 LiDAR sweeps holding 1.98 billion points, 1,061,139 inertial samples and 63 surveyed reference positions.
| Sequence | Site | Platform | Cameras | Frames | Sweeps | Duration | Marks |
|--------------------|--------|------------|-----------|----------|----------|------------|---------|
| `site1_handheld_1` | site1 | handheld | 5 | 10,240 | 2,047 | 3m 25s | 4 |
| `site1_handheld_2` | site1 | handheld | 5 | 8,360 | 1,672 | 2m 47s | 4 |
| `site1_handheld_3` | site1 | handheld | 5 | 8,536 | 1,707 | 2m 51s | 4 |
| `site1_handheld_4` | site1 | handheld | 5 | 14,775 | 2,955 | 4m 55s | 3 |
| `site1_handheld_5` | site1 | handheld | 5 | 7,970 | 1,594 | 2m 39s | 3 |
| `site2_handheld_4` | site2 | handheld | 5 | 4,665 | 932 | 1m 33s | 2 |
| `site2_handheld_5` | site2 | handheld | 5 | 10,280 | 2,056 | 3m 26s | 4 |
| `site2_handheld_6` | site2 | handheld | 5 | 6,870 | 1,373 | 2m 17s | 2 |
| `site2_robot_1` | site2 | robot | 8 | 55,954 | 6,996 | 11m 40s | 7 |
| `site2_robot_2` | site2 | robot | 8 | 24,472 | 3,059 | 5m 06s | 3 |
| `site2_robot_3` | site2 | robot | 8 | 28,730 | 3,591 | 5m 59s | 5 |
| `site3_handheld_1` | site3 | handheld | 5 | 4,864 | 972 | 1m 37s | 4 |
| `site3_handheld_2` | site3 | handheld | 5 | 7,360 | 1,482 | 2m 28s | 6 |
| `site3_handheld_3` | site3 | handheld | 5 | 9,399 | 1,896 | 3m 10s | 8 |
| `site3_handheld_4` | site3 | handheld | 5 | 5,350 | 1,069 | 1m 47s | 4 |
Episodes carry the fields `sequence`, `site`, `platform`, `recorded`, `scene`, `cameras`, `camera_sources`, `lidar`, `imu`, `is_challenge_sequence`, `has_flashing_lights`, `num_camera_frames`, `num_lidar_scans`, `num_lidar_points`, `num_imu_samples`, `num_control_points`, `control_point_names` and `duration`.
## Notes on the conversion
Camera frames are published at 10 Hz on both rigs, so they sit on the LiDAR’s own clock. The handheld cameras record at 40 Hz and every fourth frame is kept; the robot’s cameras already record at 10 Hz and every frame is kept.
Camera frames are JPEG at quality 92. The source frames are uncompressed 8-bit greyscale.
The LiDAR’s per-point time is published as `time_offset`, in seconds relative to the sweep. The source records it as an absolute float64 timestamp, and the point cloud format packs fields as 32-bit floats, which quantize a value near 1.67e9 into steps of about two minutes.
The two rigs stamp a sweep at opposite ends of it, so the handheld PandarXT-32 gives `time_offset` in `[0, +0.1]` and the robot’s BPearl in `[-0.1, 0]`. Both are the firing time relative to that sweep’s own stamp; the sign is the sensor’s convention, not a defect.
The cameras are fisheye. Their four distortion coefficients are published under the `equidistant` model name they were calibrated with.
The robot names its cameras by mounting position rather than by number. They are published as `/cam0` through `/cam7` in the order the release’s own calibration lists them, and each episode carries `camera_sources` giving the source topic for each.
Reference positions are the sparse surveyed marks the release ships. There is no continuous reference trajectory in the 2023 data. The marks are timed, not just placed: each one is a position at a recorded instant during the run, which is what an estimated trajectory is scored against, so they are carried on `/ground-truth.plot` at those instants as well as drawn together in one scene. They measure position only, and sit in their own `ground-truth` frame, since the surveying instrument has no measured relation to the rig.
Every sensor publishes in its own frame, so each episode carries the rig’s frame tree on `/tf`, taken from the release’s own calibration: each camera and the LiDAR against the inertial frame, for whichever of the two rigs recorded the run. The rest of the source `calibration` directory, which is mostly the calibration reports, is not reproduced here.
## License & attribution
The source dataset is released under
[CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/),
and this conversion is distributed under the same license. Use is limited to non-commercial purposes, attribution is required, and adaptations must be distributed under the same or a compatible license.
Changes from the source: conversion from ROS 1 bags to the FiftyOne MCAP flavor, JPEG encoding of the camera frames, camera frame rate reduced to the LiDAR rate, the LiDAR per-point timestamp rebased to a per-sweep offset, and the reference positions and the calibrated frame tree carried as streams alongside the episodes.
## Citation
```bibtex
@misc{hilti2023challenge,
title = {Hilti SLAM Challenge 2023},
author = {{Hilti Research}},
year = {2023},
note = {https://hilti-challenge.com/dataset-2023}
}
```