#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Hilti SLAM Challenge 2021 → FiftyOne (Native Multimodal MCAP)
The [Hilti SLAM Challenge 2021](https://hilti-challenge.com/dataset-2021) recordings, converted from ROS 1 bags to native multimodal MCAP episodes.
This is the first of the Hilti challenges and the one with the densest sensor suite. Twelve sequences were walked through offices, a lab, basements, a parking deck, campuses and construction sites, chosen for the things that break a SLAM system in the field: bare corridors with almost no geometric or visual structure, illumination that changes as the operator moves between floors and outdoors, stairs, and long loops.
The rig is a surveying pole carrying a Sevensense Alphasense head with five global-shutter cameras at 1440x1080 and an inertial unit, an Ouster OS0-64 spinning LiDAR, a Livox MID70 solid-state LiDAR and an ADIS16445 inertial unit. An FPGA locks the cameras to the ADIS unit, PTP locks the cameras to the two LiDARs, and every sensor’s clock agrees to within a millisecond.
Two LiDARs on one clock is what sets this year apart. The Ouster sweeps a full circle with a wide vertical field and the Livox stares forward through a narrow one, so the same scene arrives twice with completely different sampling.
Together they run to 57m 49s of recording across 8 locations.
## Installation
```bash
pip install fiftyone
```
## Usage
```python
import fiftyone as fo
import fiftyone.utils.huggingface as fouh
dataset = fouh.load_from_hub(
"Voxel51/Hilti-SLAM-Challenge-2021",
name="Hilti-SLAM-Challenge-2021",
persistent=True,
)
fo.launch_app(dataset)
```
The sequences with a continuous reference trajectory:
```python
view = dataset.match({"has_dense_ground_truth": True})
fo.launch_app(dataset, view=view)
```
## Reference trajectories
10 sequences carry sparse 3-DoF reference positions, measured with a total station while the rig was held still, and 2 carry a continuous 6-DoF trajectory from an Optitrack motion-capture system. The challenge quotes both as accurate to better than 5 mm. Each episode’s `reference` field names the frame the release solved it in, `prism` or `pole` for the total station and `imu` for the motion capture.
The sparse references are timed, not just placed: each one is a position at a recorded instant during the run, which is what an estimated trajectory is scored against. Every episode carries the reference on `/ground-truth.plot` at those instants, sparse or dense alike.
## What you get
Each episode carries:
- `/cam0` through `/cam4`, the five cameras at 1440x1080, as
`foxglove.CompressedImage`
- `/cam0-calibration` through `/cam4-calibration`, the intrinsics,
as `foxglove.CameraCalibration`
- `/lidar-points`, the Ouster sweeps, as `foxglove.PointCloud` with
`x`, `y`, `z`, `intensity`, `time_offset`, `reflectivity`,
`ring`, `ambient` and `range`
- `/livox-points`, the Livox sweeps, as `foxglove.PointCloud` with
`x`, `y`, `z`, `intensity`, `tag`, `line` and `time_offset`
- `/imu.plot`, `/imu-adis.plot` and `/os-imu.plot`, the Alphasense,
ADIS16445 and Ouster inertial units at 200 Hz, 800 Hz and 100 Hz
- `/ground-truth.plot`, the reference position at each instant the
release measured one
- `/ground-truth-pose`, the continuous reference trajectory, as
`foxglove.PoseInFrame`, on the sequences that have one
- `/ground-truth-control-points`, the surveyed positions drawn
together, as `foxglove.SceneUpdate`, on the sequences that have
them
- `/tf`, the rig’s frame tree, as `foxglove.FrameTransform`
- `/sequence`, naming the run
Across the whole set that comes to 173,150 camera frames, 34,696 Ouster sweeps holding 2.92 billion points, 34,696 Livox sweeps holding 0.29 billion, 3,871,119 inertial samples across the three units, 144,877 reference poses and 142 surveyed positions.
| Sequence | Scene | Frames | Ouster | Livox | Reference | Duration |
|---------------------------|-----------------------|----------|----------|---------|---------------------|------------|
| `Basement_1` | Basement | 5,634 | 1,130 | 1,130 | 5 marks (pole) | 1m 53s |
| `Basement_3` | Basement | 16,542 | 3,308 | 3,308 | 17 marks (prism) | 5m 31s |
| `Basement_4` | Basement | 17,500 | 3,503 | 3,503 | 17 marks (prism) | 5m 50s |
| `Campus_1` | Campus | 21,445 | 4,298 | 4,298 | 12 marks (prism) | 7m 10s |
| `Campus_2` | Campus | 18,669 | 3,749 | 3,748 | 19 marks (prism) | 6m 15s |
| `Construction_Site_1` | Construction Site | 9,953 | 1,995 | 1,996 | 10 marks (prism) | 3m 20s |
| `Construction_Site_2` | Construction Site | 19,948 | 3,992 | 3,991 | 19 marks (prism) | 6m 39s |
| `IC_Office_1` | IC Office | 10,001 | 2,004 | 2,003 | 13 marks (pole) | 3m 20s |
| `LAB_Survey_2` | LAB Survey | 6,778 | 1,357 | 1,357 | 134,156 poses (imu) | 2m 16s |
| `Office_Mitte_1` | Office Mitte | 13,170 | 2,641 | 2,642 | 9 marks (pole) | 4m 24s |
| `Parking_1` | Parking | 29,043 | 5,824 | 5,825 | 21 marks (pole) | 9m 42s |
| `UZH_Tracking_Area_Run_2` | UZH Tracking Area Run | 4,467 | 895 | 895 | 10,721 poses (imu) | 1m 29s |
Episodes carry the fields `sequence`, `scene`, `recorded`, `cameras`, `camera_stride`, `reference`, `ground_truth`, `has_dense_ground_truth`, `num_camera_frames`, `num_lidar_scans`, `num_lidar_points`, `num_livox_scans`, `num_livox_points`, `num_imu_samples`, `num_imu_adis_samples`, `num_os_imu_samples`, `num_ground_truth_poses`, `num_control_points` and `duration`.
## Notes on the conversion
The cameras and both LiDARs record at 10 Hz, so every frame is kept and every visual and 3D stream in an episode shares one clock. The stride is measured from each bag’s own message counts rather than assumed, and it comes out at 1 on all twelve; the 2022 and 2023 rigs run their cameras at 40 Hz against the same 10 Hz LiDAR, so the measurement is what keeps a rig’s own rate from being guessed at.
Camera frames are JPEG at quality 92. The source frames are uncompressed 8-bit greyscale. The cameras are fisheye, and their four distortion coefficients are published under the `equidistant` model name they were calibrated with.
Both LiDARs publish a per-point firing time as `time_offset`, in seconds relative to the sweep. The Ouster records it as nanoseconds since the sweep and the Livox as nanoseconds since the sweep start; the point cloud format packs fields as 32-bit floats, which cannot hold either at their native scale.
A scanner fires whether or not anything comes back, and both drivers write a miss as a point at the origin. Those are dropped, so a sweep holds returns only. The Ouster fires 131,072 times per sweep and between 46% and 89% of those returned depending on how open the scene was; the Livox averages 8,444 points per sweep.
The sparse references measure position only, so `/ground-truth.plot` carries `x`, `y` and `z` on those sequences and adds the quaternion on the two with a continuous trajectory.
Every sensor publishes in its own frame, so each episode carries the rig’s whole frame tree on `/tf`: the five cameras, both LiDARs and all three inertial units against the Alphasense inertial frame, and the `prism` and `pole_tip` targets the total station sighted. The surveying instruments have no measured relation to the rig, since recovering it is the task, so the reference streams sit in their own `ground-truth` frame rather than hanging off the tree.
The release’s `calibration.yaml` is republished at the repo root. It is the source of `/tf` and of the camera intrinsics, and it also carries the inertial biases and the measured gravity vector, which no stream schema has a place for. The CAD model of the rig is not reproduced here.
## License & attribution
The source dataset is released under
[CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/),
and this conversion is distributed under the same license. Use is limited to non-commercial purposes, attribution is required, and adaptations must be distributed under the same or a compatible license.
Changes from the source: conversion from ROS 1 bags to the FiftyOne MCAP flavor, JPEG encoding of the camera frames, the Livox custom message decoded into a point cloud, the LiDAR per-point timestamps rebased to a per-sweep offset, no-return points dropped, and the reference trajectories and the calibrated frame tree carried as streams alongside the episodes.
## Citation
```bibtex
@article{slamchallenge2021,
author = {Helmberger, Michael and Morin, Kristian and Berner, Beda and Kumar, Nitish and Wang, Danwei and Yue, Yufeng and Cioffi, Giovanni and Scaramuzza, Davide},
title = {The {Hilti} {SLAM} Challenge Dataset},
journal = {IEEE Robotics and Automation Letters},
year = {2022},
volume = {7},
number = {3},
pages = {7518--7525},
doi = {10.1109/LRA.2022.3183759}
}
```