Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Dataset Card for GrandTour sample#

preview

This is a FiftyOne dataset with 3 samples. Each sample is one mission of a legged robot, stored as a native multimodal MCAP episode.

The source is GrandTour, the legged robotics dataset from the Robotic Systems Lab at ETH Zurich. An ANYmal D quadruped carries the Boxi sensor payload through cities, buildings, forests, mountains and ice: HDR and depth cameras, a Hesai LiDAR, a tactical-grade inertial unit, a GNSS/INS receiver and the robot’s own joint sensing, with a total station tracking a prism on the payload for reference. The full release holds 49 missions; this sample carries 3, chosen to contrast: ALB-3, Albisgütli - Forest 3; ETH-2, ETH - Mainbuilding Indoor; SNOW-1, Jungfraujoch - Snow 1.

Installation#

If you haven’t already, install FiftyOne:

pip install -U fiftyone

Usage#

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = fouh.load_from_hub(
    "Voxel51/GrandTour-Sample",
    name="GrandTour-Sample",
    persistent=True,
)

# Launch the App
session = fo.launch_app(dataset)

Dataset Details#

Dataset Description#

3 missions totalling 13m 33s of walking, 24,390 camera frames, 12,701 depth frames and 8,363 LiDAR scans holding 433.5 million points.

Curated by: Robotic Systems Lab, ETH Zurich (source release)
Funded by: [More Information Needed]
Shared by: Voxel51 (FiftyOne conversion)
Language(s): Not applicable (sensor data)
License: MIT

Dataset Sources#

Repository: leggedrobotics/grand_tour_dataset (source release); Voxel51/GrandTour-Sample (this conversion)
Paper: GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation (arXiv:2602.18164)
Demo: [More Information Needed]

Uses#

Direct Use#

Multi-modal perception and state estimation for legged robots, per the title of the source paper. Each mission pairs three HDR cameras, a depth camera, LiDAR scans, an inertial unit, joint sensing and three independent odometry or GNSS/INS pose estimates, with the total station’s prism positions as a reference measurement. The three missions contrast a forest slope, an indoor hall without GNSS and a snow field.

Out-of-Scope Use#

[More Information Needed]

Dataset Structure#

Topology

An ungrouped FiftyOne dataset with media_type="multimodal". One sample is one mission, and its filepath is a self-contained .fo.mcap file under data/. There are 3 samples (ALB-3, ETH-2, SNOW-1), no sample tags, and an empty dataset.info. The split field reads Train for all three samples. The dataset has no FiftyOne label fields; every signal lives inside the MCAP file as a channel and is shown by the App’s multimodal viewer.

Sample fields

Field	FiftyOne type	Description
`id`, `filepath`, `tags`, `metadata`, `created_at`, `last_modified_at`	built-in	Standard FiftyOne sample fields
`mission`	`StringField`	Mission code, e.g. `ALB-3`
`name`	`StringField`	Mission name, e.g. `Albisgütli - Forest 3`
`location`	`StringField`	Where the mission was recorded
`description`	`StringField`	Free-text description of the walk, from the release’s mission index
`labels`	`ListField(StringField)`	Terrain, environment and weather labels from the release’s mission index, e.g. `Forest`, `Outdoor`, `Snow`
`split`	`StringField`	Split named by the release’s mission index; `Train` for all three samples
`gnss`	`StringField`	GNSS status named by the release’s mission index (`Working` or `No`)
`recorded`	`StringField`	Recording date
`duration`	`FloatField`	Mission length in seconds
`num_camera_frames`	`IntField`	Camera frames
`num_depth_frames`	`IntField`	Depth frames
`num_lidar_scans`	`IntField`	LiDAR scans
`num_lidar_points`	`IntField`	Points across all LiDAR scans
`num_imu_samples`	`IntField`	Inertial samples
`num_gnss_fixes`	`IntField`	GNSS/INS fixes
`num_prism_positions`	`IntField`	Prism positions measured by the total station
`lidar_odometry_path_m`	`FloatField`	Length of the LiDAR-inertial odometry path in metres
`release_notes`	`StringField`	Notes from the release about the mission; empty unless the release has one

Episode contents

Each MCAP episode contains these channels:

Channel	Schema or content
`/hdr-front`, `/hdr-left`, `/hdr-right`	The three HDR cameras at 1920x1280, `foxglove.CompressedImage` (JPEG)
`/depth-front`	The upper front depth camera at 848x480, `foxglove.CompressedImage` (16-bit PNG, millimetres)
`<camera>-calibration`	A calibration topic beside each camera, `foxglove.CameraCalibration`
`/lidar-points`	The motion-compensated Hesai scans, `foxglove.PointCloud` with `x`, `y`, `z`, `intensity`, `ring` and `time_offset`
`/imu.plot`	The STIM320 inertial unit
`/joint-positions.plot`, `/joint-velocities.plot`, `/joint-torques.plot`	The twelve leg joints
`/legged-odometry`, `/lidar-odometry`, `/gnss-ins-odometry`	The robot’s own estimate, the LiDAR-inertial estimate and the GNSS/INS solution, `foxglove.PoseInFrame`, each with its position on a `.plot` channel
`/gnss`	The GNSS/INS fixes, `foxglove.LocationFix`
`/prism.plot`	The prism position the total station measured
`/command.plot`	The operator’s velocity command
`/battery.plot`	The robot’s battery
`/tf`	The static transforms of the robot and its payload, each sensor’s pose in its parent frame, `foxglove.FrameTransform`
`/mission`	Names the mission

Missions

Mission	Where	Duration	Camera frames	Depth frames	LiDAR scans	GNSS fixes	Path
`ALB-3`	Uetliberg, Albisguetli	3m 33s	6,390	3,374	2,217	44,601	78.0 m
`ETH-2`	ETH Zurich Main Building	7m 02s	12,678	6,508	4,306	0	210.6 m
`SNOW-1`	Jungfraujoch - Moenchsjoch Fenced	2m 57s	5,322	2,819	1,840	37,201	36.5 m

Mission	Recorded	GNSS	LiDAR points	Inertial samples	Prism positions
`ALB-3`	2024-11-15	Working	117,036,703	104,689	3,317
`ETH-2`	2024-10-01	No	234,590,162	209,141	4,684
`SNOW-1`	2024-11-03	Working	81,859,032	86,045	1,201

Parsing decisions

Why MCAP: the dataset is one multimodal sample per mission, so the cameras, LiDAR, inertial unit, joint streams, odometry and plots play back on one shared clock instead of being split into per-frame samples.
Source layout: the release ships each mission as one zarr group per sensor topic, tarred, with each camera’s frames in a tar of their own.
Camera frames: carried as the release ships them, JPEG for colour and 16-bit PNG for depth, on the timestamps the zarr groups record. Every camera frame the release indexes is carried, and every LiDAR scan that holds points.
Transforms: the release stores each static transform as the map from its base frame into the sensor’s frame. /tf carries the inverse, each sensor’s pose in its parent frame, which is what foxglove.FrameTransform describes.
Depth camera orientation: the upper front depth camera is mounted upside down, so its frames arrive turned half a turn, as its transform records.
LiDAR: the scans are the release’s motion-compensated ones, with the padding of its fixed-size arrays dropped. The per-point time is published as time_offset, in seconds from the scan’s own stamp, since the release records an absolute time that a 32-bit float would quantize into steps of minutes.
Release notes: for ETH-2 the release notes that the VLP-16 ANYmal-D LiDAR and the lower_front and lower_rear RealSense cameras are not present, and that its HDR timestamps are imprecise because they use the GMSL2 arrival time instead of the hardware timestamp (the same applies to ETH-1 and ETH-3 in the full release). These notes are carried in the release_notes field.
Mission metadata: the per-mission fields mission, name, location, description, labels, split, gnss, recorded and release_notes are taken from the release’s own mission index where it describes them.
Left out: this sample carries the three HDR cameras and the upper front depth camera. The release’s five Alphasense cameras, its ZED 2i stereo pair and depth, its five other depth cameras, its Livox and Velodyne LiDARs, its other inertial units, the real-time GNSS solution, the raw GNSS observations, the point cloud maps and the Gaussian splats are not reproduced, and are available from the release itself.

Dataset Creation#

Curation Rationale#

This sample carries 3 of the release’s 49 missions, chosen to contrast: a forest slope on the Uetliberg, an indoor hall at ETH Zurich and a snow field on the Jungfraujoch. The rationale for the source release itself is [More Information Needed].

Source Data#

Data Collection and Processing#

Recorded with an ANYmal D quadruped carrying the Boxi sensor payload, with a total station tracking a prism on the payload for reference. The FiftyOne conversion reads the release’s zarr groups and image archives and writes one MCAP episode per mission; see Parsing decisions above for the changes made.

Who are the source data producers?#

The GrandTour authors at the Robotic Systems Lab, ETH Zurich.

Annotations#

Annotation process#

The dataset has no FiftyOne label fields. The mission-level fields (description, labels, split, gnss, release_notes) come from the release’s own mission index; how that index was produced is [More Information Needed]. The prism positions measured by the total station and the three pose estimates (legged, LiDAR-inertial and GNSS/INS) are carried as MCAP channels.

Who are the annotators?#

[More Information Needed]

Personal and Sensitive Information#

The ETH-2 mission is labeled People and its description notes multiple people around, so its camera frames may show people. [More Information Needed] on anonymization.

Citation#

The following references accompany the dataset:

BibTeX:

@article{frey2026grandtour,
  title={GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation},
  author={Frey, Jonas and Tuna, Turcan and Fu, Frank and Patterson, Katharine and Xu, Tianao and Fallon, Maurice and Cadena, Cesar and Hutter, Marco},
  journal={arXiv preprint arXiv:2602.18164},
  year={2026}
}

@INPROCEEDINGS{Tuna-Frey-Fu-RSS-25,
    AUTHOR    = {Jonas Frey AND Turcan Tuna AND Lanke Frank Tarimo Fu AND Cedric Weibel AND Katharine Patterson AND Benjamin Krummenacher AND Matthias Müller AND Julian Nubert AND Maurice Fallon AND Cesar Cadena AND Marco Hutter},
    TITLE     = {{Boxi: Design Decisions in the Context of Algorithmic Performance for Robotics}},
    BOOKTITLE = {Proceedings of Robotics: Science and Systems},
    YEAR      = {2025},
    ADDRESS   = {Los Angeles, United States},
    MONTH     = {June}
}

APA:

Frey, J., Tuna, T., Fu, F., Patterson, K., Xu, T., Fallon, M., Cadena, C., & Hutter, M. (2026). GrandTour: A legged robotics dataset in the wild for multi-modal perception and state estimation. arXiv preprint arXiv:2602.18164.

Frey, J., Tuna, T., Fu, L. F. T., Weibel, C., Patterson, K., Krummenacher, B., Müller, M., Nubert, J., Fallon, M., Cadena, C., & Hutter, M. (2025). Boxi: Design decisions in the context of algorithmic performance for robotics. In Proceedings of Robotics: Science and Systems. Los Angeles, United States.

More Information#

The source release declares the MIT License on its Hugging Face card without naming a copyright holder. This conversion is distributed under the same license, with the work credited to the GrandTour authors at the Robotic Systems Lab, ETH Zurich:

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Changes from the source: 3 of the release’s missions, converted from its zarr and image archives to the FiftyOne MCAP flavor, with the LiDAR padding dropped, the per-point time rebased to each scan, the release’s camera intrinsics carried as calibration streams, and its static transforms inverted into each sensor’s pose in its parent frame.

Dataset Card Authors#

[More Information Needed]

Dataset Card Contact#

[More Information Needed]