Note
This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.
Dataset Card for TUM RGB-D SLAM benchmark#
This is a FiftyOne dataset with 47 samples. Each sample is one recorded sequence from the benchmark, stored as a native multimodal MCAP episode.
The source is the TUM RGB-D SLAM Dataset and Benchmark from the Computer Vision Group at the Technical University of Munich. A Microsoft Kinect records color and depth at 640x480 and 30 Hz while an eight-camera motion-capture system tracks it at 100 Hz. The sequences are recorded with three Kinects, which the benchmark names freiburg1, freiburg2 and freiburg3, handheld, on a Pioneer robot, and over scenes built to test structure against texture, moving people and object reconstruction. This conversion carries the 47 sequences of the benchmark’s main table, every one with its ground truth.
Installation#
If you haven’t already, install FiftyOne:
pip install -U fiftyone
Usage#
import fiftyone as fo
import fiftyone.utils.huggingface as fouh
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = fouh.load_from_hub(
"Voxel51/TUM-RGBD",
name="TUM-RGBD",
persistent=True,
)
# Launch the App
session = fo.launch_app(dataset)
Dataset Details#
Dataset Description#
47 sequences, 9 recorded with freiburg1, 18 with freiburg2 and 20 with freiburg3, totalling 48m 12s of recording, 81,413 color frames and 80,683 depth frames.
Curated by: Computer Vision Group, Technical University of Munich (source release)
Funded by: [More Information Needed]
Shared by: Voxel51 (FiftyOne conversion)
Language(s): Not applicable (sensor data)
License: CC BY 4.0
Dataset Sources#
Repository: TUM RGB-D SLAM Dataset and Benchmark (source release); Voxel51/TUM-RGBD (this conversion)
Paper: A Benchmark for the Evaluation of RGB-D SLAM Systems (IROS 2012)
Demo: [More Information Needed]
Uses#
Direct Use#
RGB-D SLAM and visual odometry, evaluated against the motion-capture ground truth shipped with each sequence, and work with the registered color and depth images.
Out-of-Scope Use#
[More Information Needed]
Dataset Structure#
Topology
An ungrouped FiftyOne dataset with media_type="multimodal". One sample is one sequence, and its filepath is a .fo.mcap file. There are 47 samples (9 freiburg1, 18 freiburg2, 20 freiburg3) in six categories, no sample tags, and an empty dataset.info. The dataset has no FiftyOne label fields; every signal lives inside the MCAP file as a channel and is shown by the App’s multimodal viewer.
Sample fields
Field |
FiftyOne type |
Description |
|---|---|---|
|
built-in |
Standard FiftyOne sample fields |
|
|
Sequence name, e.g. |
|
|
The Kinect that recorded the sequence: |
|
|
The benchmark’s category for the sequence, e.g. |
|
|
Recording date |
|
|
Sequence length in seconds |
|
|
Color frames |
|
|
Depth frames |
|
|
Accelerometer samples; 0 on the freiburg3 sequences |
|
|
Motion-capture poses of the Kinect |
|
|
Trajectory length in metres, as listed on the benchmark’s download page for the sequence |
|
|
Share of depth pixels holding a measurement |
|
|
Mean of the measured depth pixels, in metres |
Episode contents
Each MCAP episode contains these channels:
Channel |
Schema or content |
|---|---|
|
Color images, |
|
Depth images, |
|
Intrinsics beside each image channel, |
|
The Kinect’s accelerometer, on the freiburg1 and freiburg2 sequences only |
|
Motion-capture pose of the Kinect, |
|
Position of the ground-truth pose as a plot |
|
Names the sequence and its category |
Categories
Category |
Sequences |
Duration |
Color frames |
|---|---|---|---|
Testing and Debugging |
|
4m 43s |
8,480 |
Handheld SLAM |
|
12m 48s |
22,864 |
Robot SLAM |
|
7m 34s |
8,803 |
Structure vs. Texture |
|
4m 19s |
7,679 |
Dynamic Objects |
|
6m 33s |
11,544 |
3D Object Reconstruction |
|
12m 16s |
22,043 |
Parsing decisions
Why MCAP: the dataset is one multimodal sample per sequence, so the color and depth images, the accelerometer and the ground-truth poses play back on one shared clock instead of being split into per-frame samples.
Depth: the depth images are carried as the benchmark ships them, 16-bit PNG in which 5000 is one metre and 0 is no measurement, pre-registered to the color image so that their pixels correspond one to one.
Video encoding: the color images are re-encoded to Annex-B H.264 without B-frames, one access unit per frame, on each image’s own timestamp.
Calibration: both calibration channels carry the benchmark’s published intrinsics for the Kinect that recorded the sequence, which the registered depth shares. The freiburg3 images are already undistorted, so their distortion is zero.
Ground truth: the ground truth is the benchmark’s
groundtruth.txt, the pose of the Kinect in the motion-capture frame, carried as/ground-truthin a frame namedworld.Accelerometer: the accelerometer is carried as recorded. The benchmark’s freiburg3 archives hold no accelerometer readings, so those episodes have none.
Left out: the validation sequences, whose ground truth the benchmark withholds, and the calibration recordings are not included.
Dataset Creation#
Curation Rationale#
[More Information Needed]
Source Data#
Data Collection and Processing#
A Microsoft Kinect records color and depth at 640x480 and 30 Hz while an eight-camera motion-capture system tracks it at 100 Hz. Three Kinects, named freiburg1, freiburg2 and freiburg3 by the benchmark, record the sequences, handheld, on a Pioneer robot, and over scenes built to test structure against texture, moving people and object reconstruction. The FiftyOne conversion reads the benchmark’s image archives and writes one MCAP episode per sequence; see Parsing decisions above for the changes made.
Who are the source data producers?#
The Computer Vision Group at the Technical University of Munich.
Annotations#
Annotation process#
The dataset has no human annotations. The ground truth in each sequence is the pose of the Kinect recorded by the eight-camera motion-capture system at 100 Hz, as listed in the benchmark’s groundtruth.txt.
Who are the annotators?#
Not applicable.
Personal and Sensitive Information#
The Dynamic Objects category contains sequences with people moving through the scene. [More Information Needed]
Citation#
The benchmark asks that work using the data cite:
BibTeX:
@InProceedings{sturm12iros,
author = {J. Sturm and N. Engelhard and F. Endres and W. Burgard and D. Cremers},
title = "A Benchmark for the Evaluation of RGB-D SLAM Systems",
booktitle = "Proc. of the International Conference on Intelligent Robot Systems (IROS)",
year = "2012",
month = "Oct."
}
APA:
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012, October). A benchmark for the evaluation of RGB-D SLAM systems. In Proc. of the International Conference on Intelligent Robot Systems (IROS).
More Information#
The TUM RGB-D benchmark’s data is distributed under the Creative Commons Attribution 4.0 International license (CC-BY-4.0), and this conversion is distributed under the same license.
Changes from the source: conversion from the benchmark’s image archives to the FiftyOne MCAP flavor, H.264 encoding of the color images, the benchmark’s published intrinsics carried as camera calibration, and the validation and calibration recordings left out.
Dataset Card Contact#
[More Information Needed]