#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Dataset Card for YUTO MMS Multimodal (MCAP)
A FiftyOne build of **YUTO MMS** (York University Teledyne Optech Mobile
Mapping System Dataset), a SLAM benchmark from the AUSM Lab at York
University. This build repackages the source dataset’s per-sequence raw
sensor folders as time-synchronized [MCAP](https://mcap.dev/) recordings for
FiftyOne’s native [multimodal dataset
support](https://docs.voxel51.com/user_guide/multimodal.html) (FiftyOne
1.19+). Each sample is one episode (one continuous drive), viewable in
FiftyOne’s tiled multimodal viewer with a synchronized panoramic-camera
image, a real-RGB-colorized LiDAR point cloud, a progressively-accumulated
world-frame map point cloud, GPS/INS fix, and IMU telemetry.
YUTO MMS ships 4 sequences (A/B/C/D, ~134 GB total on Zenodo); this build
currently includes **2 episodes (Sequences A and D)**, with B and C in
progress (see [Curation Rationale](#curation-rationale)). The source
dataset carries **no object/semantic/segmentation labels of any kind** —
it is raw sensor data plus a photogrammetric 6-DOF ground-truth camera
trajectory, intended for SLAM/odometry benchmarking, not perception tasks.
This repackaging does not add, remove, or alter any ground truth; every
coordinate transform applied (LiDAR→IMU, world→IMU) is derived from the
source dataset’s own real calibration files, never fabricated — see
[Dataset Creation](#dataset-creation) for exactly what was computed and
how it was verified.
This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 2 samples.
## Installation
If you haven’t already, install FiftyOne:
```bash
pip install -U fiftyone
```
## Usage
```python
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("Voxel51/yuto-mms-multimodal")
# Launch the App
session = fo.launch_app(dataset)
```
## Dataset Details
### Dataset Description
YUTO MMS was collected with Teledyne Optech’s **Maverick** mobile mapping
system (<9 kg, roof-rack mountable) across two collection days: Sequence A
at the Teledyne Optech headquarters parking lot in Vaughan, Ontario (June
21, 2019), and Sequences B, C, D around York University’s Keele Campus in
Toronto (August 12, 2020) — a main-campus loop (B), the adjacent York
Village residential neighborhood (C), and a residential-area loop (D). All
four were recorded under sunny weather. The Maverick rig combines a
**Velodyne HDL-32E LiDAR** (32-beam, 360° horizontal FOV, tilted 45° from
the camera’s optical axis, ~15.26 revolutions/s, ±3 cm absolute accuracy)
with a **Ladybug 5 panoramic camera** (six Sony ICX655 CCD sensors stitched
into one 8000×4000 equirectangular image, ~7.5 FPS) and a **NovAtel
SPAN-IGM-S1 GNSS/IMU** (STIM300 IMU + NovAtel OEM615 receiver, GPS/IMU at
125 Hz). Ground-truth camera trajectories were produced offline via
Teledyne Optech’s LMS Pro software (RTK GPS/IMU + photogrammetric bundle
adjustment), reporting cm-level accuracy (planar-fit RMS 0.010 m over
~7.9M calibration points, per the source paper’s Table 5).
- **Curated by:** AUSM Lab, Department of Earth and Space Science and
Engineering, Lassonde School of Engineering, York University — Yujia
Zhang, SeyedMostafa Ahmadi, Jungwon Kang, Zahra Arjmandi, Gunho Sohn
(corresponding author). This MCAP/FiftyOne multimodal repackaging
(episode authoring, dataset card) was prepared independently by Harpreet
Sahota.
- **Funded by:** Natural Sciences and Engineering Research Council of
Canada (NSERC), grant CRDPJ 537080-18.
- **Shared by:** Harpreet Sahota (this repackaging); the original YUTO MMS
dataset is shared by the AUSM Lab via
[ausmlab.github.io/yutomms](https://ausmlab.github.io/yutomms/) and
Zenodo.
- **Language(s):** N/A (sensor data — panoramic camera, LiDAR, GPS/INS,
IMU; no text).
- **License:** **Conflicting.** The website/README/tool page all state
**CC BY-NC-SA 4.0**. The live Zenodo record for every sequence checked
directly (A: 13203376; B: 13235145, 10576909, 10576911, 10577975; C:
10556128, 10556151, 10559929, 10574428, 10575667, 10575673, 10575677; D:
10560181, 10570781, 10572097 — via each record’s own `rel="license"`
HTTP header) instead serves **CC BY 4.0**. These have materially
different terms (NC/SA vs. none) — verify current status before
redistributing.
### Dataset Sources
- **Repository:** [ausmlab/yutomms](https://github.com/ausmlab/yutomms)
(site source); devkits:
[MaverickProjectLidar2Image](https://github.com/yujiazhang777/MaverickProjectLidar2Image)
(LiDAR→panorama projection, Matlab); `data_player` is linked from the
site as “code for generating rosbag” but is a generic MulRan-dataset ROS
player whose on-disk layout does not match YUTO MMS’s real files — not
YUTO-specific despite the link.
- **Paper:** Zhang, Y., Ahmadi, S., Kang, J., Arjmandi, Z., & Sohn, G.
(2024). YUTO MMS: A comprehensive SLAM dataset for urban mobile mapping
with tilted LiDAR and panoramic camera integration. *The International
Journal of Robotics Research*, 44(1), 3–21.
[doi:10.1177/02783649241261079](https://doi.org/10.1177/02783649241261079)
(paywalled on SAGE; free full text at
[PMC11685038](https://pmc.ncbi.nlm.nih.gov/articles/PMC11685038/)).
- **Demo:** [ausmlab.github.io/yutomms](https://ausmlab.github.io/yutomms/)
(official site, download page, and devkit links).
## Uses
### Direct Use
- Exercising/demoing FiftyOne’s multimodal MCAP support: synchronized
playback of a panoramic-camera image, a colorized 3D LiDAR point cloud, a
growing accumulated map, a GPS/INS map track, and IMU telemetry from a
real mobile-mapping-system recording.
- SLAM/odometry algorithm prototyping and evaluation against a real,
photogrammetrically-derived 6-DOF ground-truth camera trajectory
(`/tf`’s `world`→`camera` transform).
- Studying tilted-LiDAR-specific SLAM challenges — restricted/non-
rectangular camera-LiDAR overlap and sparse upper-beam coverage — which
is the entire motivation for the source dataset’s existence (per the
source paper’s Introduction).
- Inspecting real-RGB-colorized LiDAR points (each point sampled directly
from its nearest-in-time panorama via the dataset’s own LiDAR↔camera
boresight, not a synthetic colormap) and the `/lidar_map` topic’s
progressively-accumulated world-frame map, useful for visually verifying
a calibration chain end-to-end.
### Out-of-Scope Use
- Object detection, segmentation, classification, or any perception task —
**no such labels exist anywhere in the source dataset.** The source
paper’s entire “Dataset” section (images, LiDAR, GPS/IMU, calibration,
ground-truth trajectory) contains no annotation pipeline of any kind.
- Reproducing the source paper’s SLAM benchmark (Tables 7–8: ORB-SLAM2,
VINS-Mono, RPV-SLAM, HDPV-SLAM, LOAM, Google Cartographer,
PVL-Cartographer ATE/RTE/RRE) — this build repackages raw sensor streams
into MCAP episodes; it does not run or reproduce any SLAM system.
- Assuming a shipped per-image depth map exists. The source paper (Section
5.3.4) states one JPG depth map ships per panoramic image, but **no such
files exist** in the real downloaded/listed contents of any of the 4
sequences (verified directly — only `PanoramicImages/*.jpg` is present;
see [Data Collection and Processing](#data-collection-and-processing)).
- Assuming a shipped camera↔LiDAR synchronization file exists. The source
paper (Section 5.3.2) describes a dedicated sync `.txt`, but no such file
is present in any real downloaded sequence — `GroundTruth_*.txt`’s
`Filename` column is the only real join key between modalities.
- Sub-episode or per-frame sampling as independent, i.i.d. samples — each
sample is one continuous drive; camera, LiDAR, IMU, and GPS/INS are
asynchronous multi-rate streams of the same trajectory, not synchronized
single-instant frames.
## Dataset Structure
This is a flat (ungrouped) FiftyOne dataset with `media_type: "multimodal"`
and **2 samples**. Each sample is one **episode**, stored as one `.mcap`
file covering an entire sequence’s continuous drive; FiftyOne infers the
multimodal media type automatically from the `.mcap` extension. There is
no per-frame image or point-cloud sample — the episode is the sample unit,
and every stream inside it is decoded live by FiftyOne’s multimodal viewer.
The dataset carries no per-sample tags and `dataset.info` is empty (no
extra dataset-level metadata beyond the per-sample fields below).
### Episodes in this dataset
| `sequence_id` | `episode_tag` | `collection_site` | `route_length_m` | Duration | `message_count` | `channel_count` |
|-----------------|-----------------|--------------------------------------------------------|--------------------|------------|-------------------|-------------------|
| A | ep000 | Teledyne Optech headquarters, Vaughan, Ontario, Canada | 324 | 94.6s | 15,942 | 6 |
| D | ep000 | York University Keele Campus, Toronto | 3634 | 1296.0s | 348,726 | 6 |
Sequence A’s episode spans its full recording uniformly (all topics cover
the same ~94.6s). Sequence D’s episode is **not** uniform: `/camera`,
`/lidar`, and `/lidar_map` only span the first ~630s (10.5 min — the
Ladybug camera and LiDAR genuinely stopped recording partway through,
verified against the raw ground-truth and LiDAR-timestamp files), while
`/imu` and `/gps` continue for the full 1296s (21.6 min, verified against
the raw `.bag_imu.csv`/`.bag_ins.csv` files directly) — this is real
logged data, not a bug in this repackaging.
### Fields
| Field | FiftyOne type | Description |
|-------------------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `filepath` | `StringField` | Absolute path to the episode’s `.mcap` file — the sample’s multimodal media |
| `sequence_id` | `StringField` | Source sequence letter (`A`, `B`, `C`, or `D`), verbatim from the source dataset’s naming |
| `episode_tag` | `StringField` | Local identifier for the episode within a sequence (`ep000` — one episode per sequence, no splitting) |
| `duration_s` | `FloatField` | Episode duration in seconds, computed from the MCAP’s message-time span |
| `message_count` | `IntField` | Total MCAP message count across all channels in the episode |
| `channel_count` | `IntField` | Total MCAP channel (topic) count in the episode (6 for every episode) |
| `topics` | `ListField(StringField)` | Every MCAP topic present (see [MCAP topics](#mcap-topics-inside-each-episode) below) |
| `schemas` | `ListField(StringField)` | Every distinct Foxglove schema present in the episode |
| `has_image` | `BooleanField` | Whether the episode has an Image-tile-decodable stream (`foxglove.CompressedImage`) — `True` for every episode |
| `has_pointcloud` | `BooleanField` | Whether it has a decodable point-cloud stream for the 3D tile (`foxglove.PointCloud`) — `True` for every episode |
| `has_gps` | `BooleanField` | Whether it has a decodable GPS fix stream for the Map tile (`foxglove.LocationFix`) — `True` for every episode |
| `has_imu` | `BooleanField` | Whether it has an `/imu` topic (raw JSON, Plot-tile only) — `True` for every episode |
| `route_length_m` | `IntField` | Real driven route length in meters, from the source paper’s Table 3 |
| `collection_site` | `StringField` | Real-world collection site, verified per-sequence from the shipped ground-truth filename (`_HQ_` vs `_YU_campus_`) and the source paper — **not** assumed uniform across sequences (Sequence A differs from B/C/D) |
| `license_note` | `StringField` | Per-sequence license-conflict note (see [License](#dataset-description)) |
Standard FiftyOne bookkeeping fields (`id`, `tags`, `metadata`,
`created_at`, `last_modified_at`) are also present but not source-specific.
### MCAP topics (inside each episode)
| Topic | Schema | Tile | Notes |
|--------------|-----------------------------------|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `/camera` | `foxglove.CompressedImage` (jpeg) | Image | Panoramic image, downsampled 0.5× + re-encoded at JPEG quality 85 for embed size only — the colorization below always samples from the full-resolution source file, so this doesn’t touch point-cloud fidelity |
| `/lidar` | `foxglove.PointCloud` | 3D | Per-scan point cloud in the IMU frame: `x,y,z,intensity` (float32) + real `red,green,blue` (uint8) sampled from the nearest-in-time panorama via the LiDAR↔camera boresight — not a synthetic colormap. Points with no valid panorama projection fall back to greyscale-from-intensity (never triggered on Sequences A or D: 100% of points projected in-bounds on both) |
| `/lidar_map` | `foxglove.PointCloud` | 3D | Growing world-frame accumulation of `/lidar`, voxel-deduplicated (0.2 m for A, 0.75 m for D — tuned per-sequence, not shared) and logged at a fixed rate (2 Hz for A, 0.5 Hz for D) so scrubbing forward shows the map built up “as of” that time, with no fabricated points |
| `/imu` | generic JSON | Plot | Raw accelerometer (`ax,ay,az`, m/s²) + gyroscope (`gx,gy,gz`, rad/s) |
| `/gps` | `foxglove.LocationFix` | Map | INS lat/lon/altitude, already WGS-84 (no projection needed), plus `status`/`service`/`position_cov_type` as metadata key-value pairs |
| `/tf` | `foxglove.FrameTransform` | (enables 3D) | `world`→`camera` (real photogrammetric pose, from the ground-truth `.txt`) and `world`→`imu` (**derived**: the real camera pose composed with the real static camera↔IMU boresight from the sequence’s own calibration file — not independently measured, not fabricated) |
### Label types and why
**No FiftyOne sample-level label fields (`Detections`, `Classifications`,
etc.) are attached, and none should be** — the source dataset has no
object/semantic/segmentation annotations of any kind (verified against the
source paper, whose entire “Dataset” section covers only raw sensor
streams and the ground-truth trajectory). The closest thing to a “label” is
the per-image 6-DOF camera pose, which is carried as the `/tf` topic’s
`world`→`camera` transform (a continuous-time stream, not a fixed-length
list a sample-level field could hold) rather than as a sample field.
### Parsing decisions
- **One sample = one episode**, one `.mcap` per sequence’s entire
continuous drive — no per-frame or per-message splitting.
- **Calibration is parsed fresh from each sequence’s own shipped
`*_LCP.LCP`/`*_CCP.ccp` files**, never hardcoded from another sequence.
This was necessary, not theoretical: Sequence A’s calibration unit ID is
`6100012`; Sequences B/C/D’s is `6100018`, with boresight angles
differing by several degrees (different collection day/rig calibration).
Independently cross-checked: the source paper’s own Table 4 lists the
B/C/D LiDAR boresight rotation as `[179.579, -44.646, 0.601]` degrees —
matching this repackaging’s parsed value from Sequence D’s real `.LCP`
file to 3 decimal places.
- **`world`→`imu` is derived, not independently measured.** Only the
camera has a directly-measured absolute pose (from the ground-truth
`.txt`). `world`→`imu` is computed by composing that real camera pose
with the real static camera↔IMU boresight from the same calibration
file — verified by checking the derived IMU position stays a constant
lever-arm offset from the camera position across the whole sequence (a
rigid rig requires exactly that).
- **LiDAR filename-to-timestamp convention differs by sequence.** Sequence
A’s `.bin` filenames are the GPS-time-of-week in nanoseconds directly;
Sequence D’s are a sequential capture index, with real timestamps in a
sibling `LidarScanTimestamp.txt` (one line per scan). The convention is
auto-detected per sequence (checks for that file first), not assumed.
- **Depth maps and a camera↔LiDAR sync file, both described in the source
paper, are not included** — neither exists in any real downloaded
sequence (see [Out-of-Scope Use](#out-of-scope-use)); nothing was
fabricated to fill that gap.
## Dataset Creation
### Curation Rationale
The full YUTO MMS dataset is 4 sequences across 15 separate Zenodo records,
~134 GB total (A: 3.8 GB; B: 45.6 GB, 4 parts; C: 59.2 GB, 7 parts; D: 25.2
GB, 3 parts). Sequence A was authored first because, at under 4 GB, it
contains every modality and file type the other sequences do (per the
source devkit’s own directory diagram), making it sufficient to fully
characterize the format before committing to the larger sequences.
Sequence D was authored second. Sequences B and C are in progress — larger
sequences require re-deriving the `/lidar_map` voxel-size/update-rate
tuning per sequence (output size scales with route length/duration, not
just point count — reusing Sequence A’s tuning on Sequence D during
authoring produced an oversized intermediate file before this was caught
and fixed) rather than reusing a previous sequence’s settings.
### Source Data
#### Data Collection and Processing
Per the source paper: Sequence A was collected at the Teledyne Optech
headquarters parking lot in Vaughan, Ontario on June 21, 2019 (sunny, ~3.4
m/s average driving speed, no dynamic objects — a controlled lot, one small
324 m loop). Sequences B, C, and D were collected around York University’s
Keele Campus in Toronto on August 12, 2020 (sunny, ~6.1 m/s average driving
speed, dynamic objects present — cars, pedestrians, cyclists on public
roads): B is a 7035 m main-campus loop plus several smaller loops, C is a
9137 m route through the adjacent York Village residential neighborhood
with many medium loops, and D is a 3634 m route through open/residential
areas with a large loop plus smaller loops. Ground-truth camera
trajectories were produced offline by Teledyne Optech’s LMS Pro software:
RTK-corrected GPS/IMU plus a photogrammetric bundle block adjustment
(automated tie-point measurement, boresight/interior-orientation
corrections), reported at cm-level precision (planar-fit RMS 0.010 m,
std 0.014 m over ~7.9M points, per the source paper’s Table 5).
For this repackaging: raw per-sequence files (panoramic JPGs, raw `.bin`
LiDAR point dumps, GPS/INS and IMU CSVs, calibration `.LCP`/`.ccp` XML, and
the ground-truth `.txt`) were downloaded directly from Zenodo, parsed with
a from-scratch Python script (not the source devkit’s Matlab code, though
its projection/boresight math was replicated and cross-checked against
it), and packed into one `.mcap` file per episode using the
[foxglove-sdk](https://github.com/foxglove/foxglove-sdk). LiDAR points were
additionally colorized with real RGB (sampled from the nearest-in-time
panorama via the same LiDAR↔camera boresight the source devkit’s own
projection script uses) and accumulated into a world-frame map topic — see
[MCAP topics](#mcap-topics-inside-each-episode). No sensor data or ground
truth was synthesized, relabeled, or altered beyond the coordinate-frame
compositions documented in [Parsing decisions](#parsing-decisions).
#### Who are the source data producers?
AUSM Lab, Department of Earth and Space Science and Engineering, Lassonde
School of Engineering, York University — original data collection, sensor
calibration, and ground-truth trajectory post-processing (via Teledyne
Optech’s LMS Pro software).
### Annotations
#### Annotation process
**None.** No manual or automated object/semantic annotation exists
anywhere in this dataset. The only “ground truth” is the photogrammetrically
post-processed 6-DOF camera trajectory described above, which is a
sensor-fusion/bundle-adjustment output, not a manual annotation.
#### Who are the annotators?
N/A — no annotation process exists (see above).
#### Personal and Sensitive Information
The source dataset’s panoramic images were captured while driving through
public roads and a university campus during regular daytime activity
(dynamic objects present on B/C/D: cars, pedestrians, cyclists). No
statement about face/license-plate anonymization is made in the source
paper or website, unlike some comparable driving datasets — this
repackaging performs no additional processing, anonymization, or
redaction beyond what the AUSM Lab already released publicly.
## Citation
**BibTeX:**
```bibtex
@article{zhang2024yutomms,
author = {Yujia Zhang and SeyedMostafa Ahmadi and Jungwon Kang and
Zahra Arjmandi and Gunho Sohn},
title = {YUTO MMS: A comprehensive SLAM dataset for urban mobile
mapping with tilted LiDAR and panoramic camera integration},
journal = {The International Journal of Robotics Research},
volume = {44},
number = {1},
pages = {3--21},
year = {2024},
doi = {10.1177/02783649241261079}
}
```
**APA:**
Zhang, Y., Ahmadi, S., Kang, J., Arjmandi, Z., & Sohn, G. (2024). YUTO MMS:
A comprehensive SLAM dataset for urban mobile mapping with tilted LiDAR and
panoramic camera integration. *The International Journal of Robotics
Research*, 44(1), 3–21. https://doi.org/10.1177/02783649241261079
## More Information
This repository is an independently-curated MCAP/FiftyOne repackaging of
the official YUTO MMS dataset. It is not an official AUSM Lab/York
University artifact, and currently covers 2 of the source dataset’s 4
sequences (see [Curation Rationale](#curation-rationale)). For the full
dataset (all 4 sequences, the Matlab devkit, and the original directory
layout), see [ausmlab.github.io/yutomms](https://ausmlab.github.io/yutomms/)
and [github.com/ausmlab/yutomms](https://github.com/ausmlab/yutomms).
## Dataset Card Authors
Harpreet Sahota ([@harpreetsahota](https://huggingface.co/harpreetsahota))
— MCAP repackaging and this card. Original dataset producers are listed
under [Dataset Description](#dataset-description).
## Dataset Card Contact
Harpreet Sahota — https://huggingface.co/harpreetsahota