#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Dataset Card for CANOE Multimodal (MCAP)
A FiftyOne build of **CANOE** (Canadian Aquatic Navigation for Observation of
the Environment), a multi-sensor marine navigation dataset collected by
ASRL (UTIAS) on an uncrewed surface vessel (USV). This build repackages 4
of CANOE’s 8 public sequences as time-synchronized [MCAP](https://mcap.dev/)
recordings for FiftyOne’s native
[multimodal dataset support](https://docs.voxel51.com/user_guide/multimodal.html)
(FiftyOne 1.19+). Each sample is one episode, viewable in FiftyOne’s tiled
multimodal viewer with synchronized stereo camera, radar, sonar, lidar point
cloud, GPS/INS track, and IMU/motor/velocity telemetry.
CANOE itself is a raw multi-sensor dataset for benchmarking
odometry/localization algorithms in aquatic environments — it ships no
object/semantic annotations, only dense ground-truth 6-DoF pose from
post-processed GNSS/INS. This repackaging does not add or alter any
ground truth; see [Dataset Creation](#dataset-creation) for exactly what
was kept, converted, and left out.
This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 4 samples.
## Installation
If you haven’t already, install FiftyOne:
```bash
pip install -U fiftyone
```
## Usage
```python
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("harpreetsahota/canoe-multimodal")
# Launch the App
session = fo.launch_app(dataset)
```
## Dataset Details
### Dataset Description
CANOE is a multi-sensor marine navigation dataset collected on an
uncrewed surface vessel (USV) traversing a lake and a reservoir in
Ontario, Canada, over three days (8 public sequences, 14+ hours, 50 km).
Sensor suite: a 360° Navtech RAS6 radar (dual range modes), a 128-beam
Ouster OS1 lidar with integrated IMU, a Teledyne Bumblebee stereo camera,
an Oculus M3000d imaging sonar, dual Starfish 990F sidescan sonar, motor
power sensors, and a NovAtel OEM7 GNSS/INS post-processed to ~2-4 cm RMS
position accuracy. The intended purpose is benchmarking odometry and
localization algorithms for aquatic navigation; there are no
object/semantic annotations, only raw sensor streams plus dense
ground-truth pose. This FiftyOne build covers 4 of the 8 public
sequences, chosen for sensor/environment diversity (see
[Curation Rationale](#curation-rationale)).
- **Curated by:** Autonomous Space Robotics Laboratory (ASRL), University
of Toronto Institute for Aerospace Studies (UTIAS) — original CANOE
data collection, sensor calibration, and ground-truth pose
post-processing. This MCAP/FiftyOne multimodal repackaging (episode
authoring, dataset card) was prepared independently by Harpreet Sahota.
- **Funded by:** [More Information Needed]
- **Shared by:** Harpreet Sahota (this repackaging); the original CANOE
dataset is shared by ASRL via the [AWS Open Data
Registry](https://registry.opendata.aws/canoe/) (`s3://canoe-data`) and
https://www.canoedataset.com.
- **Language(s):** N/A (sensor data — radar, lidar, imagery, sonar, GPS,
IMU; no text).
- **License:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/),
inherited from the source CANOE release.
### Dataset Sources
- **Repository (devkit):** https://github.com/utiasASRL/pycanoe
- **Paper:** [More Information Needed] — no published paper is referenced
by the AWS Open Data Registry listing, the devkit, or
https://www.canoedataset.com at the time of this card; see
[Citation](#citation) for the registry’s suggested access citation.
- **Demo:** https://www.canoedataset.com (official docs/download/leaderboard
site) and https://registry.opendata.aws/canoe/ (AWS Open Data listing)
## Uses
### Direct Use
- Exercising and demoing FiftyOne’s multimodal MCAP support: synchronized
playback of stereo camera, decoded radar/sonar pseudo-imagery, 3D lidar
point cloud, GPS/map track, and IMU/motor/velocity telemetry across real
aquatic robotics recordings.
- Browsing/inspecting USV traversals of a lake (short and long routes) and
a reservoir, including both radar operating modes (1000 m long-range
and 150 m Doppler).
- Prototyping odometry/localization analyses against the dense
ground-truth pose (`/tf`, `/odom`, `/gps`, `/velocity`) provided at each
sensor’s native timestamp.
### Out-of-Scope Use
- Reproducing CANOE’s official odometry/localization benchmark (the SE(2)/SE(3)
Lat/Long/Yaw RMSE leaderboard on canoedataset.com) — that protocol
operates over full, un-windowed sequences; this build covers only 4 of
the 8 public sequences, 3 of which are short fixed-duration windows
(30-60s), not full traversals.
- Any task requiring object/semantic annotations — CANOE ships none
(purely raw sensor streams plus ground-truth pose).
- Commercial or non-commercial use inconsistent with the CC BY 4.0
attribution requirement.
## Dataset Structure
This is a flat (ungrouped) FiftyOne dataset with `media_type: "multimodal"`
and **4 samples**. Each sample is one **episode**, stored as one `.mcap`
file; FiftyOne infers the multimodal media type automatically from the
`.mcap` extension. There are no separate per-frame image or point-cloud
samples — the episode is the sample unit, and every stream inside it
(camera, radar, sonar, lidar, GPS, IMU, motor) is decoded live by
FiftyOne’s multimodal viewer.
The dataset carries no per-sample tags and `dataset.info` is empty (no
extra dataset-level metadata beyond the per-sample fields below).
### Episodes in this dataset
| `sequence_id` | `route` | Coverage | `radar_max_range_m` |
|--------------------------|---------------------|--------------------------------------------------------------------------|-----------------------|
| `canoe-2025-08-21-19-16` | Sample | Full sequence (~72s active sensors; GPS/INS logs continuously for ~581s) | 1000 (long-range) |
| `canoe-2025-08-20-16-07` | Day 1 – Lake Short | 30s window | 1000 (long-range) |
| `canoe-2025-08-20-18-07` | Day 1 – Lake Long | 60s window | 1000 (long-range) |
| `canoe-2025-08-28-14-13` | Reservoir Doppler 1 | 60s window | 150 (Doppler) |
`route` gives the human-readable route name from canoedataset.com’s
Download tab. The **Sample** sequence is the only one downloaded and
authored in full — at 8.9 GB it is small enough to skip windowing; it is
also the only episode where the GNSS/INS trace outlasts the active
camera/lidar/radar/sonar capture window (continuous INS logging vs. a
shorter active-sensor window), which is expected, not a data error. The
other 3 episodes are fixed-duration windows starting at the earliest
sensor timestamp for that sequence. 4 of the 8 public CANOE sequences are
not (yet) included in this build: Day 2 – Lake Short, Day 2 – Lake Long,
Reservoir Doppler 2, and Reservoir 3.
### Fields
| Field | FiftyOne type | Description |
|---------------------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `filepath` | `StringField` | Absolute path to the episode’s merged `.mcap` file — the sample’s multimodal media |
| `tags` | `ListField(StringField)` | Empty for every sample in this build |
| `sequence_id` | `StringField` | Raw CANOE sequence ID (`canoe-YYYY-MM-DD-HH-MM`), verbatim from the S3 folder name |
| `route` | `StringField` | Human-readable route label, from the Download tab at https://www.canoedataset.com/#/download |
| `radar_max_range_m` | `IntField` | Radar operating mode for this sequence’s data: `1000` (long-range) or `150` (Doppler), from `calib/radar_config.yaml` |
| `duration_s` | `FloatField` | Episode duration in seconds, computed from the merged MCAP’s message-time span |
| `message_count` | `IntField` | Total MCAP message count across all channels in the episode |
| `channel_count` | `IntField` | Total MCAP channel (topic) count in the episode (13 for every episode in this build) |
| `topics` | `ListField(StringField)` | Every MCAP topic present: `/cam_left`, `/cam_left/calibration`, `/cam_left_lidar_overlay`, `/cam_right`, `/gps`, `/imu`, `/lidar`, `/motor`, `/odom`, `/radar`, `/sonar`, `/tf`, `/velocity` |
| `schemas` | `ListField(StringField)` | Every distinct Foxglove schema present: `foxglove.CameraCalibration`, `foxglove.CompressedImage`, `foxglove.FrameTransform`, `foxglove.LocationFix`, `foxglove.Odometry`, `foxglove.PointCloud`, `foxglove.RawImage`, plus one generic JSON schema shared by `/imu`, `/motor`, `/velocity` |
| `has_image` | `BooleanField` | Whether the episode has an Image-tile-decodable stream (`foxglove.CompressedImage` or `foxglove.RawImage`) — `True` for every episode (cameras, and the decoded radar/sonar pseudo-images) |
| `has_pointcloud` | `BooleanField` | Whether it has a decodable point-cloud stream for the 3D tile (`foxglove.PointCloud`, i.e. lidar) — `True` for every episode |
| `has_gps` | `BooleanField` | Whether it has a decodable GPS fix stream for the Map tile (`foxglove.LocationFix`) — `True` for every episode |
| `has_imu` | `BooleanField` | Whether it has a schema-decoded IMU stream for a dedicated Plot tile — **`False` for every episode in this build**, even though a `/imu` topic exists (see [Parsing decisions](#parsing-decisions)) |
| `has_logs` | `BooleanField` | Whether it has a decodable log stream for the Logs tile — `False` for every episode (CANOE has no log messages) |
| `source` | `StringField` | Constant description string: `"CANOE (utiasASRL) - lake/reservoir USV traversal"` |
Standard FiftyOne bookkeeping fields (`id`, `metadata`, `created_at`,
`last_modified_at`) are also present but not source-specific.
### Label types and why
**No FiftyOne label fields are attached to this dataset.** CANOE ships no
object/semantic/classification annotations — the actual multimodal
content (camera images, decoded radar/sonar imagery, lidar point cloud,
GPS track, and telemetry) lives entirely inside each sample’s `.mcap`
file and is decoded live by FiftyOne’s multimodal viewer (Image, 3D, Map,
and Plot/Message tiles), not represented as separate
`Detection`/`Segmentation`/etc. label objects. The `has_*` boolean fields
above exist purely so users can filter or query episodes by which tiles
will populate with real data, without opening every MCAP file first —
e.g. `dataset.match(F("has_image") & F("has_gps"))`.
### Parsing decisions
- **One sample = one episode.** Each sample is a single, continuous
window of sensor data, never split into per-frame samples — FiftyOne’s
multimodal viewer handles playback within an episode.
- **Full sequences are far too large for one MCAP episode** (from
~714 GB / ~1.7 hours for the shortest example sequence, to
considerably more for the longer-route sequences), so 3 of the 4
episodes are fixed-duration
(30-60s) windows starting at the earliest timestamp across all sensor
folders for that sequence, downloaded via targeted S3 requests rather
than a full sequence sync. The **Sample** sequence (8.9 GB total) was
downloaded and authored in full instead, since it is small enough to
skip windowing entirely.
- **Lidar’s fixed-size preallocated array is filtered.** Each `.bin` file
preallocates a 131,072-point array per frame; most entries are
`(0,0,0)` “no return” placeholders. Points are filtered to range > 0.5 m
before being packed into the `foxglove.PointCloud` message (roughly
70-93% of raw points are dropped per frame across these episodes),
otherwise the 3D tile shows a dense blob at the origin.
- **Radar and sonar have no native MCAP polar-sensor schema.** Both are
decoded from their raw Oxford-style/encoder-embedded PNG encoding via
the `pycanoe` devkit’s `polar_to_cart()`, clipped and normalized to
`[0, 255]` uint8, and logged as `foxglove.RawImage` (mono8)
pseudo-images — not real camera views, but the closest available
Image-tile representation of a polar scan.
- **GPS/lat-lon lives only in a giant whole-sequence file.**
`novatel/novatel_original.csv` is interpolated to 1000 Hz and can be up
to ~3 GB per sequence. For the 3 windowed episodes, only the matching
time window was extracted via a targeted HTTP Range GET (estimating the
byte offset from the observed row rate) rather than downloading the
full file. For the Sample episode (downloaded in full), the same raw
file is parsed directly, skipping its free-text metadata header block
that precedes the actual CSV header row.
- **Ground-truth pose and velocity (`/tf`, `/odom`, `/velocity`) use the
lidar frame’s interpolated pose as the representative boat state**,
logged at every lidar frame’s timestamp from `novatel/lidar_poses.csv`.
`/odom` carries the full `foxglove.Odometry` message (pose + ENU/body
linear and angular velocity); `/velocity` duplicates the same
velocity/body-rate values as a flat JSON dict for convenient Plot-tile
inspection without decoding `Odometry`.
- **`/imu`, `/motor`, and `/velocity` are logged as generic JSON
messages**, not `foxglove.Imu` or another schema-decoded type — this is
why `has_imu` is `False` for every episode despite an `/imu` topic
being present. They still populate FiftyOne’s Message/Plot tile via raw
JSON, just without a dedicated schema-typed decoder.
- **Sensors are not hardware-synchronized.** Two separate onboard
computers drive different sensor groups (radar/lidar/camera vs.
sonar/sidescan/motor/GPS); only GPS, radar, and lidar are tightly
PTP-synced. Real per-sensor timestamps are used as-is, which is
correct/expected behavior rather than an alignment bug.
- **Lidar is also projected onto the left camera image**, matching the
`pycanoe` devkit’s own `tutorials/view_calib.ipynb` “Lidar-(Left) Cam
Overlay” cell (transform lidar points into the `cam_left` frame via
`calib/T_cam_left_lidar.txt`, keep forward-facing points within 50 m,
project with `calib/P_cam.txt`, color by depth). Two ways to view this
are provided: a pre-baked `/cam_left_lidar_overlay` JPEG stream (always
visible, no configuration needed, depth-colored jet colormap like the
devkit’s static plots) and a `/cam_left/calibration`
(`foxglove.CameraCalibration`) message that lets FiftyOne’s multimodal
viewer project the *live* `/lidar` point cloud onto the plain `/cam_left`
tile via its built-in “Pointcloud projections” toggle, and draw
`cam_left`’s frustum in the 3D tile. Both read the same calibration and
agree; only the rendering path (pre-baked image vs. live GPU projection)
differs. No calibration message is published for `/cam_right`, since
`P_cam.txt`/`T_cam_left_lidar.txt` are only valid for the left camera.
- **Sidescan sonar is out of scope for this build.**
`sidescan.xtf`/`sidescan.csv` aggregate all pings for a whole sequence
into single files with no per-frame breakdown; they are not decoded
into any MCAP channel here (only the small `waterfall-preview.png` was
kept as a representative preview asset during recon, and is not part of
the MCAP itself).
## Dataset Creation
### Curation Rationale
Full CANOE sequences are enormous — one example sequence (Day 1 – Lake
Short) alone is 297,684 objects / ~714 GB spanning ~1.7 hours, and the
longer-route sequences (e.g. Day 1 – Lake Long) are larger still — and the
public `--no-sign-request` S3 bucket allows arbitrary slicing, so
downloading all 8 sequences in full was not
practical for a lightweight FiftyOne showcase. 4 episodes were selected
for sensor/environment diversity instead of exhaustive coverage: the
officially curated, all-sensor “Sample” sequence in full (small enough at
8.9 GB to skip windowing), plus fixed-duration windows of two lake routes
and one reservoir route — the reservoir episode was specifically chosen
to also exercise the radar’s 150 m Doppler mode, distinct from the 1000 m
long-range mode used everywhere else.
### Source Data
#### Data Collection and Processing
Per ASRL’s dataset documentation (`DATA_REFERENCE.md` in the `pycanoe`
devkit, mirrored on canoedataset.com): data was collected on a lake and
reservoir in Ontario, Canada, over three days, traversing a “short” and
“long” lake route (each twice) and a reservoir route (three times) — over
14 hours and 50 km total across all 8 public sequences. Sensor suite:
Navtech RAS6 360° radar (4 Hz, 400 azimuths/rotation, dual range modes: 1000 m
long-range at 0.292 m/bin, or 150 m Doppler at 0.0438 m/bin), 128-beam
Ouster OS1 lidar with integrated IMU (10 Hz rotation, >1M points/sec,
strongest-return only), Teledyne Bumblebee rectified stereo camera (10 Hz,
1152×2048), Oculus M3000d imaging sonar (~14 Hz, 512 beams × 378 range
bins, 30 m max range), dual Starfish 990F sidescan sonar (not included in
this build), motor power sensors (2.5 Hz), and a NovAtel OEM7 GNSS/INS
post-processed with Inertial Explorer (using base-station corrections) to
~2-4 cm RMS position accuracy, interpolated to 1000 Hz.
For this repackaging: raw per-frame sensor files were downloaded from the
public `s3://canoe-data` bucket (targeted time-windowed slices for 3 of
the 4 episodes; a full sync for the Sample sequence), decoded with the
official `pycanoe` devkit (Oxford-style polar decoding for radar/sonar,
custom binary parsing for lidar, ENU-frame pose/velocity interpolation
from the NovAtel pose files), and packed into one merged `.mcap` file per
episode using `foxglove-sdk`. No sensor data was synthesized, relabeled,
or altered beyond the conversions and filtering documented in
[Parsing decisions](#parsing-decisions).
#### Who are the source data producers?
Autonomous Space Robotics Laboratory (ASRL), University of Toronto
Institute for Aerospace Studies (UTIAS) — original CANOE data collection,
sensor calibration, and ground-truth pose post-processing.
### Annotations
#### Annotation process
None. CANOE ships no object/semantic/classification labels — it is a
raw-sensor-plus-ground-truth-pose dataset intended for SLAM/odometry
benchmarking. This is corroborated by canoedataset.com’s Leaderboard tab
(currently empty, “Check back soon!”), whose column schema — `Method, Ref Sensor, Test Sensor, Lat RMSE, Long RMSE, Yaw RMSE, Consistency, Runtime, Environment`, split into SE(2)/SE(3) Odometry tabs — is purely pose-error
metrics, not annotation-based.
#### Who are the annotators?
N/A — no annotations exist. The closest artifact to a label is the
post-processed NovAtel GNSS/INS ground-truth pose, produced by ASRL using
NovAtel’s Inertial Explorer software suite (see
[Data Collection and Processing](#data-collection-and-processing)).
#### Personal and Sensitive Information
None identified. All sensor streams (radar, lidar, camera, sonar, GPS,
IMU, motor) record an uncrewed surface vessel’s traversal of a lake and
reservoir. Camera imagery may incidentally capture other lake/reservoir
users, boats, or shoreline structures visible from the water, but this
repackaging performs no additional processing, re-identification, or
redaction beyond what ASRL already released publicly. ASRL’s own
`DATA_LICENSE.md` notes that privacy concerns, including requests to
remove personally identifiable information, can be sent to
canoe@robotics.utias.utoronto.ca.
## Citation
No published paper or BibTeX entry exists for CANOE at the time of this
card. The AWS Open Data Registry listing gives the following suggested
access citation:
**BibTeX:**
[More Information Needed]
**APA:**
Autonomous Space Robotics Laboratory (ASRL), University of Toronto
Institute for Aerospace Studies. (n.d.). *CANOE (Canadian Aquatic
Navigation for Observation of the Environment) Dataset*. Registry of Open
Data on AWS. Retrieved from https://registry.opendata.aws/canoe/
> “CANOE (Canadian Aquatic Navigation for Observation of the Environment)
> Dataset was accessed on `DATE` from https://registry.opendata.aws/canoe.”
> — suggested citation text per the AWS Open Data Registry listing.
## More Information
This repository is an independently-curated, derived subset of the
official CANOE dataset, repackaged as MCAP for FiftyOne’s multimodal
support. It is not an official CANOE/ASRL artifact. For the full dataset
(all 8 public sequences, full un-windowed traversals, the devkit, and the
odometry/localization leaderboard), see https://www.canoedataset.com,
https://registry.opendata.aws/canoe/, and
https://github.com/utiasASRL/pycanoe.
## Dataset Card Authors
Harpreet Sahota ([@harpreetsahota](https://huggingface.co/harpreetsahota))
— MCAP repackaging and this card. Original dataset producers are listed
under [Dataset Description](#dataset-description).
## Dataset Card Contact
Harpreet Sahota — https://huggingface.co/harpreetsahota