Note
This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.
Dataset Card for CANOE Multimodal (MCAP)#
A FiftyOne build of CANOE (Canadian Aquatic Navigation for Observation of the Environment), a multi-sensor marine navigation dataset collected by ASRL (UTIAS) on an uncrewed surface vessel (USV). This build repackages 4 of CANOE’s 8 public sequences as time-synchronized MCAP recordings for FiftyOne’s native multimodal dataset support (FiftyOne 1.19+). Each sample is one episode, viewable in FiftyOne’s tiled multimodal viewer with synchronized stereo camera, radar, sonar, lidar point cloud, GPS/INS track, and IMU/motor/velocity telemetry.
CANOE itself is a raw multi-sensor dataset for benchmarking odometry/localization algorithms in aquatic environments — it ships no object/semantic annotations, only dense ground-truth 6-DoF pose from post-processed GNSS/INS. This repackaging does not add or alter any ground truth; see Dataset Creation for exactly what was kept, converted, and left out.
This is a FiftyOne dataset with 4 samples.
Installation#
If you haven’t already, install FiftyOne:
pip install -U fiftyone
Usage#
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("harpreetsahota/canoe-multimodal")
# Launch the App
session = fo.launch_app(dataset)
Dataset Details#
Dataset Description#
CANOE is a multi-sensor marine navigation dataset collected on an uncrewed surface vessel (USV) traversing a lake and a reservoir in Ontario, Canada, over three days (8 public sequences, 14+ hours, 50 km). Sensor suite: a 360° Navtech RAS6 radar (dual range modes), a 128-beam Ouster OS1 lidar with integrated IMU, a Teledyne Bumblebee stereo camera, an Oculus M3000d imaging sonar, dual Starfish 990F sidescan sonar, motor power sensors, and a NovAtel OEM7 GNSS/INS post-processed to ~2-4 cm RMS position accuracy. The intended purpose is benchmarking odometry and localization algorithms for aquatic navigation; there are no object/semantic annotations, only raw sensor streams plus dense ground-truth pose. This FiftyOne build covers 4 of the 8 public sequences, chosen for sensor/environment diversity (see Curation Rationale).
Curated by: Autonomous Space Robotics Laboratory (ASRL), University of Toronto Institute for Aerospace Studies (UTIAS) — original CANOE data collection, sensor calibration, and ground-truth pose post-processing. This MCAP/FiftyOne multimodal repackaging (episode authoring, dataset card) was prepared independently by Harpreet Sahota.
Funded by: [More Information Needed]
Shared by: Harpreet Sahota (this repackaging); the original CANOE dataset is shared by ASRL via the AWS Open Data Registry (
s3://canoe-data) and https://www.canoedataset.com.Language(s): N/A (sensor data — radar, lidar, imagery, sonar, GPS, IMU; no text).
License: CC BY 4.0, inherited from the source CANOE release.
Dataset Sources#
Repository (devkit): https://github.com/utiasASRL/pycanoe
Paper: [More Information Needed] — no published paper is referenced by the AWS Open Data Registry listing, the devkit, or https://www.canoedataset.com at the time of this card; see Citation for the registry’s suggested access citation.
Demo: https://www.canoedataset.com (official docs/download/leaderboard site) and https://registry.opendata.aws/canoe/ (AWS Open Data listing)
Uses#
Direct Use#
Exercising and demoing FiftyOne’s multimodal MCAP support: synchronized playback of stereo camera, decoded radar/sonar pseudo-imagery, 3D lidar point cloud, GPS/map track, and IMU/motor/velocity telemetry across real aquatic robotics recordings.
Browsing/inspecting USV traversals of a lake (short and long routes) and a reservoir, including both radar operating modes (1000 m long-range and 150 m Doppler).
Prototyping odometry/localization analyses against the dense ground-truth pose (
/tf,/odom,/gps,/velocity) provided at each sensor’s native timestamp.
Out-of-Scope Use#
Reproducing CANOE’s official odometry/localization benchmark (the SE(2)/SE(3) Lat/Long/Yaw RMSE leaderboard on canoedataset.com) — that protocol operates over full, un-windowed sequences; this build covers only 4 of the 8 public sequences, 3 of which are short fixed-duration windows (30-60s), not full traversals.
Any task requiring object/semantic annotations — CANOE ships none (purely raw sensor streams plus ground-truth pose).
Commercial or non-commercial use inconsistent with the CC BY 4.0 attribution requirement.
Dataset Structure#
This is a flat (ungrouped) FiftyOne dataset with media_type: "multimodal"
and 4 samples. Each sample is one episode, stored as one .mcap
file; FiftyOne infers the multimodal media type automatically from the
.mcap extension. There are no separate per-frame image or point-cloud
samples — the episode is the sample unit, and every stream inside it
(camera, radar, sonar, lidar, GPS, IMU, motor) is decoded live by
FiftyOne’s multimodal viewer.
The dataset carries no per-sample tags and dataset.info is empty (no
extra dataset-level metadata beyond the per-sample fields below).
Episodes in this dataset#
|
|
Coverage |
|
|---|---|---|---|
|
Sample |
Full sequence (~72s active sensors; GPS/INS logs continuously for ~581s) |
1000 (long-range) |
|
Day 1 – Lake Short |
30s window |
1000 (long-range) |
|
Day 1 – Lake Long |
60s window |
1000 (long-range) |
|
Reservoir Doppler 1 |
60s window |
150 (Doppler) |
route gives the human-readable route name from canoedataset.com’s
Download tab. The Sample sequence is the only one downloaded and
authored in full — at 8.9 GB it is small enough to skip windowing; it is
also the only episode where the GNSS/INS trace outlasts the active
camera/lidar/radar/sonar capture window (continuous INS logging vs. a
shorter active-sensor window), which is expected, not a data error. The
other 3 episodes are fixed-duration windows starting at the earliest
sensor timestamp for that sequence. 4 of the 8 public CANOE sequences are
not (yet) included in this build: Day 2 – Lake Short, Day 2 – Lake Long,
Reservoir Doppler 2, and Reservoir 3.
Fields#
Field |
FiftyOne type |
Description |
|---|---|---|
|
|
Absolute path to the episode’s merged |
|
|
Empty for every sample in this build |
|
|
Raw CANOE sequence ID ( |
|
|
Human-readable route label, from the Download tab at https://www.canoedataset.com/#/download |
|
|
Radar operating mode for this sequence’s data: |
|
|
Episode duration in seconds, computed from the merged MCAP’s message-time span |
|
|
Total MCAP message count across all channels in the episode |
|
|
Total MCAP channel (topic) count in the episode (13 for every episode in this build) |
|
|
Every MCAP topic present: |
|
|
Every distinct Foxglove schema present: |
|
|
Whether the episode has an Image-tile-decodable stream ( |
|
|
Whether it has a decodable point-cloud stream for the 3D tile ( |
|
|
Whether it has a decodable GPS fix stream for the Map tile ( |
|
|
Whether it has a schema-decoded IMU stream for a dedicated Plot tile — |
|
|
Whether it has a decodable log stream for the Logs tile — |
|
|
Constant description string: |
Standard FiftyOne bookkeeping fields (id, metadata, created_at,
last_modified_at) are also present but not source-specific.
Label types and why#
No FiftyOne label fields are attached to this dataset. CANOE ships no
object/semantic/classification annotations — the actual multimodal
content (camera images, decoded radar/sonar imagery, lidar point cloud,
GPS track, and telemetry) lives entirely inside each sample’s .mcap
file and is decoded live by FiftyOne’s multimodal viewer (Image, 3D, Map,
and Plot/Message tiles), not represented as separate
Detection/Segmentation/etc. label objects. The has_* boolean fields
above exist purely so users can filter or query episodes by which tiles
will populate with real data, without opening every MCAP file first —
e.g. dataset.match(F("has_image") & F("has_gps")).
Parsing decisions#
One sample = one episode. Each sample is a single, continuous window of sensor data, never split into per-frame samples — FiftyOne’s multimodal viewer handles playback within an episode.
Full sequences are far too large for one MCAP episode (from ~714 GB / ~1.7 hours for the shortest example sequence, to considerably more for the longer-route sequences), so 3 of the 4 episodes are fixed-duration (30-60s) windows starting at the earliest timestamp across all sensor folders for that sequence, downloaded via targeted S3 requests rather than a full sequence sync. The Sample sequence (8.9 GB total) was downloaded and authored in full instead, since it is small enough to skip windowing entirely.
Lidar’s fixed-size preallocated array is filtered. Each
.binfile preallocates a 131,072-point array per frame; most entries are(0,0,0)“no return” placeholders. Points are filtered to range > 0.5 m before being packed into thefoxglove.PointCloudmessage (roughly 70-93% of raw points are dropped per frame across these episodes), otherwise the 3D tile shows a dense blob at the origin.Radar and sonar have no native MCAP polar-sensor schema. Both are decoded from their raw Oxford-style/encoder-embedded PNG encoding via the
pycanoedevkit’spolar_to_cart(), clipped and normalized to[0, 255]uint8, and logged asfoxglove.RawImage(mono8) pseudo-images — not real camera views, but the closest available Image-tile representation of a polar scan.GPS/lat-lon lives only in a giant whole-sequence file.
novatel/novatel_original.csvis interpolated to 1000 Hz and can be up to ~3 GB per sequence. For the 3 windowed episodes, only the matching time window was extracted via a targeted HTTP Range GET (estimating the byte offset from the observed row rate) rather than downloading the full file. For the Sample episode (downloaded in full), the same raw file is parsed directly, skipping its free-text metadata header block that precedes the actual CSV header row.Ground-truth pose and velocity (
/tf,/odom,/velocity) use the lidar frame’s interpolated pose as the representative boat state, logged at every lidar frame’s timestamp fromnovatel/lidar_poses.csv./odomcarries the fullfoxglove.Odometrymessage (pose + ENU/body linear and angular velocity);/velocityduplicates the same velocity/body-rate values as a flat JSON dict for convenient Plot-tile inspection without decodingOdometry./imu,/motor, and/velocityare logged as generic JSON messages, notfoxglove.Imuor another schema-decoded type — this is whyhas_imuisFalsefor every episode despite an/imutopic being present. They still populate FiftyOne’s Message/Plot tile via raw JSON, just without a dedicated schema-typed decoder.Sensors are not hardware-synchronized. Two separate onboard computers drive different sensor groups (radar/lidar/camera vs. sonar/sidescan/motor/GPS); only GPS, radar, and lidar are tightly PTP-synced. Real per-sensor timestamps are used as-is, which is correct/expected behavior rather than an alignment bug.
Lidar is also projected onto the left camera image, matching the
pycanoedevkit’s owntutorials/view_calib.ipynb“Lidar-(Left) Cam Overlay” cell (transform lidar points into thecam_leftframe viacalib/T_cam_left_lidar.txt, keep forward-facing points within 50 m, project withcalib/P_cam.txt, color by depth). Two ways to view this are provided: a pre-baked/cam_left_lidar_overlayJPEG stream (always visible, no configuration needed, depth-colored jet colormap like the devkit’s static plots) and a/cam_left/calibration(foxglove.CameraCalibration) message that lets FiftyOne’s multimodal viewer project the live/lidarpoint cloud onto the plain/cam_lefttile via its built-in “Pointcloud projections” toggle, and drawcam_left’s frustum in the 3D tile. Both read the same calibration and agree; only the rendering path (pre-baked image vs. live GPU projection) differs. No calibration message is published for/cam_right, sinceP_cam.txt/T_cam_left_lidar.txtare only valid for the left camera.Sidescan sonar is out of scope for this build.
sidescan.xtf/sidescan.csvaggregate all pings for a whole sequence into single files with no per-frame breakdown; they are not decoded into any MCAP channel here (only the smallwaterfall-preview.pngwas kept as a representative preview asset during recon, and is not part of the MCAP itself).
Dataset Creation#
Curation Rationale#
Full CANOE sequences are enormous — one example sequence (Day 1 – Lake
Short) alone is 297,684 objects / ~714 GB spanning ~1.7 hours, and the
longer-route sequences (e.g. Day 1 – Lake Long) are larger still — and the
public --no-sign-request S3 bucket allows arbitrary slicing, so
downloading all 8 sequences in full was not
practical for a lightweight FiftyOne showcase. 4 episodes were selected
for sensor/environment diversity instead of exhaustive coverage: the
officially curated, all-sensor “Sample” sequence in full (small enough at
8.9 GB to skip windowing), plus fixed-duration windows of two lake routes
and one reservoir route — the reservoir episode was specifically chosen
to also exercise the radar’s 150 m Doppler mode, distinct from the 1000 m
long-range mode used everywhere else.
Source Data#
Data Collection and Processing#
Per ASRL’s dataset documentation (DATA_REFERENCE.md in the pycanoe
devkit, mirrored on canoedataset.com): data was collected on a lake and
reservoir in Ontario, Canada, over three days, traversing a “short” and
“long” lake route (each twice) and a reservoir route (three times) — over
14 hours and 50 km total across all 8 public sequences. Sensor suite:
Navtech RAS6 360° radar (4 Hz, 400 azimuths/rotation, dual range modes: 1000 m
long-range at 0.292 m/bin, or 150 m Doppler at 0.0438 m/bin), 128-beam
Ouster OS1 lidar with integrated IMU (10 Hz rotation, >1M points/sec,
strongest-return only), Teledyne Bumblebee rectified stereo camera (10 Hz,
1152Ă—2048), Oculus M3000d imaging sonar (~14 Hz, 512 beams Ă— 378 range
bins, 30 m max range), dual Starfish 990F sidescan sonar (not included in
this build), motor power sensors (2.5 Hz), and a NovAtel OEM7 GNSS/INS
post-processed with Inertial Explorer (using base-station corrections) to
~2-4 cm RMS position accuracy, interpolated to 1000 Hz.
For this repackaging: raw per-frame sensor files were downloaded from the
public s3://canoe-data bucket (targeted time-windowed slices for 3 of
the 4 episodes; a full sync for the Sample sequence), decoded with the
official pycanoe devkit (Oxford-style polar decoding for radar/sonar,
custom binary parsing for lidar, ENU-frame pose/velocity interpolation
from the NovAtel pose files), and packed into one merged .mcap file per
episode using foxglove-sdk. No sensor data was synthesized, relabeled,
or altered beyond the conversions and filtering documented in
Parsing decisions.
Who are the source data producers?#
Autonomous Space Robotics Laboratory (ASRL), University of Toronto Institute for Aerospace Studies (UTIAS) — original CANOE data collection, sensor calibration, and ground-truth pose post-processing.
Annotations#
Annotation process#
None. CANOE ships no object/semantic/classification labels — it is a
raw-sensor-plus-ground-truth-pose dataset intended for SLAM/odometry
benchmarking. This is corroborated by canoedataset.com’s Leaderboard tab
(currently empty, “Check back soon!”), whose column schema — Method, Ref Sensor, Test Sensor, Lat RMSE, Long RMSE, Yaw RMSE, Consistency, Runtime, Environment, split into SE(2)/SE(3) Odometry tabs — is purely pose-error
metrics, not annotation-based.
Who are the annotators?#
N/A — no annotations exist. The closest artifact to a label is the post-processed NovAtel GNSS/INS ground-truth pose, produced by ASRL using NovAtel’s Inertial Explorer software suite (see Data Collection and Processing).
Personal and Sensitive Information#
None identified. All sensor streams (radar, lidar, camera, sonar, GPS,
IMU, motor) record an uncrewed surface vessel’s traversal of a lake and
reservoir. Camera imagery may incidentally capture other lake/reservoir
users, boats, or shoreline structures visible from the water, but this
repackaging performs no additional processing, re-identification, or
redaction beyond what ASRL already released publicly. ASRL’s own
DATA_LICENSE.md notes that privacy concerns, including requests to
remove personally identifiable information, can be sent to
canoe@robotics.utias.utoronto.ca.
Citation#
No published paper or BibTeX entry exists for CANOE at the time of this card. The AWS Open Data Registry listing gives the following suggested access citation:
BibTeX:
[More Information Needed]
APA:
Autonomous Space Robotics Laboratory (ASRL), University of Toronto Institute for Aerospace Studies. (n.d.). CANOE (Canadian Aquatic Navigation for Observation of the Environment) Dataset. Registry of Open Data on AWS. Retrieved from https://registry.opendata.aws/canoe/
“CANOE (Canadian Aquatic Navigation for Observation of the Environment) Dataset was accessed on
DATEfrom https://registry.opendata.aws/canoe.” — suggested citation text per the AWS Open Data Registry listing.
More Information#
This repository is an independently-curated, derived subset of the official CANOE dataset, repackaged as MCAP for FiftyOne’s multimodal support. It is not an official CANOE/ASRL artifact. For the full dataset (all 8 public sequences, full un-windowed traversals, the devkit, and the odometry/localization leaderboard), see https://www.canoedataset.com, https://registry.opendata.aws/canoe/, and https://github.com/utiasASRL/pycanoe.
Dataset Card Contact#
Harpreet Sahota — https://huggingface.co/harpreetsahota