Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Dataset Card for NTNU Underwater Multi-Camera Visual-Inertial#

preview

This is a FiftyOne dataset with 8 samples. Each sample is one run of an underwater robot, stored as a native multimodal MCAP episode.

The source is the Multi-Camera Underwater Visual-Inertial Dataset from the Autonomous Robots Lab, NTNU, converted from ROS 1 bags. Ariel, the lab’s underwater robot built on the BlueROV2 Heavy configuration, is piloted by hand through the Trondheim Fjord and the lab’s Marine Cybernetics Laboratory pool. An Alphasense rig records five monochrome cameras with its own inertial unit, while the vehicle’s autopilot logs a second inertial unit, a barometer, a downward rangefinder, the battery and the thruster outputs. Each run comes with the reference trajectory the authors computed with ReAqROVIO from four cameras and the inertial unit.

Installation#

If you haven’t already, install FiftyOne:

pip install -U fiftyone

Usage#

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = fouh.load_from_hub(
    "Voxel51/NTNU-Underwater-Multicam",
    name="NTNU-Underwater-Multicam",
    persistent=True,
)

# Launch the App
session = fo.launch_app(dataset)

Dataset Details#

Dataset Description#

8 runs, 6 in the fjord and 2 in the pool, totalling 63m 19s of recording over 1,439 m of reference trajectory, 379,896 camera frames and 759,943 inertial samples.

Curated by: Autonomous Robots Lab, NTNU (source release)
Funded by: [More Information Needed]
Shared by: Voxel51 (FiftyOne conversion)
Language(s): Not applicable (sensor data)
License: BSD 3-Clause

Dataset Sources#

Repository: ntnu-arl/underwater-datasets (source release); Voxel51/NTNU-Underwater-Multicam (this conversion)
Paper: An Online Self-calibrating Refractive Camera Model with Application to Underwater Odometry (ICRA 2024)
Demo: [More Information Needed]

Uses#

Direct Use#

Underwater visual-inertial odometry and SLAM with multiple cameras and two inertial units, evaluated against the ReAqROVIO reference trajectory shipped with each run.

Out-of-Scope Use#

[More Information Needed]

Dataset Structure#

Topology

An ungrouped FiftyOne dataset with media_type="multimodal". One sample is one run, and its filepath is a .fo.mcap file. There are 8 samples (6 fjord_*, 2 mclab_*), no sample tags, and an empty dataset.info. The dataset has no FiftyOne label fields; every signal lives inside the MCAP file as a channel and is shown by the App’s multimodal viewer.

Sample fields

Field	FiftyOne type	Description
`id`, `filepath`, `tags`, `metadata`, `created_at`, `last_modified_at`	built-in	Standard FiftyOne sample fields
`sequence`	`StringField`	Run name, e.g. `fjord_1` (derived from the source bag)
`environment`	`StringField`	Where the run was recorded: Trondheim Fjord or the pool
`recorded`	`StringField`	Recording date
`duration`	`FloatField`	Run length in seconds
`num_camera_frames`	`IntField`	Camera frames across all five cameras
`num_imu_samples`	`IntField`	Inertial samples across both inertial units
`num_reference_poses`	`IntField`	Poses on the ReAqROVIO reference trajectory
`reference_path_m`	`FloatField`	Length of the reference trajectory in metres
`max_depth_m`	`FloatField`	Maximum depth, from the barometer-to-depth mapping
`rangefinder_min_m`, `rangefinder_max_m`	`FloatField`	Range of the downward rangefinder readings, in metres
`battery_start_v`, `battery_end_v`	`FloatField`	Battery voltage at the start and end of the run

Episode contents

Each MCAP episode contains these channels:

Channel	Schema or content
`/stereo-left`, `/stereo-right`, `/top-camera`, `/left-camera`, `/right-camera`	Five 720x540 cameras at 20 Hz, `foxglove.CompressedVideo`
`<camera>-calibration`	Intrinsics for each camera, `foxglove.CameraCalibration`
`/tf`	Where each camera sits relative to the inertial unit, `foxglove.FrameTransform`
`/imu.plot`, `/autopilot-imu.plot`	Rig inertial unit at 200 Hz; autopilot inertial unit at 20 Hz
`/barometer.plot`, `/rangefinder.plot`, `/battery.plot`, `/thrusters.plot`	Barometer reading and its depth; downward rangefinder; battery voltage and current; the eight thruster outputs
`/autopilot-pose`	The autopilot’s own position estimate, `foxglove.PoseInFrame`
`/reference-pose`, `/reference.plot`	ReAqROVIO reference trajectory as `foxglove.PoseInFrame`, with its position as a plot
`/sequence`	Names the run

Runs

Run	Where	Duration	Camera frames	Reference path	Max depth	Height over bottom
`fjord_1`	fjord	5m 18s	31,830	142 m	7.1 m	0.4 to 9.6 m
`fjord_2`	fjord	9m 58s	59,785	207 m	7.4 m	0.3 to 9.4 m
`fjord_3`	fjord	5m 54s	35,421	123 m	7.1 m	0.3 to 6.6 m
`fjord_4`	fjord	9m 57s	59,745	166 m	7.3 m	0.4 to 5.2 m
`fjord_5`	fjord	7m 56s	47,583	235 m	7.7 m	0.4 to 4.5 m
`fjord_6`	fjord	11m 19s	67,883	306 m	7.3 m	0.4 to 6.8 m
`mclab_1`	pool	6m 34s	39,417	135 m	0.5 m	0.3 to 1.2 m
`mclab_2`	pool	6m 22s	38,232	125 m	0.6 m	0.3 to 1.2 m

Parsing decisions

Why MCAP: the dataset is one multimodal sample per run, so the five cameras, inertial units, plots and poses play back on one shared clock instead of being split into per-frame samples.
Video encoding: the cameras publish raw 8-bit monochrome frames, which are re-encoded to Annex-B H.264 without B-frames, one access unit per frame, on each frame’s own timestamp.
Calibration: the release calibrates the stereo pair underwater and the other three cameras in air only. The stereo pair carries its underwater intrinsics and the other three their in-air ones, which refraction at the housing port changes once the vehicle is submerged. All five use Kalibr’s pinhole-equidistant model, which the calibration schema names equidistant. The camera-to-inertial extrinsics behind /tf were calibrated in air.
Depth: the barometer reading is carried as published, and depth_m beside it applies the mapping the release gives, (reading - 2660) / 241 metres below the surface. The release states that mapping for fresh water, so in the salt water of the fjord it reads a little deep.
Rangefinder: it reports its own limits as 0 to 1,000 m, so every reading is carried.
Thrusters and clock rollover: the thruster outputs are the autopilot’s eight output channels as published. The autopilot stamps them from the flight controller’s 32-bit microsecond clock, which rolls over every 71.6 minutes of uptime, so each autopilot stamp is moved by the whole rollovers that bring it nearest the moment the bag recorded it.
Poses: the autopilot’s pose estimate is carried as published and is not the reference. The reference is the ReAqROVIO trajectory the release ships beside each bag.
Left out: the bags’ /tf and /rosout topics and the autopilot’s odometry message, whose pose /autopilot-pose already carries, are not reproduced.

Dataset Creation#

Curation Rationale#

[More Information Needed]

Source Data#

Data Collection and Processing#

Recorded with the Ariel underwater robot, piloted by hand in the Trondheim Fjord and the Marine Cybernetics Laboratory pool, using an Alphasense camera rig and the vehicle’s autopilot sensors. The FiftyOne conversion reads the ROS 1 bags without a ROS install and writes one MCAP episode per bag; see Parsing decisions above for the changes made.

Who are the source data producers?#

The Autonomous Robots Lab at NTNU.

Annotations#

Annotation process#

The dataset has no human annotations. The reference trajectory in each run is computed by the source authors with ReAqROVIO from four cameras and the inertial unit.

Who are the annotators?#

Not applicable.

Personal and Sensitive Information#

[More Information Needed]

Citation#

The authors ask that work using the dataset cite:

BibTeX:

@INPROCEEDINGS{10610643,
  author={Singh, Mohit and Dharmadhikari, Mihir and Alexis, Kostas},
  booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
  title={An Online Self-calibrating Refractive Camera Model with Application to Underwater Odometry},
  year={2024},
  pages={10005-10011},
  doi={10.1109/ICRA57147.2024.10610643}
}

APA:

Singh, M., Dharmadhikari, M., & Alexis, K. (2024). An online self-calibrating refractive camera model with application to underwater odometry. In 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 10005-10011). https://doi.org/10.1109/ICRA57147.2024.10610643

More Information#

The source release is distributed under the BSD 3-Clause License, and this conversion is distributed under the same license, whose notice follows.

BSD 3-Clause License

Copyright (c) 2023, NTNU Autonomous Robots Lab

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
   list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its
   contributors may be used to endorse or promote products derived from
   this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Changes from the source: conversion from ROS 1 bags to the FiftyOne MCAP flavor, H.264 encoding of the five camera streams, the release’s Kalibr calibrations carried as camera intrinsics and rig transforms, the release’s barometer mapping applied as a depth track, autopilot stamps moved past the flight controller’s clock rollovers, the reference trajectory carried as a pose stream, and the bags’ /tf, /rosout and odometry topics left out.

Dataset Card Authors#

[More Information Needed]

Dataset Card Contact#

[More Information Needed]