#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Dataset Card for NTNU Underwater Multi-Camera Visual-Inertial
This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 8 samples. Each sample is one run of an underwater robot, stored as a native multimodal MCAP episode.
The source is the [Multi-Camera Underwater Visual-Inertial Dataset](https://huggingface.co/datasets/ntnu-arl/underwater-datasets) from the [Autonomous Robots Lab, NTNU](https://www.autonomousrobotslab.com/), converted from ROS 1 bags. Ariel, the lab’s underwater robot built on the BlueROV2 Heavy configuration, is piloted by hand through the Trondheim Fjord and the lab’s Marine Cybernetics Laboratory pool. An Alphasense rig records five monochrome cameras with its own inertial unit, while the vehicle’s autopilot logs a second inertial unit, a barometer, a downward rangefinder, the battery and the thruster outputs. Each run comes with the reference trajectory the authors computed with ReAqROVIO from four cameras and the inertial unit.
## Installation
If you haven’t already, install FiftyOne:
```bash
pip install -U fiftyone
```
## Usage
```python
import fiftyone as fo
import fiftyone.utils.huggingface as fouh
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = fouh.load_from_hub(
"Voxel51/NTNU-Underwater-Multicam",
name="NTNU-Underwater-Multicam",
persistent=True,
)
# Launch the App
session = fo.launch_app(dataset)
```
## Dataset Details
### Dataset Description
8 runs, 6 in the fjord and 2 in the pool, totalling 63m 19s of recording over 1,439 m of reference trajectory, 379,896 camera frames and 759,943 inertial samples.
- **Curated by:** Autonomous Robots Lab, NTNU (source release)
- **Funded by:** [More Information Needed]
- **Shared by:** Voxel51 (FiftyOne conversion)
- **Language(s):** Not applicable (sensor data)
- **License:** BSD 3-Clause
### Dataset Sources
- **Repository:** [ntnu-arl/underwater-datasets](https://huggingface.co/datasets/ntnu-arl/underwater-datasets) (source release); [Voxel51/NTNU-Underwater-Multicam](https://huggingface.co/datasets/Voxel51/NTNU-Underwater-Multicam) (this conversion)
- **Paper:** [An Online Self-calibrating Refractive Camera Model with Application to Underwater Odometry](https://doi.org/10.1109/ICRA57147.2024.10610643) (ICRA 2024)
- **Demo:** [More Information Needed]
## Uses
### Direct Use
Underwater visual-inertial odometry and SLAM with multiple cameras and two inertial units, evaluated against the ReAqROVIO reference trajectory shipped with each run.
### Out-of-Scope Use
[More Information Needed]
## Dataset Structure
**Topology**
An ungrouped FiftyOne dataset with `media_type="multimodal"`. One sample is one run, and its `filepath` is a `.fo.mcap` file. There are 8 samples (6 `fjord_*`, 2 `mclab_*`), no sample tags, and an empty `dataset.info`. The dataset has no FiftyOne label fields; every signal lives inside the MCAP file as a channel and is shown by the App’s multimodal viewer.
**Sample fields**
| Field | FiftyOne type | Description |
|------------------------------------------------------------------------|-----------------|---------------------------------------------------------|
| `id`, `filepath`, `tags`, `metadata`, `created_at`, `last_modified_at` | built-in | Standard FiftyOne sample fields |
| `sequence` | `StringField` | Run name, e.g. `fjord_1` (derived from the source bag) |
| `environment` | `StringField` | Where the run was recorded: Trondheim Fjord or the pool |
| `recorded` | `StringField` | Recording date |
| `duration` | `FloatField` | Run length in seconds |
| `num_camera_frames` | `IntField` | Camera frames across all five cameras |
| `num_imu_samples` | `IntField` | Inertial samples across both inertial units |
| `num_reference_poses` | `IntField` | Poses on the ReAqROVIO reference trajectory |
| `reference_path_m` | `FloatField` | Length of the reference trajectory in metres |
| `max_depth_m` | `FloatField` | Maximum depth, from the barometer-to-depth mapping |
| `rangefinder_min_m`, `rangefinder_max_m` | `FloatField` | Range of the downward rangefinder readings, in metres |
| `battery_start_v`, `battery_end_v` | `FloatField` | Battery voltage at the start and end of the run |
**Episode contents**
Each MCAP episode contains these channels:
| Channel | Schema or content |
|---------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|
| `/stereo-left`, `/stereo-right`, `/top-camera`, `/left-camera`, `/right-camera` | Five 720x540 cameras at 20 Hz, `foxglove.CompressedVideo` |
| `-calibration` | Intrinsics for each camera, `foxglove.CameraCalibration` |
| `/tf` | Where each camera sits relative to the inertial unit, `foxglove.FrameTransform` |
| `/imu.plot`, `/autopilot-imu.plot` | Rig inertial unit at 200 Hz; autopilot inertial unit at 20 Hz |
| `/barometer.plot`, `/rangefinder.plot`, `/battery.plot`, `/thrusters.plot` | Barometer reading and its depth; downward rangefinder; battery voltage and current; the eight thruster outputs |
| `/autopilot-pose` | The autopilot’s own position estimate, `foxglove.PoseInFrame` |
| `/reference-pose`, `/reference.plot` | ReAqROVIO reference trajectory as `foxglove.PoseInFrame`, with its position as a plot |
| `/sequence` | Names the run |
**Runs**
| Run | Where | Duration | Camera frames | Reference path | Max depth | Height over bottom |
|-----------|---------|------------|-----------------|------------------|-------------|----------------------|
| `fjord_1` | fjord | 5m 18s | 31,830 | 142 m | 7.1 m | 0.4 to 9.6 m |
| `fjord_2` | fjord | 9m 58s | 59,785 | 207 m | 7.4 m | 0.3 to 9.4 m |
| `fjord_3` | fjord | 5m 54s | 35,421 | 123 m | 7.1 m | 0.3 to 6.6 m |
| `fjord_4` | fjord | 9m 57s | 59,745 | 166 m | 7.3 m | 0.4 to 5.2 m |
| `fjord_5` | fjord | 7m 56s | 47,583 | 235 m | 7.7 m | 0.4 to 4.5 m |
| `fjord_6` | fjord | 11m 19s | 67,883 | 306 m | 7.3 m | 0.4 to 6.8 m |
| `mclab_1` | pool | 6m 34s | 39,417 | 135 m | 0.5 m | 0.3 to 1.2 m |
| `mclab_2` | pool | 6m 22s | 38,232 | 125 m | 0.6 m | 0.3 to 1.2 m |
**Parsing decisions**
- **Why MCAP:** the dataset is one multimodal sample per run, so the five cameras, inertial units, plots and poses play back on one shared clock instead of being split into per-frame samples.
- **Video encoding:** the cameras publish raw 8-bit monochrome frames, which are re-encoded to Annex-B H.264 without B-frames, one access unit per frame, on each frame’s own timestamp.
- **Calibration:** the release calibrates the stereo pair underwater and the other three cameras in air only. The stereo pair carries its underwater intrinsics and the other three their in-air ones, which refraction at the housing port changes once the vehicle is submerged. All five use Kalibr’s pinhole-equidistant model, which the calibration schema names `equidistant`. The camera-to-inertial extrinsics behind `/tf` were calibrated in air.
- **Depth:** the barometer reading is carried as published, and `depth_m` beside it applies the mapping the release gives, `(reading - 2660) / 241` metres below the surface. The release states that mapping for fresh water, so in the salt water of the fjord it reads a little deep.
- **Rangefinder:** it reports its own limits as 0 to 1,000 m, so every reading is carried.
- **Thrusters and clock rollover:** the thruster outputs are the autopilot’s eight output channels as published. The autopilot stamps them from the flight controller’s 32-bit microsecond clock, which rolls over every 71.6 minutes of uptime, so each autopilot stamp is moved by the whole rollovers that bring it nearest the moment the bag recorded it.
- **Poses:** the autopilot’s pose estimate is carried as published and is not the reference. The reference is the ReAqROVIO trajectory the release ships beside each bag.
- **Left out:** the bags’ `/tf` and `/rosout` topics and the autopilot’s odometry message, whose pose `/autopilot-pose` already carries, are not reproduced.
## Dataset Creation
### Curation Rationale
[More Information Needed]
### Source Data
#### Data Collection and Processing
Recorded with the Ariel underwater robot, piloted by hand in the Trondheim Fjord and the Marine Cybernetics Laboratory pool, using an Alphasense camera rig and the vehicle’s autopilot sensors. The FiftyOne conversion reads the ROS 1 bags without a ROS install and writes one MCAP episode per bag; see Parsing decisions above for the changes made.
#### Who are the source data producers?
The Autonomous Robots Lab at NTNU.
### Annotations
#### Annotation process
The dataset has no human annotations. The reference trajectory in each run is computed by the source authors with ReAqROVIO from four cameras and the inertial unit.
#### Who are the annotators?
Not applicable.
#### Personal and Sensitive Information
[More Information Needed]
## Citation
The authors ask that work using the dataset cite:
**BibTeX:**
```bibtex
@INPROCEEDINGS{10610643,
author={Singh, Mohit and Dharmadhikari, Mihir and Alexis, Kostas},
booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
title={An Online Self-calibrating Refractive Camera Model with Application to Underwater Odometry},
year={2024},
pages={10005-10011},
doi={10.1109/ICRA57147.2024.10610643}
}
```
**APA:**
Singh, M., Dharmadhikari, M., & Alexis, K. (2024). An online self-calibrating refractive camera model with application to underwater odometry. In *2024 IEEE International Conference on Robotics and Automation (ICRA)* (pp. 10005-10011). https://doi.org/10.1109/ICRA57147.2024.10610643
## More Information
The source release is distributed under the BSD 3-Clause License, and this conversion is distributed under the same license, whose notice follows.
```default
BSD 3-Clause License
Copyright (c) 2023, NTNU Autonomous Robots Lab
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```
Changes from the source: conversion from ROS 1 bags to the FiftyOne MCAP flavor, H.264 encoding of the five camera streams, the release’s Kalibr calibrations carried as camera intrinsics and rig transforms, the release’s barometer mapping applied as a depth track, autopilot stamps moved past the flight controller’s clock rollovers, the reference trajectory carried as a pose stream, and the bags’ `/tf`, `/rosout` and odometry topics left out.
## Dataset Card Authors
[More Information Needed]
## Dataset Card Contact
[More Information Needed]