#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Dataset Card for CitrusFarm sample (FiftyOne multimodal MCAP)
This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 1 sample. The sample is one sequence of a ground robot driving through a citrus farm, stored as a native multimodal MCAP episode.
The source is [CitrusFarm](https://ucr-robotics.github.io/Citrus-Farm-Dataset/), the multimodal agricultural robotics dataset from the ARCS Lab at the University of California Riverside, converted from ROS 1 bags. A Clearpath Jackal drives the rows of citrus trees at the university’s Agricultural Experimental Station carrying a FLIR Blackfly monochrome camera, a FLIR ADK thermal camera, a Mapir Survey3 camera that sees red, green and near-infrared, a Stereolabs ZED 2i stereo camera with its depth, a Velodyne LiDAR, a Microstrain inertial unit and a Piksi GPS-RTK receiver, and records its own wheel odometry. The release holds seven sequences from three fields, 1.3 TB in all; this sample is the smallest of them, `06_14B_Jackal`, from field 14B.
## Installation
If you haven’t already, install FiftyOne:
```bash
pip install -U fiftyone
```
## Usage
```python
import fiftyone as fo
import fiftyone.utils.huggingface as fouh
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = fouh.load_from_hub(
"Voxel51/CitrusFarm-Sample",
name="CitrusFarm-Sample",
persistent=True,
)
# Launch the App
session = fo.launch_app(dataset)
```
## Dataset Details
### Dataset Description
One sequence, 5m 03s of driving over 357 m, with 3,026 monochrome, 3,024 thermal, 3,025 red-green-NIR and 3,022 ZED frames, 2,998 LiDAR scans and 3,024 GPS-RTK fixes.
- **Curated by:** ARCS Lab, University of California Riverside (source release)
- **Funded by:** [More Information Needed]
- **Shared by:** Voxel51 (FiftyOne conversion)
- **Language(s):** Not applicable (sensor data)
- **License:** CC-BY-SA-4.0
### Dataset Sources
- **Repository:** [CitrusFarm project page](https://ucr-robotics.github.io/Citrus-Farm-Dataset/) (source release); [Voxel51/CitrusFarm-Sample](https://huggingface.co/datasets/Voxel51/CitrusFarm-Sample) (this conversion)
- **Paper:** Multimodal Dataset for Localization, Mapping and Crop Monitoring in Citrus Tree Farms (International Symposium on Visual Computing, 2023)
- **Demo:** [More Information Needed]
## Uses
### Direct Use
Localization, mapping and crop monitoring in citrus tree farms, using the thermal, multispectral, stereo, depth, LiDAR, inertial and GPS-RTK streams recorded on one shared clock, with the release’s ground-truth trajectory as a position reference.
### Out-of-Scope Use
[More Information Needed]
## Dataset Structure
**Topology**
An ungrouped FiftyOne dataset with `media_type="multimodal"`. One sample is one sequence, and its `filepath` is a `.fo.mcap` file (`data/06_14B_Jackal.fo.mcap`). There is 1 sample, no sample tags, and an empty `dataset.info`. The dataset has no FiftyOne label fields; every signal lives inside the MCAP file as a channel and is shown by the App’s multimodal viewer.
**Sample fields**
| Field | FiftyOne type | Description |
|------------------------------------------------------------------------|-----------------|----------------------------------------------------|
| `id`, `filepath`, `tags`, `metadata`, `created_at`, `last_modified_at` | built-in | Standard FiftyOne sample fields |
| `sequence` | `StringField` | Sequence name, `06_14B_Jackal` |
| `field` | `StringField` | The farm field the sequence was recorded in, `14B` |
| `recorded` | `StringField` | Recording date, `2023-07-18` |
| `duration` | `FloatField` | Sequence length in seconds |
| `num_mono_frames` | `IntField` | Frames from the Blackfly monochrome camera |
| `num_thermal_frames` | `IntField` | Frames from the ADK thermal camera |
| `num_rgn_frames` | `IntField` | Frames from the Survey3 red-green-NIR camera |
| `num_zed_frames` | `IntField` | Frames from the ZED color images |
| `num_depth_frames` | `IntField` | Frames of ZED depth |
| `num_lidar_scans` | `IntField` | Velodyne LiDAR scans |
| `num_lidar_points` | `IntField` | Points across all LiDAR scans |
| `num_gnss_fixes` | `IntField` | GPS-RTK fixes |
| `valid_depth_fraction` | `FloatField` | Share of ZED depth pixels holding a value |
| `ground_truth_path_m` | `FloatField` | Length of the ground-truth trajectory in metres |
**Sample values**
| Field | Value |
|------------------------|-----------------|
| `sequence` | `06_14B_Jackal` |
| `field` | `14B` |
| `recorded` | `2023-07-18` |
| `duration` | 302.535 s |
| `num_mono_frames` | 3,026 |
| `num_thermal_frames` | 3,024 |
| `num_rgn_frames` | 3,025 |
| `num_zed_frames` | 3,022 |
| `num_depth_frames` | 3,023 |
| `num_lidar_scans` | 2,998 |
| `num_lidar_points` | 76,519,363 |
| `num_gnss_fixes` | 3,024 |
| `valid_depth_fraction` | 0.4703 |
| `ground_truth_path_m` | 356.6 |
**Episode contents**
Each MCAP episode contains these channels:
| Channel | Schema or content |
|------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|
| `/mono` | Blackfly monochrome camera at 1440x1080, `foxglove.CompressedVideo` |
| `/thermal` | ADK thermal camera at 640x512, `foxglove.CompressedVideo` |
| `/rgn` | Survey3 camera’s red, green and near-infrared channels at 1280x720, `foxglove.CompressedVideo` |
| `/zed-left`, `/zed-right` | The ZED’s rectified color images at 1280x720, `foxglove.CompressedVideo` |
| `/zed-depth` | The ZED’s depth registered to its left image, `foxglove.CompressedImage` (16-bit PNG, millimetres, 0 where the ZED gives no depth) |
| `-calibration` | A calibration topic beside each camera, `foxglove.CameraCalibration` |
| `/lidar-points` | Velodyne scans, `foxglove.PointCloud` with `x`, `y`, `z`, `intensity` and `ring` |
| `/imu.plot` | Microstrain inertial unit |
| `/gnss` | GPS-RTK fixes, `foxglove.LocationFix` |
| `/wheel-odometry`, `/zed-odometry` | The robot’s and the ZED’s own pose estimates, `foxglove.PoseInFrame`, each with its position on a `.plot` channel |
| `/ground-truth.plot` | The release’s ground-truth trajectory |
| `/tf` | The sensors’ poses from the release’s calibration, `foxglove.FrameTransform` |
| `/sequence` | Names the sequence and its field |
The episode also carries the metadata fields `sequence`, `field`, `recorded`, `duration`, `num_mono_frames`, `num_thermal_frames`, `num_rgn_frames`, `num_zed_frames`, `num_depth_frames`, `num_lidar_scans`, `num_lidar_points`, `num_gnss_fixes`, `valid_depth_fraction` and `ground_truth_path_m`, which are the sample fields above.
**Parsing decisions**
- **Why MCAP:** the dataset is one multimodal sample per sequence, so the five cameras, LiDAR, inertial unit, GPS-RTK fixes and poses play back on one shared clock instead of being split into per-frame samples.
- **Reading the bags:** the release splits each sequence into ROS 1 bags by sensor group and by size; every bag of the sequence was read together, in time order, without a ROS install.
- **Video encoding:** the cameras are re-encoded to Annex-B H.264 without B-frames, one access unit per frame, on each frame’s own timestamp.
- **Depth:** the release records the ZED depth as 32-bit floats in metres, which are carried as 16-bit PNG in millimetres, rounded, with 0 where the ZED reports none.
- **Calibration:** the calibration is the release’s: Kalibr’s intrinsics for the five cameras, and its extrinsics chained through the cameras, the inertial unit, the LiDAR, the GPS antenna and the robot’s base. Kalibr’s camera chain and its camera-to-IMU result map points from one frame into the next, so `/tf` carries their inverse, each frame’s pose in its parent; the LiDAR results are carried as the release gives them, the child’s pose in the parent.
- **Ground truth:** the ground-truth file gives positions only, so it is carried as a position track.
- **Left out:** the ZED’s confidence map and inertial unit, both magnetometers and the GPS receiver’s state are not carried.
## Dataset Creation
### Curation Rationale
[More Information Needed]
### Source Data
#### Data Collection and Processing
Recorded on 2023-07-18 in field 14B of the University of California Riverside’s Agricultural Experimental Station, with a Clearpath Jackal carrying the cameras, LiDAR, inertial unit and GPS-RTK receiver described above. The FiftyOne conversion reads the ROS 1 bags without a ROS install and writes one MCAP episode for the sequence; see Parsing decisions above for the changes made.
#### Who are the source data producers?
The ARCS Lab at the University of California Riverside.
### Annotations
#### Annotation process
The dataset has no FiftyOne label fields. The ground-truth trajectory in the episode is the release’s own; how it was produced is [More Information Needed].
#### Who are the annotators?
[More Information Needed]
#### Personal and Sensitive Information
[More Information Needed]
## Citation
The source release is cited as:
**BibTeX:**
```bibtex
@inproceedings{teng2023multimodal,
title={Multimodal Dataset for Localization, Mapping and Crop Monitoring in Citrus Tree Farms},
author={Teng, Hanzhe and Wang, Yipeng and Song, Xiaoao and Karydis, Konstantinos},
booktitle={International Symposium on Visual Computing},
pages={571--582},
year={2023}
}
```
**APA:**
Teng, H., Wang, Y., Song, X., & Karydis, K. (2023). Multimodal dataset for localization, mapping and crop monitoring in citrus tree farms. In *International Symposium on Visual Computing* (pp. 571-582).
## More Information
CitrusFarm is distributed under the Creative Commons Attribution-ShareAlike 4.0 International license (CC-BY-SA-4.0), and this conversion is distributed under the same license.
Changes from the source: one of the release’s seven sequences, converted from ROS 1 bags to the FiftyOne MCAP flavor, H.264 encoding of the five camera streams, the ZED depth carried as 16-bit millimetres, the release’s calibration carried as camera intrinsics and transforms with Kalibr’s inverted, and the confidence map, the ZED inertial unit, the magnetometers and the receiver state left out.
## Dataset Card Authors
[More Information Needed]
## Dataset Card Contact
[More Information Needed]