#### NOTE This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs. ![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-yellow) # Dataset Card for TreeScope (MCAP) ![image/png](https://huggingface.co/datasets/Voxel51/treescope-vat0723-multimodal/resolve/main/treescope.gif) This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 10 samples. ## Installation If you haven’t already, install FiftyOne: ```bash pip install -U fiftyone ``` ## Usage ```python import fiftyone as fo from fiftyone.utils.huggingface import load_from_hub # Load the dataset # Note: other available arguments include 'max_samples', etc dataset = load_from_hub("harpreetsahota/treescope-vat0723-multimodal") # Launch the App session = fo.launch_app(dataset) ``` ## Dataset Details ### Dataset Description This dataset is a curated, 10-episode multimodal subset of the **TreeScope v1.0** robotics dataset, repackaged as time-synchronized [MCAP](https://mcap.dev/) recordings for FiftyOne’s native [multimodal dataset support](https://docs.voxel51.com/user_guide/multimodal.html) (introduced in FiftyOne 1.19). Each sample is one continuous UAV flight episode from TreeScope’s `VAT-0723` site (Appomattox-Buckingham State Forest, Virginia), viewable in FiftyOne’s tiled multimodal viewer with synchronized camera, LiDAR point cloud, GPS, IMU, diagnostics, and log playback. TreeScope itself is a LiDAR dataset for precision agriculture and forestry, collected with UAV and mobile robot platforms across six forest and orchard sites, with manually-annotated semantic labels and field-measured tree diameters for benchmarking segmentation and diameter-estimation algorithms. This repackaging uses only the raw sensor streams from one site — it does not carry over TreeScope’s original semantic-segmentation or DBH ground truth (see [Dataset Creation](#dataset-creation)). - **Curated by:** Derek Cheng, Fernando Cladera, Ankit Prabhu, Xu Liu, Alan Zhu, Pratik Chaudhari, and Vijay Kumar (GRASP Laboratory, University of Pennsylvania); P. Corey Green (Virginia Tech, Forest Resources and Environmental Conservation); Reza Ehsani (UC Merced, Mechanical Engineering) — original TreeScope v1.0 data collection. This MCAP/FiftyOne multimodal repackaging (episode merging, tile-coverage curation, dataset card) was prepared independently by Harpreet Sahota. - **Funded by:** IoT4Ag ERC, funded by the National Science Foundation (NSF) under NSF Cooperative Agreement Number EEC-1941529; NIFA grant 2022-67021-36856; NSF grant CCR-2112665; and C-BRIC, a Semiconductor Research Corporation Joint University Microelectronics Program cosponsored by DARPA. - **Shared by:** Harpreet Sahota (this repackaging); the original TreeScope v1.0 dataset is shared by its authors at https://treescope.org. - **Language(s):** N/A (sensor data — LiDAR, imagery, GPS, IMU; no text). - **License:** [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/), inherited from the source TreeScope v1.0 release (non-commercial use only). ### Dataset Sources - **Repository:** https://github.com/KumarRobotics/treescope - **Paper:** [TreeScope: An Agricultural Robotics Dataset for LiDAR-Based Mapping of Trees in Forests and Orchards](https://arxiv.org/abs/2310.02162) (arXiv:2310.02162) - **Demo:** https://treescope.org (original full dataset release) ## Uses ### Direct Use - Exercising and demoing FiftyOne’s multimodal MCAP support: synchronized playback of camera, 3D point cloud, map/GPS, plot, logs, and raw-message tiles across real robotics recordings. - Browsing/inspecting under-canopy UAV forestry LiDAR flights (manual and autonomous) — point clouds, flight trajectories, and (for 6 of the 10 episodes) stereo camera and IMU streams. - Prototyping analyses against the raw sensor streams themselves, e.g. reviewing GPS flight tracks, comparing onboard vs. offline-refined odometry (`/Odometry` vs. `/refined/Odometry`), or plotting IMU/diagnostic time series — using FiftyOne’s per-episode metadata fields to filter which episodes have the streams of interest. ### Out-of-Scope Use - Reproducing TreeScope’s official semantic-segmentation or diameter-at- breast-height (DBH) benchmark results — no ground-truth labels are attached to any of these 10 episodes (see [Dataset Creation](#dataset-creation)). - Treating this as a representative sample of TreeScope v1.0 as a whole — it is 10 of the many `VAT-0723U` episodes at one of TreeScope’s six sites, selected for disk-space and multimodal tile-coverage reasons, not statistical sampling. - Commercial use, which is precluded by the source CC BY-NC-SA 4.0 license. ## Dataset Structure This is a flat (ungrouped) FiftyOne dataset with `media_type: "multimodal"` and **10 samples**. Each sample is one **episode** — a single continuous UAV flight recording — stored as one merged `.mcap` file; FiftyOne infers the multimodal media type automatically from the `.mcap` file extension. There are no video frames or point-cloud files as separate FiftyOne samples: the episode is the sample unit, and every stream inside it (camera, LiDAR, GPS, IMU, logs, etc.) is decoded live by FiftyOne’s multimodal viewer. Per-sample tags mark flight mode: `manual_flight` (4 samples) or `autonomous_flight` (6 samples). The dataset itself carries no dataset-level tags, and `dataset.info` is empty (no extra dataset-level metadata beyond the per-sample fields below). ### Fields | Field | FiftyOne type | Description | |---------------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `filepath` | `StringField` | Absolute path to the episode’s merged `.mcap` file — the sample’s multimodal media | | `tags` | `ListField(StringField)` | Flight-mode tag: `manual_flight` or `autonomous_flight` | | `site` | `StringField` | TreeScope site name; always `"VAT-0723"` in this subset | | `session_id` | `StringField` | TreeScope’s original session identifier (e.g. `VAT-0723U-AUTO-01`), verbatim from `metadata/VAT-0723.json` | | `platform` | `StringField` | Flight mode + platform, `"ULS Manual"` or `"ULS Autonomous"` (ULS = UAV Laser Scanning), verbatim from source metadata’s `attributes.type` | | `lidar_model` | `StringField` | LiDAR sensor model; always `"OS1-64"` (Ouster OS1-64) for this UAV subset | | `description` | `StringField` | One-line, human-written summary of the episode (source bag identity + notable sensor coverage) | | `duration_s` | `FloatField` | Episode duration in seconds, computed from the merged MCAP’s message-time span | | `message_count` | `IntField` | Total MCAP message count across all channels in the episode | | `channel_count` | `IntField` | Total MCAP channel (topic) count in the episode | | `topics` | `ListField(StringField)` | Every ROS topic name present in the episode’s MCAP | | `schemas` | `ListField(StringField)` | Every distinct ROS message schema name present in the episode’s MCAP | | `has_image` | `BooleanField` | Whether the episode has a camera stream FiftyOne’s Image tile can decode (schema-based: `sensor_msgs/msg/(Compressed)Image`) | | `has_pointcloud` | `BooleanField` | Whether it has a decodable point-cloud stream for the 3D tile (`sensor_msgs/msg/PointCloud2`) | | `has_gps` | `BooleanField` | Whether it has a decodable GPS fix stream for the Map tile (`sensor_msgs/msg/NavSatFix`) | | `has_imu` | `BooleanField` | Whether it has a decodable IMU stream for the Plot tile (`sensor_msgs/msg/Imu`) | | `has_logs` | `BooleanField` | Whether it has a decodable log stream for the Logs tile (`rosgraph_msgs/msg/Log`) | | `has_unrecognized_schema` | `BooleanField` | Whether the episode also contains a schema with no built-in FiftyOne decoder (Message-tile-only, e.g. raw Ouster LiDAR packets, custom quadrotor autonomy messages) | Standard FiftyOne bookkeeping fields (`id`, `metadata`, `created_at`, `last_modified_at`) are also present but not source-specific. ### Label types and why **No FiftyOne label fields are attached to this dataset.** The actual multimodal content — camera images, LiDAR point clouds, GPS tracks, IMU readings, and logs — lives entirely inside each sample’s `.mcap` file and is decoded live by FiftyOne’s multimodal viewer (Image, 3D, Map, Plot, Logs, and Message tiles), not represented as separate `Detection`/`Segmentation`/etc. label objects. The `has_*` boolean fields above exist purely so users can filter or query episodes by which tiles they’ll populate, without opening every MCAP file first — e.g. `dataset.match(F("has_image") & F("has_imu"))`. A saved view, **`full_multimodal_showcase`**, selects the 9 of 10 episodes that have every tile type populated with real, decodable data (Image, 3D, Map, Plot-capable IMU, Logs, Message); the excluded episode (`VAT-0723U-01`) genuinely lacks camera and decoded-IMU topics, kept in the full dataset for realistic sensor-coverage heterogeneity. ### Schemas present across episodes Every episode has, at minimum: `nav_msgs/msg/Odometry`, `sensor_msgs/msg/PointCloud2` (multiple point-cloud topics — raw segmented, tree-only, ground-only, and world-frame variants), `sensor_msgs/msg/NavSatFix`, `tf2_msgs/msg/TFMessage`, `rosgraph_msgs/msg/Log`, `diagnostic_msgs/msg/DiagnosticArray`, and undecoded `ouster_ros/msg/PacketMsg` (raw LiDAR/IMU packets, Message-tile only). The 9 richer episodes additionally have `sensor_msgs/msg/CompressedImage` (stereo cameras), `sensor_msgs/msg/Imu` and `MagneticField` (decoded IMU/mag), and custom autonomy schemas (`kr_mav_msgs/msg/PositionCommand`, `planning_ros_msgs/msg/{VoxelMap,Path,Trajectory}`) from the autonomous flights’ quadrotor planner — all Message-tile fodder, since these are non-standard schemas with no built-in FiftyOne decoder. ### Parsing decisions - **One sample = one episode.** Each sample corresponds to a single, continuous flight recording, never split into per-frame or per-message samples — FiftyOne’s multimodal viewer handles playback within an episode. - **Each episode’s MCAP is a merge of that session’s raw + processed ROS1 bag pair**, not a straight conversion of one bag, because: - Both bags independently publish `/Odometry` and `/tf` — the raw bag’s onboard/live estimate vs. the processed bag’s offline [Faster-LIO](https://github.com/hku-mars/FASTER-LIO)-refined trajectory. The processed bag’s copies are remapped to `/refined/Odometry` and `/refined/tf` so both trajectories stay distinguishable rather than colliding into one interleaved channel. - Both bags also independently publish `/ublox/fix` and `/ublox/fix_velocity` as byte-identical pass-through copies. Unlike the odometry case there is no “refined” GPS variant to preserve, so the processed bag’s copies are dropped entirely — each GPS fix appears exactly once in the merged MCAP. - `std_msgs/msg/Header` in the shared destination typestore is explicitly overridden with the canonical ROS 2 definition, because ROS1’s `Header` carries an extra `uint32 seq` field that silently corrupts CDR offset/size calculations for any nested-Header message type if copied verbatim from the ROS1 side. - The processed bag’s message timestamps are rebased by a constant offset (`raw_bag.start_time - processed_bag.start_time`), because they carry the wall-clock time of the *offline reprocessing job* (recorded weeks to months after the actual flight) rather than the flight’s capture time. Left uncorrected, an episode would nominally span that entire gap with real data in only two tiny slivers — unusable for scrubbing/playback. - **Candidate episodes were verified before download**, not after: a session’s raw bag is inspected via HTTP `Range` requests against its ROS1 bag-format connection records (topics + schemas, ~100 KB fetched instead of gigabytes) before committing to the full download. This also detects corrupted/unindexed bags — one candidate, `VAT-0723U-06`, was excluded this way: its bag header’s declared index position lies beyond the file’s actual size, indicating a split/concatenated recording unparseable without a full download. - **No ground truth is attached to any of these 10 episodes, deliberately.** The site’s semantic-segmentation labels (`ground_truth/labels/VAT-0723U.h5`) were downloaded and checked, but their labeled-frame timestamps fall entirely outside every chosen episode’s actual flight window; `metadata/VAT-0723.json` independently confirms `attributes.semantic_labels=false` and `attributes.dbh=false` for all 10 sessions. FiftyOne’s temporal tags were skipped for the same reason — nothing here is faked to look annotated. - **This is a small subset of one TreeScope site.** 10 of the (more numerous, and individually much larger) `VAT-0723U` episodes, at one of TreeScope’s six sites. Selection was constrained by local disk space and by wanting genuine multimodal tile-coverage diversity, not by scientific sampling. FiftyOne Enterprise-only multimodal features (MCAP indexing, projections, derived event/label timeline tracks) are out of scope entirely — this dataset only exercises open-source FiftyOne capability. ## Dataset Creation ### Curation Rationale `VAT-0723` was chosen as the only confirmed TreeScope site with GPS (`NavSatFix`) data released, making FiftyOne’s Map tile achievable faithfully (GPS hardware is mentioned in the TreeScope paper but was found missing from the released bags at other inspected sites). Its raw+processed bag pairs are also small enough, relative to TreeScope’s other sites (hundreds of GB to ~1 TB each), to download a meaningful multi-episode subset without approaching the full ~760 GB site or ~2.2 TB dataset. The selection and MCAP-merge pipeline were built to exercise every non-Enterprise FiftyOne multimodal viewer tile (Image, 3D, Map, Plot, Logs, Message) with real, decodable data — not to support TreeScope’s original segmentation or DBH benchmarks. ### Source Data #### Data Collection and Processing Per the TreeScope paper: `VAT-0723`’s UAV (ULS) data was collected with the **Falcon 4 UAV** (4.2 kg, up to 30 minutes of under-canopy flight), equipped with an **Ouster OS1-64 LiDAR** (Rev 6; 64 vertical channels, 1024 horizontal points, 120 m range, 45° vertical FoV), an Open Vision Computer (stereo cameras), a VectorNav VN-100 IMU, and a UBlox ZED-F9P GPS, flown over intensively-managed loblolly pine plots in the Appomattox-Buckingham State Forest, Virginia. Raw ROS1 bags contain the onboard sensor streams as captured; processed ROS1 bags contain [Faster-LIO](https://github.com/hku-mars/FASTER-LIO)-derived lidar-inertial odometry and velocity-corrected point-cloud sweeps, plus a [RangeNet++](https://github.com/PRBonn/lidar-bonnetal)-based semantic ground/tree-stem split. For this repackaging, each episode’s raw and processed ROS1 bag were merged into one `.mcap` file using the [`rosbags`](https://ternaris.gitlab.io/rosbags/) Python library (see [Parsing decisions](#parsing-decisions) for the exact transforms applied), then loaded into FiftyOne with the per-sample metadata fields described above. No sensor data was synthesized or altered beyond the topic remapping/dropping and timestamp rebasing documented above. #### Who are the source data producers? The GRASP Laboratory, University of Pennsylvania (Falcon 4 UAV platform and data collection), for the original TreeScope v1.0 release. ### Annotations #### Annotation process None. No semantic-segmentation labels, DBH ground truth, or temporal tags are attached to any of these 10 episodes — see [Parsing decisions](#parsing-decisions) for why. #### Who are the annotators? N/A — this subset carries no annotations. #### Personal and Sensitive Information None identified. The GPS stream in each episode records the UAV’s flight path over a forest research site, not any human or personally identifiable location. ## Citation **BibTeX:** ```bibtex @misc{cheng2023treescope, title={TreeScope: An Agricultural Robotics Dataset for LiDAR-Based Mapping of Trees in Forests and Orchards}, author={Derek Cheng and Fernando Cladera and Ankit Prabhu and Xu Liu and Alan Zhu and P. Corey Green and Reza Ehsani and Pratik Chaudhari and Vijay Kumar}, year={2023}, eprint={2310.02162}, archivePrefix={arXiv}, primaryClass={cs.RO} } ``` **APA:** Cheng, D., Cladera, F., Prabhu, A., Liu, X., Zhu, A., Green, P. C., Ehsani, R., Chaudhari, P., & Kumar, V. (2023). TreeScope: An Agricultural Robotics Dataset for LiDAR-Based Mapping of Trees in Forests and Orchards. *arXiv preprint arXiv:2310.02162*. ## More Information This repository is an independently-curated, derived subset of the official TreeScope v1.0 release, repackaged as MCAP for FiftyOne’s multimodal support. It is not an official TreeScope artifact. For the full dataset (all six sites, raw and processed bags, semantic labels, and DBH ground truth), see https://treescope.org and https://github.com/KumarRobotics/treescope. ## Dataset Card Authors Harpreet Sahota ([@harpreetsahota](https://huggingface.co/harpreetsahota)) — MCAP repackaging and this card. Original dataset authors are listed under [Dataset Description](#dataset-description). ## Dataset Card Contact Harpreet Sahota — https://huggingface.co/harpreetsahota