Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Dataset Card for Pheno4D#

This is a FiftyOne dataset with 223 samples.

image/png

Installation#

pip install -U fiftyone

Usage#

import fiftyone as fo
from huggingface_hub import snapshot_download

# Download the dataset snapshot to the current working directory
snapshot_download(
    repo_id="Voxel51/pheno4d",
    local_dir=".",
    repo_type="dataset",
)

# Load dataset from current directory using FiftyOne's native format
dataset = fo.Dataset.from_dir(
    dataset_dir=".",  # Current directory contains the dataset files
    dataset_type=fo.types.FiftyOneDataset,  # Specify FiftyOne dataset format
    name="Pheno4D",  # Assign a name to the dataset for identification
)

# Launch the App
session = fo.launch_app(dataset)

Dataset Details#

Dataset Description#

Pheno4D is a spatio-temporal 3D point cloud dataset of 7 maize plants (scanned daily over 12 days) and 7 tomato plants (scanned daily over 20 days), grown in pots in a greenhouse and measured with a sub-millimeter-accuracy laser triangulation scanner (Perceptron ScanWorks V5 mounted on a ROMER Infinite 2.0 measuring arm, σ ≈ 0.012mm). Each scan is a raw, unstructured 3D surface point cloud covering the whole potted plant, including soil — there is no color/intensity channel, only XYZ geometry. A subset of scans is manually annotated with temporally consistent per-point instance labels: every individual leaf keeps the same numeric ID across the whole time series for a given plant, and maize additionally ships two independent, parallel labeling conventions for the same scans (the Leaf Collar Method and the Leaf Tip Method — see Dataset Structure below).

Curated by: David Schunck, Federico Magistri, Radu Alexandru Rosu, André Cornelißen, Nived Chebrolu, Stefan Paulus, Jens Léon, Sven Behnke, Cyrill Stachniss, Heiner Kuhlmann, Lasse Klingbeil (Geodesy Lab and Photogrammetry & Robotics Lab, University of Bonn; Institute of Sugar Beet Research, University of Göttingen; INRES Plant Breeding, University of Bonn)
Funded by: Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy, EXC-2070 – 390732324 – PhenoRob
Shared by: [More Information Needed]
Language(s): N/A (3D point cloud data, no text)
License: Not explicitly stated for the point cloud data on the dataset’s website or in the paper’s Data Availability Statement. The paper itself is published under a Creative Commons Attribution License (CC BY), but this covers the article text, not necessarily the dataset files. Contact the dataset maintainer (Lasse Klingbeil, per the dataset website) to confirm terms before redistribution.

Dataset Sources#

Repository: https://www.ipb.uni-bonn.de/data/pheno4d/index.html
Paper: Schunck D, Magistri F, Rosu RA, Cornelißen A, Chebrolu N, Paulus S, Léon J, Behnke S, Stachniss C, Kuhlmann H, Klingbeil L. “Pheno4D: A spatio-temporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis.” PLOS ONE 16(8): e0256340, 2021. https://doi.org/10.1371/journal.pone.0256340

Uses#

Direct Use#

The source paper demonstrates the following uses of this data, which the FiftyOne dataset supports:

Semantic segmentation of raw point clouds into soil/stem/leaf classes
Instance segmentation of individual plant organs (distinguishing one leaf from another), for computing phenotypic traits like leaf area, leaf length, and leaf angle
Spatio-temporal point cloud registration — associating and non-rigidly registering the same plant’s point clouds across different scan days to track growth
Surface reconstruction of plant point clouds into meshes
Phenotyping — tracking traits such as leaf area, leaf length, stem length, and stem diameter over the measurement period

Out-of-Scope Use#

The plants were grown in pots in a controlled greenhouse environment and scanned with a stationary, tripod-mounted laser arm — this data is not representative of field-grown crops, outdoor conditions, or data captured from mobile/aerial platforms.
Only two species (maize and tomato) at early growth stages (first two-to-three weeks after sprouting) are covered; the dataset should not be assumed to generalize to other crops, later growth stages, or reproductive/fruiting stages.
Leaf instance IDs are only meaningful within a single plant’s own time series (e.g. leaf ID 5 on Maize01 is a different physical leaf than ID 5 on Maize02) — they should never be treated as a shared class taxonomy across plants.

Dataset Structure#

This is a flat (non-grouped) FiftyOne point cloud dataset: media_type="point-cloud", 223 samples, one sample per source scan. Each sample’s filepath points to a binary .pcd file converted from the original ASCII .txt point cloud released by the dataset authors.

Media breakdown:

83 maize samples (Maize01–Maize07, one plant missing one extra scan day beyond the dataset-wide gap — see Parsing decisions), 49 of which are labeled
140 tomato samples (Tomato01–Tomato07), 77 of which are labeled

No splits are defined by the source authors; no tags are set.

Fields#

Field	FiftyOne type	Description
`species`	`StringField`	`"maize"` or `"tomato"`
`plant_id`	`StringField`	Which of the 7 individual plants per species this scan belongs to, e.g. `"Maize01"`, `"Tomato03"`
`scan_date`	`StringField`	Raw `MMDD` scan-date code parsed from the source filename (verbatim from source, e.g. `"0313"`)
`day_index`	`IntField`	1-based chronological index of this scan date across the full experiment (accounts for the one calendar day skipped for all plants)
`is_labeled`	`BooleanField`	Whether this scan has manual per-point instance labels
`num_points`	`IntField`	Number of points in this cloud
`original_bbox_center`	`ListField(FloatField)`	The `[x, y, z]` center of this cloud’s bounding box in the original scanner/measuring-arm coordinate frame, before recentering (see Parsing decisions)
`has_collar_labels`	`BooleanField`	Maize only; whether the Leaf Collar Method labels are present (`None` for unlabeled scans)
`has_tip_labels`	`BooleanField`	Maize only; whether the Leaf Tip Method labels are present (`None` for unlabeled scans)
`ground_truth`	`Detections`	Tomato labeled scans only (77 samples, 1,060 detections total): one 3D box per `stem`/`leaf_<id>` instance
`ground_truth_collar`	`Detections`	Maize labeled scans only (49 samples, 141 detections total): one 3D box per `stem`/`leaf_<id>` instance, per the Leaf Collar Method
`ground_truth_tip`	`Detections`	Maize labeled scans only (49 samples, 190 detections total): one 3D box per `leaf_<id>` instance (no `stem` class), per the Leaf Tip Method

Each Detection in the three ground_truth* fields additionally carries:

location, dimensions, rotation — an axis-aligned 3D bounding box (rotation is always [0, 0, 0]) computed as the min/max extent of that instance’s points
raw_label_id — the original integer per-point label value from the source file, for tracing back to the raw annotation
num_points — the number of points belonging to that instance

Label types and why#

The source annotation is dense, per-point semantic/instance labeling on raw, unstructured 3D point clouds (each labeled point carries an integer class/instance ID inline in the same file as its XYZ coordinates) — there is no native FiftyOne label type for dense 3D point-level labels analogous to Segmentation for images. Two complementary representations are used here:

The raw per-point label values are preserved exactly as custom scalar fields (label for tomato; label_collar and label_tip for maize) embedded directly in each sample’s .pcd file. These are viewable in the FiftyOne App’s 3D visualizer via the “Shade By” render preference, which dynamically detects and lets you color by any scalar PCD attribute — this is the only place the exact per-point values live; they are not a queryable FiftyOne sample field.
Detections of axis-aligned 3D bounding boxes (one per non-soil instance — stem and each leaf_<id>) were derived from those per-point labels and attached as ground_truth / ground_truth_collar / ground_truth_tip sample fields. This is a lossy simplification (a box, not the exact point membership) but makes the labels queryable and filterable through FiftyOne’s standard label API (e.g. filter_labels, count, sorting by num_points).

Soil (label 0) is excluded from all three Detections fields in both representations above, since it represents the pot and background surface rather than a plant organ.

Saved views#

36 saved views are defined on the dataset for convenient browsing:

plant_<plant_id> (14 views, one per plant) — that single plant’s samples across all its scan days, sorted by day_index, for viewing growth progression over time
day_<scan_date> (20 views, one per calendar scan date) — every plant (of either species) scanned on that date
species_maize / species_tomato (2 views) — all samples of one species

`dataset.info`#

Empty — no dataset-level provenance/calibration metadata is stored (the point cloud coordinate frame is local to each plant’s own measurement session; there is no shared multi-plant coordinate system or sensor calibration data to record).

Parsing decisions#

Format conversion: the original ASCII .txt files (x y z [label] [label_tip], space-separated, no header) were converted to binary .pcd files with an explicit FIELDS/TYPE/COUNT header, preserving the label column(s) as additional scalar PCD fields rather than discarding them.
Coordinate recentering: the raw scanner coordinates sit roughly 700mm from the origin (the fixed position of the measuring arm relative to the mounted pot). Each cloud’s coordinates were recentered on its own bounding-box center at conversion time so that FiftyOne’s 3D viewer default camera position actually shows the point cloud; the original center is preserved in original_bbox_center for anyone who needs the original scanner frame.
File-to-field mapping: the source filenames encode both the annotation status and the (maize) label scheme via column count — <code>_<date>.txt (3 columns, unlabeled), <code>_<date>_a.txt for tomato (4 columns: x y z label) and for maize (5 columns: x y z label_collar label_tip). This was parsed directly from the file content (number of whitespace-separated columns), not just the filename suffix.
day_index construction: the dataset-wide scan calendar has one day with no measurements at all for any plant (day 0323); day_index is a 1-based rank over the actual observed scan dates, not a literal day-of-experiment count, so it stays contiguous.
Coverage gap: Maize03 is missing one additional scan (0318) beyond the dataset-wide gap — this plant has 11 scans where the other 6 maize plants have 12. This is a genuine gap in the source data, not a parsing artifact.
Grouped-dataset vs. flat: although the 3D-community-standard pattern for multi-slice 3D data is FiftyOne’s grouped-dataset (media-groups) feature, this dataset intentionally does not use it. Each scan is one independent point cloud (no co-registered multi-sensor/multi-view content per moment to justify grouping), and FiftyOne only permits a single fo3d (3D scene) group slice per grouped dataset in any case — incompatible with having 14 distinct per-plant slices. The flat structure plus the saved views above provides the same “browse by plant” / “browse by day” / “browse by species” functionality without that constraint.
Why plain .pcd and not .fo3d scenes: fo.Scene/.fo3d wrapping would only have added scene-level camera/material customization, not different label handling — the same custom-scalar-field “Shade By” coloring works identically on a bare .pcd sample. Coordinate recentering (see above) achieves the same practical visibility outcome without the scene-wrapping.
No RGB/intensity: the source sensor (a 660nm laser triangulation scanner) captures geometry only; there is no color or intensity channel in any source file, so none is present in the converted PCDs either.

Dataset Creation#

Curation Rationale#

At the time of the paper’s publication, highly accurate, temporally-repeated 3D point clouds of plants were rare and expensive to acquire, while most existing agricultural computer vision datasets were 2D images. The authors aimed to provide a large-scale, sub-millimeter-accuracy, multi-temporal 3D dataset with dense manual instance labels to support machine learning research on plant organ instance segmentation, 3D reconstruction, non-rigid registration, and growth-trait phenotyping — tasks that otherwise require researchers to collect and label their own costly 3D plant data before they can even begin methods work.

Source Data#

Data Collection and Processing#

Plants were grown in pots in a greenhouse at Campus Klein-Altendorf (University of Bonn) and scanned daily shortly after the first sprouts appeared: 7 maize plants over a 12-day period and 7 tomato plants over a 20-day period (one calendar day within each period, day 11 for maize / day 19 for tomato, had no measurements for any plant). Each plant was scanned using a Perceptron ScanWorks V5 laser triangulation scanner (660nm, up to 7,640 points per scan line, σ = 0.012mm per-point accuracy) mounted on a ROMER Infinite 2.0 articulated measuring arm (spherical measurement volume of 1.4m radius, arm-tip accuracy of 45μm), scanning the plant from multiple positions/angles to register a complete 3D point cloud in the arm’s local coordinate frame. Scans took several minutes per plant (up to ~15 minutes for larger, later-stage tomato plants). The only preprocessing applied was manual outlier removal; point clouds still include the pot and surrounding soil surface, and no homogenization/downsampling was performed.

Who are the source data producers?#

The maize and tomato plants themselves were grown and measured by the paper’s authors’ research groups at the University of Bonn (Geodesy Lab; Photogrammetry & Robotics Lab; Autonomous Intelligent Systems Lab) in collaboration with the University of Göttingen’s Institute of Sugar Beet Research and the University of Bonn’s INRES Plant Breeding group.

Annotations#

Annotation process#

Human annotators manually labeled each 3D point in a subset of scans (every second scan day per plant, plus the last two consecutive days of each measurement period) as soil, stem, or an individual leaf, with leaf labels kept consistent for the same physical leaf across the whole time series for a given plant. For tomato, the stem-to-leaf boundary was labeled directly, since it is visually distinguishable where a leaf spreads out from the stem. For maize, where leaves emerge from the whorl without an obvious visual break from the stem, two independent staging-derived labeling conventions were both applied to every labeled maize scan: the Leaf Collar Method (a leaf becomes its own instance once it develops a visible leaf collar; until then it remains part of stem) and the Leaf Tip Method (every visible leaf tip is counted as its own instance from the bottom of the plant upward, with no separate stem class at all).

Who are the annotators?#

Per the paper’s author contributions, data curation was performed by David Schunck and André Cornelißen. The paper does not name additional annotators beyond the author list.

Personal and Sensitive Information#

None. This dataset contains only 3D geometric scans of maize and tomato plants; it contains no personal, human-subject, or sensitive information.

Citation#

BibTeX:

@article{schunck2021pheno4d,
  title={Pheno4D: A spatio-temporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis},
  author={Schunck, David and Magistri, Federico and Rosu, Radu Alexandru and Cornelißen, André and Chebrolu, Nived and Paulus, Stefan and Léon, Jens and Behnke, Sven and Stachniss, Cyrill and Kuhlmann, Heiner and Klingbeil, Lasse},
  journal={PLOS ONE},
  volume={16},
  number={8},
  pages={e0256340},
  year={2021},
  publisher={Public Library of Science},
  doi={10.1371/journal.pone.0256340}
}

APA:

Schunck, D., Magistri, F., Rosu, R. A., Cornelißen, A., Chebrolu, N., Paulus, S., Léon, J., Behnke, S., Stachniss, C., Kuhlmann, H., & Klingbeil, L. (2021). Pheno4D: A spatio-temporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis. PLOS ONE, 16(8), e0256340. https://doi.org/10.1371/journal.pone.0256340

More Information#

An accompanying software API for loading and visualizing the raw data (Python and C++, with subsampling, augmentation, and day/plant selection) is available at https://github.com/AIS-Bonn/data_loaders, and a visualization GUI is available at https://github.com/AIS-Bonn/easy_pbr.

Dataset Card Authors#

Harpreet Sahota