#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# Dataset Card for TomatoMAP
This is a [FiftyOne](https://github.com/voxel51/fiftyone) dataset with 68,069 samples.
## Installation
If you haven’t already, install FiftyOne:
```bash
pip install -U fiftyone
```
## Usage
```python
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("Voxel51/tomato-map")
# Launch the App
session = fo.launch_app(dataset)
```
## Dataset Details
### Dataset Description
TomatoMAP is a multi-angle, multi-pose, multi-task imaging dataset of *Solanum
lycopersicum* (tomato) collected with a purpose-built IoT imaging station at
the Julius Kühn-Institute (JKI), Quedlinburg, Germany. It unifies three
originally-separate TomatoMAP subsets – **TomatoMAP-Cls** (BBCH growth-stage
classification), **TomatoMAP-Det** (region-of-interest object detection), and
**TomatoMAP-Seg** (fruit/flower developmental-stage instance segmentation) –
into a single grouped FiftyOne dataset, tagged by source.
101 tomato plants (all the “Money Maker” accession, non-transgenic) were
imaged over a 163-day period by 4 fixed Raspberry Pi cameras (mounted at
45/90/135/180 degree elevation angles) while a turntable rotated the plant
through 12 poses (30 degree increments), giving 48 synchronized images per
plant per acquisition session – 64,464 images in total, each with a 7-class
ROI bounding-box annotation and a session-level BBCH growth-stage label (50
distinct codes). A separate, later imaging pass with a handheld macro camera
produced 3,605 high-resolution close-up photographs of individual flower buds
and fruit clusters, 727 of which have ISAT-tool instance segmentation masks
across 10 fruit/flower developmental-stage classes.
- **Curated by:** Yujie Zhang, Sabine Struckmeyer, Andreas Kolb, Sven Reichardt (Julius Kühn-Institute & University of Siegen); parsed into FiftyOne format by Harpreet Sahota.
- **Funded by:** German Federal Ministry of Agriculture, Food and Regional Identity; compute powered by the de.NBI Cloud / ELIXIR-DE.
- **Shared by:** Harpreet Sahota (FiftyOne format on Hugging Face); original data deposited by the authors in e!DAL (IPK Gatersleben).
- **Language(s):** en (metadata, class names, and the BBCH index descriptions are in English)
- **License:** CC BY 4.0
### Dataset Sources
- **Repository:** [e!DAL / IPK Gatersleben](https://doi.org/10.5447/ipk/2025/14) (raw data); [github.com/0YJ/TomatoMAP](https://github.com/0YJ/TomatoMAP) (original TomatoMAP-Cls/Det builder + model code)
- **Paper:** [Zhang et al., “Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping,” Scientific Data (2026) 13:309](https://doi.org/10.1038/s41597-026-06926-9)
- **Demo:** [0yj.github.io/tomato_map](https://0yj.github.io/tomato_map/)
## Uses
### Direct Use
- Training/benchmarking fine-grained BBCH phenological growth-stage classifiers.
- Training/benchmarking object detectors for plant ROIs (leaf, whole plant, panicle, flower clusters, fruit clusters, axillary shoot, shoot), including studying class imbalance and overlapping-box handling.
- Training/benchmarking instance/semantic segmentation of fruit ripening stages (nascent -> mini -> unripe -> semi ripe -> fully ripe) and flower/bud size stages (2mm -> 4mm -> 6mm -> 8mm -> 12mm).
- Multi-view / sparse-view 3D reconstruction research: the 4 rig cameras have known intrinsics (`data/calibration/pi{1-4}.npz`) and nominal elevation/turntable angles, though no extrinsic rig calibration is provided.
- Longitudinal growth-stage analysis using `capture_datetime` and `plant_id` across the 163-day acquisition window.
### Out-of-Scope Use
- Disease/pathogen detection: all plants are healthy specimens of a single cultivar (“Money Maker”), non-transgenic – there is no disease-state variation to learn from (unlike e.g. PlantVillage or Tomato-Village).
- Cross-referencing individual plants or sessions between the det/cls samples and the seg samples: they come from different camera systems, different (only lightly overlapping) capture windows, and carry no shared plant/session identifier – see `dataset.info["seg_source"]` for the evidence. Do not assume a `plant_id`-level relationship between the two subsets.
- Genotype/accession comparison studies: every plant in the det/cls subset is the same accession, grown in the same cabin, so there is no genetic or environmental variation encoded in the metadata.
## Dataset Structure
This is a single **grouped** FiftyOne dataset (`media_type="group"`, group field `group`) with **68,069 samples across 19,721 groups**, unifying two source subsets that are tagged but not otherwise sample-joinable (see Out-of-Scope Use):
- **`det` (64,464 samples, tag `"det"`)** – the TomatoMAP-Cls/Det rig images. Grouped by `image_set_id` (one turntable pose held still, shot simultaneously by the 4 fixed cameras), with slices `"pos_1"`..`"pos_4"` corresponding to the 4 camera elevations. 16,116 groups x 4 slices = 64,464 samples.
- **`seg` (3,605 samples, tag `"seg"`)** – the TomatoMAP-Seg macrophotographs. Each image is its own group (no natural grouping partner), populating a single slice `"macro"`. Further tagged `"labeled"` (727 samples with ISAT annotations) or `"unlabeled"` (2,878 samples, bonus un-annotated photos).
### Fields
| Field | FiftyOne type | Subset | Description |
|--------------------------------------------------------------------|---------------------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `filepath` | `StringField` | both | Path to the image file |
| `tags` | `ListField(StringField)` | both | `"det"`/`"seg"` (source subset) plus `"labeled"`/`"unlabeled"` (seg only) |
| `group` | `Group` | both | FiftyOne group field; `.name` is the slice (`pos_1`..`pos_4` or `macro`) |
| `metadata` | `ImageMetadata` | both | Standard FiftyOne image metadata (width/height/size/mime type) |
| `ground_truth` | `Detections` | both | Det: 7-class YOLO ROI boxes. Seg: instance masks over 10 fruit/flower-stage classes (each `Detection` also carries `isat_group` (int instance id), `area_px`, `iscrowd`). One shared field name since the two class vocabularies never overlap. |
| `classification` | `Classification` | det | BBCH growth-stage label, e.g. `"bbch_70"` – one label per (`plant_id`, capture date) session, inherited by all 48 images from that session |
| `bbch_stage` | `IntField` | det | Raw BBCH code (13-89; 50 distinct codes appear in the data) |
| `bbch_description` | `StringField` | det | Human-readable description of the BBCH code |
| `image_set_id` | `IntField` | det | Groups the 4 simultaneous camera exposures of one turntable pose (verbatim from `metadata/raw_pheno.csv` / parsed from filename) |
| `plant_id` | `IntField` | det | Plant identifier, 1-101 |
| `piid` | `IntField` | det | Rig camera/position id, 1-4 |
| `pose_id` | `IntField` | det | Turntable pose id, 1-12 |
| `pose_degrees` | `IntField` | det | Turntable rotation in degrees (0-330, 30 degree steps) |
| `camera_label` | `StringField` | det | Human-readable camera description, e.g. `"bottom (45 deg)"` |
| `camera_angle_deg` | `IntField` | det | Camera elevation angle code (verbatim from `metadata/camera.csv`) |
| `camera_position_desc` | `StringField` | det | Camera position description (verbatim from `metadata/camera.csv`) |
| `accession` | `StringField` | det | Plant accession (constant: `"Money Maker"`) |
| `transgene` | `BooleanField` | det | Transgenic status (constant: `False`) |
| `cabin` | `StringField` | det | Greenhouse cabin id (constant: `"H01504"`) |
| `capture_datetime` | `DateTimeField` | both | Capture timestamp – det: parsed from the filename’s 14-digit timestamp; seg: parsed from the image’s EXIF `DateTimeOriginal` |
| `f_number`, `exposure_time`, `iso`, `focal_length`, `camera_model` | `FloatField`/`StringField`/`IntField` | seg | EXIF capture settings for the macro camera (verbatim from `metadata/TomatoMAP-Seg_meta.csv`) |
`dataset.classes["ground_truth"]` holds all 17 classes (7 det ROIs + 10 seg stages); `dataset.classes["classification"]` holds the 50 `bbch_*` labels present in the data.
### `dataset.info`
Dataset-level metadata (not attached to individual samples): paper/dataset DOIs, code and project-homepage links, license, a description of each subset’s imaging setup and capture window, a pointer to the (sample-unattached) camera calibration files, and known data-quality notes – including that 1 of the 64,464 det images has no YOLO label file, and that the source paper’s Table 3 lists 91,120 instances for BBCH 80-89 while the exact count in the released data is 9,120 (the other two coarse buckets, 10,560 and 29,328, match the paper exactly, and 10,560 + 29,328 + 9,120 sums with the unclassified/other-stage images to the full 64,464 – this looks like a typo in the publication, not a data issue).
### Saved views
| View | Samples | Description |
|----------------------------------|-------------|----------------------------------------------|
| `det` | 64,464 | The det/cls subset |
| `seg` | 3,605 | The seg subset |
| `seg-labeled` | 727 | Seg images with ISAT annotations |
| `seg-unlabeled` | 2,878 | Seg images with no annotation on file |
| `bbch-vegetative-flowering` | 10,560 | Det images with BBCH stage 60-69 |
| `bbch-flowering` | 29,328 | Det images with BBCH stage 70-79 |
| `bbch-fruit-development` | 9,120 | Det images with BBCH stage 80-89 |
| `det-missing-labels` | 1 | QA view: det images with zero ROI detections |
| `camera-pos_1` .. `camera-pos_4` | 16,116 each | One fixed rig camera elevation each |
### Indexes
`plant_id`, `image_set_id`, `piid`, `pose_id`, `bbch_stage`, `classification.label`, `ground_truth.detections.label`, `capture_datetime` – chosen for the fields most useful to filter/sort/group by, on top of FiftyOne’s default indexes (`id`, `filepath`, `group.id`, `group.name`, `tags`, etc.). Constant fields (`accession`, `transgene`, `cabin`) are intentionally not indexed.
### Parsing decisions
- **BBCH is a session-level label, not per-image.** `metadata/BBCH_classification.xlsx` gives one BBCH code per (`plant_id`, calendar date); every image captured in that plant’s 12-pose x 4-camera session on that date inherits the same `classification` value.
- **`metadata/raw_pheno.csv` only covers ~12% of images** (7,776 of 64,464, 61 of 101 plants) – it was used as an authoritative cross-check where available, but filename parsing (`pi{cam}_{seq}_{plant}_{pose}_{timestamp}.jpg`) is the primary, and only complete, source of the `plant_id`/`piid`/`pose_id`/`image_set_id`/`capture_datetime` fields.
- **Seg masks are rasterized from ISAT polygon annotations** (`TomatoMAP-Seg/labels/*.json`) into per-detection boolean masks cropped to each object’s bounding box.
- **Seg and det/cls are treated as unrelated pools on purpose** – see Out-of-Scope Use above.
## Dataset Creation
### Curation Rationale
Observer bias and inconsistency in manual plant phenotyping limit the accuracy and reproducibility of fine-grained trait analysis. TomatoMAP was built to provide a large, standardized, multi-view imaging dataset – captured under a fixed IoT-based acquisition protocol – to train and validate real-time, accuracy/efficiency-balanced computer vision models (MobileNetv3, YOLOv11, Mask R-CNN in the original paper) as a substitute for manual phenotyping, and to quantify how closely such models agree with human experts (via Cohen’s Kappa and inter-rater agreement analysis).
### Source Data
#### Data Collection and Processing
Det/Cls images were captured by a custom data-acquisition station: 4 OV5647 5MP color CMOS cameras (three with 90 deg lenses, one with a 170 deg fisheye lens) mounted at 45/90/135/180 degree vertical inclination, aimed at a turntable that rotated a potted plant through 12 poses at 30 degree increments, synchronized across all 4 cameras at each rotational step. 101 plants were imaged this way over a 163-day period (2023-08-16 to 2024-01-26 in the released data, irregular intervals of 1-13 days), yielding 64,464 images at 1080x1440 resolution. Camera intrinsics/distortion were calibrated with a planar chessboard pattern.
Seg macrophotographs were captured separately with a Panasonic Lumix DMC-FZ1000 at 3648x5472 resolution, over a different (later, mostly non-overlapping) date range – the ISAT annotation files’ embedded folder paths (`MPTSTD_dataset_boost/task{1,2,3}`) indicate this was its own imaging task, not the same rig/plant cohort as the det/cls subset.
#### Who are the source data producers?
Researchers at the Julius Kühn-Institute (Institute for Breeding Research on Horticultural Crops, Quedlinburg) and the Computer Graphics Group, University of Siegen, using an automated greenhouse imaging system of their own design.
### Annotations
#### Annotation process
- **Det (ROI boxes):** a progressive, AI-assisted labeling workflow. An initial 1,780 images were manually labeled in Label Studio to train a first assistive model; that model pre-labeled 2,504 more images, which were expert-reviewed/corrected and merged in to train a second assistive model; a third round applied the second model to a 6,000-image pool with further expert validation; all annotation files were then cross-checked by five experts. Bounding boxes were drawn to tightly enclose visible extent (plus morphologically plausible occluded extent for partially visible objects); intra-class overlap over 70% was disallowed, cross-class overlap (e.g. panicle containing flower/fruit clusters) was permitted.
- **Cls (BBCH stage):** assigned per (plant, date) session according to the standardized BBCH developmental scale for vegetables/fruiting crops.
- **Seg (instance masks):** annotated with the Interactive Semi-Automatic Annotation Tool (ISAT), using Segment Anything Model 2 (SAM2) for proposal generation, followed by manual refinement, producing pixel-wise polygon masks with per-instance group ids.
#### Who are the annotators?
Domain experts at JKI, with annotations cross-validated among five annotators (self-inspection, senior-annotator spot review of >=10% of samples, and cross-annotator validation), plus AI-vs-human agreement analysis against 5 named domain experts in the source paper.
#### Personal and Sensitive Information
None. The dataset contains only plant imagery and greenhouse/imaging-hardware metadata; no personal or human-subject data is present.
## Citation
**BibTeX:**
```bibtex
@article{zhang2026tomatomap,
author = {Zhang, Yujie and Struckmeyer, Sabine and Kolb, Andreas and Reichardt, Sven},
title = {Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping},
journal = {Scientific Data},
year = {2026},
volume = {13},
pages = {309},
doi = {10.1038/s41597-026-06926-9}
}
@misc{zhang2025tomatomapdata,
author = {Zhang, Yujie and Struckmeyer, Sabine and Kolb, Andreas and Reichardt, Sven},
title = {Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping},
year = {2025},
publisher = {e!DAL -- Leibniz Institute of Plant Genetics and Crop Plant Research (IPK)},
doi = {10.5447/ipk/2025/14}
}
```
**APA:**
Zhang, Y., Struckmeyer, S., Kolb, A., & Reichardt, S. (2026). Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping. *Scientific Data*, 13, 309. https://doi.org/10.1038/s41597-026-06926-9
## More Information
The original TomatoMAP-Cls/Det builder notebooks and model training/evaluation code are at [github.com/0YJ/TomatoMAP](https://github.com/0YJ/TomatoMAP). This FiftyOne version was built directly from the raw rig images + YOLO labels + ISAT segmentation JSONs (not from the authors’ pre-built train/val/test split, since that split was generated with a random seed the authors’ code controls, not distributed as a fixed file) – see `dataset.info` and the saved views above for how to reconstruct comparable subsets.
## Dataset Card Authors
[Harpreet Sahota](https://huggingface.co/harpreetsahota) (FiftyOne format and this card)