Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Dataset Card for TomatoMAP#

image/png

This is a FiftyOne dataset with 68,069 samples.

Installation#

If you haven’t already, install FiftyOne:

pip install -U fiftyone

Usage#

import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub

# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("Voxel51/tomato-map")

# Launch the App
session = fo.launch_app(dataset)

Dataset Details#

Dataset Description#

TomatoMAP is a multi-angle, multi-pose, multi-task imaging dataset of Solanum lycopersicum (tomato) collected with a purpose-built IoT imaging station at the Julius Kühn-Institute (JKI), Quedlinburg, Germany. It unifies three originally-separate TomatoMAP subsets – TomatoMAP-Cls (BBCH growth-stage classification), TomatoMAP-Det (region-of-interest object detection), and TomatoMAP-Seg (fruit/flower developmental-stage instance segmentation) – into a single grouped FiftyOne dataset, tagged by source.

101 tomato plants (all the “Money Maker” accession, non-transgenic) were imaged over a 163-day period by 4 fixed Raspberry Pi cameras (mounted at 45/90/135/180 degree elevation angles) while a turntable rotated the plant through 12 poses (30 degree increments), giving 48 synchronized images per plant per acquisition session – 64,464 images in total, each with a 7-class ROI bounding-box annotation and a session-level BBCH growth-stage label (50 distinct codes). A separate, later imaging pass with a handheld macro camera produced 3,605 high-resolution close-up photographs of individual flower buds and fruit clusters, 727 of which have ISAT-tool instance segmentation masks across 10 fruit/flower developmental-stage classes.

Curated by: Yujie Zhang, Sabine Struckmeyer, Andreas Kolb, Sven Reichardt (Julius Kühn-Institute & University of Siegen); parsed into FiftyOne format by Harpreet Sahota.
Funded by: German Federal Ministry of Agriculture, Food and Regional Identity; compute powered by the de.NBI Cloud / ELIXIR-DE.
Shared by: Harpreet Sahota (FiftyOne format on Hugging Face); original data deposited by the authors in e!DAL (IPK Gatersleben).
Language(s): en (metadata, class names, and the BBCH index descriptions are in English)
License: CC BY 4.0

Dataset Sources#

Repository: e!DAL / IPK Gatersleben (raw data); github.com/0YJ/TomatoMAP (original TomatoMAP-Cls/Det builder + model code)
Paper: Zhang et al., “Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping,” Scientific Data (2026) 13:309
Demo: 0yj.github.io/tomato_map

Uses#

Direct Use#

Training/benchmarking fine-grained BBCH phenological growth-stage classifiers.
Training/benchmarking object detectors for plant ROIs (leaf, whole plant, panicle, flower clusters, fruit clusters, axillary shoot, shoot), including studying class imbalance and overlapping-box handling.
Training/benchmarking instance/semantic segmentation of fruit ripening stages (nascent -> mini -> unripe -> semi ripe -> fully ripe) and flower/bud size stages (2mm -> 4mm -> 6mm -> 8mm -> 12mm).
Multi-view / sparse-view 3D reconstruction research: the 4 rig cameras have known intrinsics (data/calibration/pi{1-4}.npz) and nominal elevation/turntable angles, though no extrinsic rig calibration is provided.
Longitudinal growth-stage analysis using capture_datetime and plant_id across the 163-day acquisition window.

Out-of-Scope Use#

Disease/pathogen detection: all plants are healthy specimens of a single cultivar (“Money Maker”), non-transgenic – there is no disease-state variation to learn from (unlike e.g. PlantVillage or Tomato-Village).
Cross-referencing individual plants or sessions between the det/cls samples and the seg samples: they come from different camera systems, different (only lightly overlapping) capture windows, and carry no shared plant/session identifier – see dataset.info["seg_source"] for the evidence. Do not assume a plant_id-level relationship between the two subsets.
Genotype/accession comparison studies: every plant in the det/cls subset is the same accession, grown in the same cabin, so there is no genetic or environmental variation encoded in the metadata.

Dataset Structure#

This is a single grouped FiftyOne dataset (media_type="group", group field group) with 68,069 samples across 19,721 groups, unifying two source subsets that are tagged but not otherwise sample-joinable (see Out-of-Scope Use):

det (64,464 samples, tag "det") – the TomatoMAP-Cls/Det rig images. Grouped by image_set_id (one turntable pose held still, shot simultaneously by the 4 fixed cameras), with slices "pos_1".."pos_4" corresponding to the 4 camera elevations. 16,116 groups x 4 slices = 64,464 samples.
seg (3,605 samples, tag "seg") – the TomatoMAP-Seg macrophotographs. Each image is its own group (no natural grouping partner), populating a single slice "macro". Further tagged "labeled" (727 samples with ISAT annotations) or "unlabeled" (2,878 samples, bonus un-annotated photos).

Fields#

Field	FiftyOne type	Subset	Description
`filepath`	`StringField`	both	Path to the image file
`tags`	`ListField(StringField)`	both	`"det"`/`"seg"` (source subset) plus `"labeled"`/`"unlabeled"` (seg only)
`group`	`Group`	both	FiftyOne group field; `.name` is the slice (`pos_1`..`pos_4` or `macro`)
`metadata`	`ImageMetadata`	both	Standard FiftyOne image metadata (width/height/size/mime type)
`ground_truth`	`Detections`	both	Det: 7-class YOLO ROI boxes. Seg: instance masks over 10 fruit/flower-stage classes (each `Detection` also carries `isat_group` (int instance id), `area_px`, `iscrowd`). One shared field name since the two class vocabularies never overlap.
`classification`	`Classification`	det	BBCH growth-stage label, e.g. `"bbch_70"` – one label per (`plant_id`, capture date) session, inherited by all 48 images from that session
`bbch_stage`	`IntField`	det	Raw BBCH code (13-89; 50 distinct codes appear in the data)
`bbch_description`	`StringField`	det	Human-readable description of the BBCH code
`image_set_id`	`IntField`	det	Groups the 4 simultaneous camera exposures of one turntable pose (verbatim from `metadata/raw_pheno.csv` / parsed from filename)
`plant_id`	`IntField`	det	Plant identifier, 1-101
`piid`	`IntField`	det	Rig camera/position id, 1-4
`pose_id`	`IntField`	det	Turntable pose id, 1-12
`pose_degrees`	`IntField`	det	Turntable rotation in degrees (0-330, 30 degree steps)
`camera_label`	`StringField`	det	Human-readable camera description, e.g. `"bottom (45 deg)"`
`camera_angle_deg`	`IntField`	det	Camera elevation angle code (verbatim from `metadata/camera.csv`)
`camera_position_desc`	`StringField`	det	Camera position description (verbatim from `metadata/camera.csv`)
`accession`	`StringField`	det	Plant accession (constant: `"Money Maker"`)
`transgene`	`BooleanField`	det	Transgenic status (constant: `False`)
`cabin`	`StringField`	det	Greenhouse cabin id (constant: `"H01504"`)
`capture_datetime`	`DateTimeField`	both	Capture timestamp – det: parsed from the filename’s 14-digit timestamp; seg: parsed from the image’s EXIF `DateTimeOriginal`
`f_number`, `exposure_time`, `iso`, `focal_length`, `camera_model`	`FloatField`/`StringField`/`IntField`	seg	EXIF capture settings for the macro camera (verbatim from `metadata/TomatoMAP-Seg_meta.csv`)

dataset.classes["ground_truth"] holds all 17 classes (7 det ROIs + 10 seg stages); dataset.classes["classification"] holds the 50 bbch_* labels present in the data.

`dataset.info`#

Dataset-level metadata (not attached to individual samples): paper/dataset DOIs, code and project-homepage links, license, a description of each subset’s imaging setup and capture window, a pointer to the (sample-unattached) camera calibration files, and known data-quality notes – including that 1 of the 64,464 det images has no YOLO label file, and that the source paper’s Table 3 lists 91,120 instances for BBCH 80-89 while the exact count in the released data is 9,120 (the other two coarse buckets, 10,560 and 29,328, match the paper exactly, and 10,560 + 29,328 + 9,120 sums with the unclassified/other-stage images to the full 64,464 – this looks like a typo in the publication, not a data issue).

Saved views#

View	Samples	Description
`det`	64,464	The det/cls subset
`seg`	3,605	The seg subset
`seg-labeled`	727	Seg images with ISAT annotations
`seg-unlabeled`	2,878	Seg images with no annotation on file
`bbch-vegetative-flowering`	10,560	Det images with BBCH stage 60-69
`bbch-flowering`	29,328	Det images with BBCH stage 70-79
`bbch-fruit-development`	9,120	Det images with BBCH stage 80-89
`det-missing-labels`	1	QA view: det images with zero ROI detections
`camera-pos_1` .. `camera-pos_4`	16,116 each	One fixed rig camera elevation each

Indexes#

plant_id, image_set_id, piid, pose_id, bbch_stage, classification.label, ground_truth.detections.label, capture_datetime – chosen for the fields most useful to filter/sort/group by, on top of FiftyOne’s default indexes (id, filepath, group.id, group.name, tags, etc.). Constant fields (accession, transgene, cabin) are intentionally not indexed.

Parsing decisions#

BBCH is a session-level label, not per-image. metadata/BBCH_classification.xlsx gives one BBCH code per (plant_id, calendar date); every image captured in that plant’s 12-pose x 4-camera session on that date inherits the same classification value.
metadata/raw_pheno.csv only covers ~12% of images (7,776 of 64,464, 61 of 101 plants) – it was used as an authoritative cross-check where available, but filename parsing (pi{cam}_{seq}_{plant}_{pose}_{timestamp}.jpg) is the primary, and only complete, source of the plant_id/piid/pose_id/image_set_id/capture_datetime fields.
Seg masks are rasterized from ISAT polygon annotations (TomatoMAP-Seg/labels/*.json) into per-detection boolean masks cropped to each object’s bounding box.
Seg and det/cls are treated as unrelated pools on purpose – see Out-of-Scope Use above.

Dataset Creation#

Curation Rationale#

Observer bias and inconsistency in manual plant phenotyping limit the accuracy and reproducibility of fine-grained trait analysis. TomatoMAP was built to provide a large, standardized, multi-view imaging dataset – captured under a fixed IoT-based acquisition protocol – to train and validate real-time, accuracy/efficiency-balanced computer vision models (MobileNetv3, YOLOv11, Mask R-CNN in the original paper) as a substitute for manual phenotyping, and to quantify how closely such models agree with human experts (via Cohen’s Kappa and inter-rater agreement analysis).

Source Data#

Data Collection and Processing#

Det/Cls images were captured by a custom data-acquisition station: 4 OV5647 5MP color CMOS cameras (three with 90 deg lenses, one with a 170 deg fisheye lens) mounted at 45/90/135/180 degree vertical inclination, aimed at a turntable that rotated a potted plant through 12 poses at 30 degree increments, synchronized across all 4 cameras at each rotational step. 101 plants were imaged this way over a 163-day period (2023-08-16 to 2024-01-26 in the released data, irregular intervals of 1-13 days), yielding 64,464 images at 1080x1440 resolution. Camera intrinsics/distortion were calibrated with a planar chessboard pattern.

Seg macrophotographs were captured separately with a Panasonic Lumix DMC-FZ1000 at 3648x5472 resolution, over a different (later, mostly non-overlapping) date range – the ISAT annotation files’ embedded folder paths (MPTSTD_dataset_boost/task{1,2,3}) indicate this was its own imaging task, not the same rig/plant cohort as the det/cls subset.

Who are the source data producers?#

Researchers at the Julius Kühn-Institute (Institute for Breeding Research on Horticultural Crops, Quedlinburg) and the Computer Graphics Group, University of Siegen, using an automated greenhouse imaging system of their own design.

Annotations#

Annotation process#

Det (ROI boxes): a progressive, AI-assisted labeling workflow. An initial 1,780 images were manually labeled in Label Studio to train a first assistive model; that model pre-labeled 2,504 more images, which were expert-reviewed/corrected and merged in to train a second assistive model; a third round applied the second model to a 6,000-image pool with further expert validation; all annotation files were then cross-checked by five experts. Bounding boxes were drawn to tightly enclose visible extent (plus morphologically plausible occluded extent for partially visible objects); intra-class overlap over 70% was disallowed, cross-class overlap (e.g. panicle containing flower/fruit clusters) was permitted.
Cls (BBCH stage): assigned per (plant, date) session according to the standardized BBCH developmental scale for vegetables/fruiting crops.
Seg (instance masks): annotated with the Interactive Semi-Automatic Annotation Tool (ISAT), using Segment Anything Model 2 (SAM2) for proposal generation, followed by manual refinement, producing pixel-wise polygon masks with per-instance group ids.

Who are the annotators?#

Domain experts at JKI, with annotations cross-validated among five annotators (self-inspection, senior-annotator spot review of >=10% of samples, and cross-annotator validation), plus AI-vs-human agreement analysis against 5 named domain experts in the source paper.

Personal and Sensitive Information#

None. The dataset contains only plant imagery and greenhouse/imaging-hardware metadata; no personal or human-subject data is present.

Citation#

BibTeX:

@article{zhang2026tomatomap,
  author  = {Zhang, Yujie and Struckmeyer, Sabine and Kolb, Andreas and Reichardt, Sven},
  title   = {Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping},
  journal = {Scientific Data},
  year    = {2026},
  volume  = {13},
  pages   = {309},
  doi     = {10.1038/s41597-026-06926-9}
}

@misc{zhang2025tomatomapdata,
  author    = {Zhang, Yujie and Struckmeyer, Sabine and Kolb, Andreas and Reichardt, Sven},
  title     = {Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping},
  year      = {2025},
  publisher = {e!DAL -- Leibniz Institute of Plant Genetics and Crop Plant Research (IPK)},
  doi       = {10.5447/ipk/2025/14}
}

APA:

Zhang, Y., Struckmeyer, S., Kolb, A., & Reichardt, S. (2026). Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping. Scientific Data, 13, 309. https://doi.org/10.1038/s41597-026-06926-9

More Information#

The original TomatoMAP-Cls/Det builder notebooks and model training/evaluation code are at github.com/0YJ/TomatoMAP. This FiftyOne version was built directly from the raw rig images + YOLO labels + ISAT segmentation JSONs (not from the authors’ pre-built train/val/test split, since that split was generated with a random seed the authors’ code controls, not distributed as a fixed file) – see dataset.info and the saved views above for how to reconstruct comparable subsets.

Dataset Card Authors#

Harpreet Sahota (FiftyOne format and this card)