#### NOTE
This is a **Hugging Face dataset**. For large datasets, ensure `huggingface_hub>=1.1.3` to avoid rate limits. Learn more in the Hugging Face integration docs.
# SegFly: Aerial RGB-Thermal Segmentation - FiftyOne subset
A grouped [FiftyOne](https://docs.voxel51.com) dataset — a curated subset of **SegFly**
(Gross et al., ECCV 2026) of pixel-aligned aerial **RGB-thermal (RGB-T) pairs** with
semantic-segmentation masks and derived instance detections.
- **This dataset:** [`Voxel51/SegFly`](https://huggingface.co/datasets/Voxel51/SegFly) — load directly with `load_from_hub`
- **Loader (FiftyOne remote zoo):** [`github.com/Burhan-Q/SegFly`](https://github.com/Burhan-Q/SegFly)
- **Original dataset:** [`markus-42/SegFly`](https://huggingface.co/datasets/markus-42/SegFly) · [Project page](https://markus-42.github.io/publications/2026/segfly/) · [arXiv](https://arxiv.org/abs/2603.17920) · [Source code](https://github.com/markus-42/SegFly)
> This is an **unofficial** redistribution of a subset of SegFly for use with FiftyOne.
> All credit for the dataset belongs to the original authors (see [Citation](#citation)).
## Installation
```bash
pip install fiftyone huggingface_hub
```
`huggingface_hub` is used to download the media from Hugging Face.
## Usage
```python
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load this dataset directly from the Hub (470 RGB-T pair groups)
dataset = load_from_hub("Voxel51/SegFly", persistent=True)
# per-class instance counts / label filtering come from the `instances` field
print(dataset.count_values("instances.detections.label"))
session = fo.launch_app(dataset)
```
Or load it through the FiftyOne remote zoo loader (also supports `max_samples`):
```python
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset(
"https://github.com/Burhan-Q/SegFly",
max_samples=100, # optional; limits the number of samples
)
```
Every group has a base `rgb` slice and a `thermal` slice, so the App’s slice
selector toggles `rgb`↔`thermal` on the **same** sample (like `quickstart-groups`).
Segmentation masks render with the SegFly benchmark color scheme.
## What’s included (curated subset)
A ~0.6 GB set of pixel-aligned RGB-T pairs:
| Scene | Altitude | Modality | Split | Groups |
|------------|------------|-----------------------|---------|----------|
| `scene_03` | 30m | thermal (RGB-T pairs) | train | 470 |
**Total: 470 groups / 940 samples** (470 `thermal` + 470 `rgb` slice samples).
(The full SegFly release is 35,613 samples / 191 GB across 9 scenes; this dataset
is the curated RGB-T pair subset only.)
### Group model
One group per thermal capture, with a **uniform** slice set (like `quickstart-groups`),
base slice `rgb`:
- slice `rgb` — the pixel-registered RGB frame (base)
- slice `thermal` — the LWIR frame
Both slices carry two label fields (sharing the aligned mask):
- `ground_truth` — [`fo.Segmentation`](https://docs.voxel51.com/api/fiftyone.core.labels.html#fiftyone.core.labels.Segmentation), the semantic mask
- `instances` — [`fo.Detections`](https://docs.voxel51.com/api/fiftyone.core.labels.html#fiftyone.core.labels.Detections) **derived** from the mask: countable classes
(Vehicle, Truck, Building, Roof, Ground Obstacle, Rock, Cable, Cable Tower, Crane, Person,
Bicycle) as one detection per connected region; amorphous classes (Road, Walkway, Dirt,
Gravel, Grass, Vegetation, Tree, Water, Parking Lot, Construction) as one per class. This is
what enables App per-class **filtering** and per-class **instance counts** (the semantic
`Segmentation` field alone cannot be filtered/counted by class).
Because every group has both slices, toggling `rgb`↔`thermal` in the App stays on the
same sample. Per-sample fields: `scene`, `altitude`, `modality`. Split is a sample tag.
Note: `instances` are derived from the semantic masks via connected components (not source
instance annotations); “stuff” classes are stored as a single region per image.
### Reusing the instance derivation (e.g. on the full SegFly release)
The `instances` field ships **precomputed** in this dataset. The derivation is also
exposed as reusable functions in the loader repo, so you can apply the **same** stuff/thing
logic to any SegFly semantic mask (including the full 191 GB `markus-42/SegFly`):
```python
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
import segfly # from github.com/Burhan-Q/SegFly (add the cloned repo dir to your path)
# a single mask -> instance detections
dets = segfly.segmentation_to_instances(sample["ground_truth"])
# or populate an `instances` field across a whole dataset (grouped or flat)
full = load_from_hub(
"markus-42/SegFly",
format="ParquetFilesDataset",
...,
)
segfly.add_instances(full) # in_field="ground_truth", out_field="instances"
print(full.count_values("instances.detections.label"))
```
`MASK_TARGETS` (class map) and `MASK_TYPES` (the stuff/thing split) are module-level
constants you can inspect or override.
## Classes
The stored masks contain the **raw OccuFly class IDs (0–36)**. `mask_targets` names
every ID that can appear:
| ID | Name | | ID | Name | | ID | Name |
|------|------------|----|------|-----------------|----|------|--------------|
| 0 | Unlabeled | | 8 | Tree | | 17 | Roof |
| 1 | Road | | 9 | Ground Obstacle | | 21 | Cable |
| 2 | Walkway | | 10 | unknown_10 | | 22 | Cable Tower |
| 3 | Dirt | | 11 | Person | | 33 | Parking Lot |
| 4 | Gravel | | 12 | Bicycle | | 34 | Construction |
| 5 | Rock | | 13 | Vehicle | | 35 | Crane |
| 6 | Grass | | 14 | Water | | 36 | Truck |
| 7 | Vegetation | | 16 | Building | | | |
> **Note.** SegFly’s published “**15 benchmark classes**” are a documented
> post-processing remap that is **not** baked into the mask files:
> `Rock(5)` and `Cable Tower(22)` → `Ground Obstacle(9)`; `Person(11)`,
> `Bicycle(12)`, `Cable(21)`, `Crane(35)` → `Unlabeled(0)`. Apply this remap if
> you need the benchmark protocol. `ID 10` appears in the data but is undocumented
> in the source and is left un-named as `unknown_10`.
## License & attribution
SegFly is released under **[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)**
(non-commercial, share-alike, attribution). This redistribution keeps the same license.
Use is **non-commercial** only; you must attribute the original authors and share
derivatives under the same terms.
## Citation
```bibtex
@inproceedings{gross2026segfly,
title={{SegFly: A Dataset and 2D-3D-2D Paradigm for Aerial RGB-Thermal Semantic Segmentation at Scale}},
author={Markus Gross and Sai Bharadhwaj Matha and Rui Song and Viswanathan Muthuveerappan and Conrad Christoph and Julius Huber and Daniel Cremers},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
year={2026},
}
```