Note

This is a Hugging Face dataset. For large datasets, ensure huggingface_hub>=1.1.3 to avoid rate limits. Learn more in the Hugging Face integration docs.

Hugging Face

SegFly: Aerial RGB-Thermal Segmentation - FiftyOne subset#

Preview of the SegFly subset in the FiftyOne App.

A grouped FiftyOne dataset — a curated subset of SegFly (Gross et al., ECCV 2026) of pixel-aligned aerial RGB-thermal (RGB-T) pairs with semantic-segmentation masks and derived instance detections.

This is an unofficial redistribution of a subset of SegFly for use with FiftyOne. All credit for the dataset belongs to the original authors (see Citation).

Installation#

pip install fiftyone huggingface_hub

huggingface_hub is used to download the media from Hugging Face.

Usage#

import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub

# Load this dataset directly from the Hub (470 RGB-T pair groups)
dataset = load_from_hub("Voxel51/SegFly", persistent=True)

# per-class instance counts / label filtering come from the `instances` field
print(dataset.count_values("instances.detections.label"))

session = fo.launch_app(dataset)

Or load it through the FiftyOne remote zoo loader (also supports max_samples):

import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset(
    "https://github.com/Burhan-Q/SegFly",
    max_samples=100,  # optional; limits the number of samples
)

Every group has a base rgb slice and a thermal slice, so the App’s slice selector toggles rgb↔thermal on the same sample (like quickstart-groups). Segmentation masks render with the SegFly benchmark color scheme.

What’s included (curated subset)#

A ~0.6 GB set of pixel-aligned RGB-T pairs:

Scene

Altitude

Modality

Split

Groups

scene_03

30m

thermal (RGB-T pairs)

train

470

Total: 470 groups / 940 samples (470 thermal + 470 rgb slice samples). (The full SegFly release is 35,613 samples / 191 GB across 9 scenes; this dataset is the curated RGB-T pair subset only.)

Group model#

One group per thermal capture, with a uniform slice set (like quickstart-groups), base slice rgb:

  • slice rgb — the pixel-registered RGB frame (base)

  • slice thermal — the LWIR frame

Both slices carry two label fields (sharing the aligned mask):

  • ground_truth — fo.Segmentation, the semantic mask

  • instances — fo.Detections derived from the mask: countable classes (Vehicle, Truck, Building, Roof, Ground Obstacle, Rock, Cable, Cable Tower, Crane, Person, Bicycle) as one detection per connected region; amorphous classes (Road, Walkway, Dirt, Gravel, Grass, Vegetation, Tree, Water, Parking Lot, Construction) as one per class. This is what enables App per-class filtering and per-class instance counts (the semantic Segmentation field alone cannot be filtered/counted by class).

Because every group has both slices, toggling rgb↔thermal in the App stays on the same sample. Per-sample fields: scene, altitude, modality. Split is a sample tag. Note: instances are derived from the semantic masks via connected components (not source instance annotations); “stuff” classes are stored as a single region per image.

Reusing the instance derivation (e.g. on the full SegFly release)#

The instances field ships precomputed in this dataset. The derivation is also exposed as reusable functions in the loader repo, so you can apply the same stuff/thing logic to any SegFly semantic mask (including the full 191 GB markus-42/SegFly):

import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub

import segfly  # from github.com/Burhan-Q/SegFly (add the cloned repo dir to your path)

# a single mask -> instance detections
dets = segfly.segmentation_to_instances(sample["ground_truth"])

# or populate an `instances` field across a whole dataset (grouped or flat)
full = load_from_hub(
    "markus-42/SegFly",
    format="ParquetFilesDataset",
    ...,
)
segfly.add_instances(full)  # in_field="ground_truth", out_field="instances"

print(full.count_values("instances.detections.label"))

MASK_TARGETS (class map) and MASK_TYPES (the stuff/thing split) are module-level constants you can inspect or override.

Classes#

The stored masks contain the raw OccuFly class IDs (0–36). mask_targets names every ID that can appear:

ID

Name

ID

Name

ID

Name

0

Unlabeled

8

Tree

17

Roof

1

Road

9

Ground Obstacle

21

Cable

2

Walkway

10

unknown_10

22

Cable Tower

3

Dirt

11

Person

33

Parking Lot

4

Gravel

12

Bicycle

34

Construction

5

Rock

13

Vehicle

35

Crane

6

Grass

14

Water

36

Truck

7

Vegetation

16

Building

Note. SegFly’s published “15 benchmark classes” are a documented post-processing remap that is not baked into the mask files: Rock(5) and Cable Tower(22) → Ground Obstacle(9); Person(11), Bicycle(12), Cable(21), Crane(35) → Unlabeled(0). Apply this remap if you need the benchmark protocol. ID 10 appears in the data but is undocumented in the source and is left un-named as unknown_10.

License & attribution#

SegFly is released under CC BY-NC-SA 4.0 (non-commercial, share-alike, attribution). This redistribution keeps the same license. Use is non-commercial only; you must attribute the original authors and share derivatives under the same terms.

Citation#

@inproceedings{gross2026segfly,
    title={{SegFly: A Dataset and 2D-3D-2D Paradigm for Aerial RGB-Thermal Semantic Segmentation at Scale}},
    author={Markus Gross and Sai Bharadhwaj Matha and Rui Song and Viswanathan Muthuveerappan and Conrad Christoph and Julius Huber and Daniel Cremers},
    booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
    year={2026},
}