Run in Google Colab

View source on GitHub

Download notebook

Adding Object Detections to a Dataset

This recipe provides a glimpse into the possibilities for integrating FiftyOne into your ML workflows. Specifically, it covers:

• Loading an object detection dataset from the Dataset Zoo

• Adding predictions from an object detector to the dataset

• Launching the FiftyOne App and visualizing/exploring your data

• Integrating the App into your data analysis workflow

Setup

If you haven’t already, install FiftyOne:

!pip install fiftyone

In this tutorial, we’ll use an off-the-shelf Faster R-CNN detection model provided by PyTorch. To use it, you’ll need to install torch and torchvision, if necessary.

!pip install torch torchvision

Loading a detection dataset

In this recipe, we’ll work with the validation split of the COCO dataset, which is conveniently available for download via the FiftyOne Dataset Zoo.

The snippet below will download the validation split and load it into FiftyOne.

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset(
    "coco-2017",
    split="validation",
    dataset_name="detector-recipe",
)

Split 'validation' already downloaded
Loading 'coco-2017' split 'validation'
 100% |████████████████████| 5000/5000 [43.3s elapsed, 0s remaining, 114.9 samples/s]
Dataset 'detector-recipe' created

Let’s inspect the dataset to see what we downloaded:

# Print some information about the dataset
print(dataset)

Name:           detector-recipe
Media type:     image
Num samples:    5000
Persistent:     False
Info:           {'classes': ['0', 'person', 'bicycle', ...]}
Tags:           ['validation']
Sample fields:
    filepath:     fiftyone.core.fields.StringField
    tags:         fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
    metadata:     fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
    ground_truth: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)

# Print a ground truth detection
sample = dataset.first()
print(sample.ground_truth.detections[0])

<Detection: {
    'id': '602fea44db78a9b44e6ae129',
    'attributes': BaseDict({}),
    'label': 'potted plant',
    'bounding_box': BaseList([
        0.37028125,
        0.3345305164319249,
        0.038593749999999996,
        0.16314553990610328,
    ]),
    'mask': None,
    'confidence': None,
    'index': None,
    'area': 531.8071000000001,
    'iscrowd': 0.0,
}>

Note that the ground truth detections are stored in the ground_truth field of the samples.

Before we go further, let’s launch the FiftyOne App and use the GUI to explore the dataset visually:

session = fo.launch_app(dataset)

Activate