Run in Google Colab

View source on GitHub

Download notebook

Finding Detection Mistakes with FiftyOne

Annotations mistakes create an artificial ceiling on the performance of your models. However, finding these mistakes by hand is at least as arduous as the original annotation work! Enter FiftyOne.

In this tutorial, we explore how FiftyOne can be used to help you find mistakes in your object detection annotations. To detect mistakes in classification datasets, check out this tutorial.

We’ll cover the following concepts:

• Loading your existing dataset into FiftyOne

• Adding model predictions to your dataset

• Computing insights into your dataset relating to possible label mistakes

• Visualizing mistakes in the FiftyOne App

So, what’s the takeaway?

FiftyOne can help you find and correct label mistakes in your datasets, enabling you to curate higher quality datasets and, ultimately, train better models!

Setup

If you haven’t already, install FiftyOne:

!pip install fiftyone

In order to compute mistakenness, your dataset needs to have two detections fields, one with your ground truth annotations and one with your model predictions.

In this example, we’ll load the quickstart dataset from the FiftyOne Dataset Zoo, which has ground truth annotations and predictions from a PyTorch Faster-RCNN model for a few samples from the COCO dataset.

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

Dataset already downloaded
Loading 'quickstart'
 100% |█████████████████| 200/200 [3.5s elapsed, 0s remaining, 45.0 samples/s]
Dataset 'quickstart' created

print(dataset)

Name:           quickstart
Media type:     image
Num samples:    200
Persistent:     False
Tags:           ['validation']
Sample fields:
    filepath:     fiftyone.core.fields.StringField
    tags:         fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
    metadata:     fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
    ground_truth: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
    uniqueness:   fiftyone.core.fields.FloatField
    predictions:  fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)

# Print a sample ground truth detection
sample = dataset.first()
print(sample.predictions.detections[0])

<Detection: {
    'id': '5f452c60ef00e6374aad9394',
    'attributes': BaseDict({}),
    'tags': BaseList([]),
    'label': 'bird',
    'bounding_box': BaseList([
        0.22192673683166503,
        0.06093006531397502,
        0.4808845520019531,
        0.8937615712483724,
    ]),
    'mask': None,
    'confidence': 0.9750854969024658,
    'index': None,
}>

Let’s start by visualizing the dataset in the FiftyOne App:

# Open the dataset in the App
session = fo.launch_app(dataset)

Activate