Run in Google Colab

View source on GitHub

Download notebook

Downloading and Evaluating Open Images

Downloading Google’s Open Images dataset is now easier than ever with the FiftyOne Dataset Zoo! You can load all three splits of Open Images V6, including image-level labels, detections, segmentations, and visual relationships.

FiftyOne also natively supports Open Images-style evaluation, so you can easily evaluate your object detection models and explore the results directly in the library.

This walkthrough covers:

• Downloading Open Images from the FiftyOne Dataset Zoo

• Computing predictions using a model from the FiftyOne Model Zoo

• Performing Open Images-style evaluation in FiftyOne to evaluate a model and compute its mAP

• Exploring the dataset and evaluation results

• Visualizing embeddings through interactive plots

So, what’s the takeaway?

Starting a new ML project takes data and time, and the datasets in the FiftyOne Dataset Zoo can help jump start the development process.

Open Images in particular is one of the largest publicly available datasets for object detections, classification, segmentation, and more. Additionally, with Open Images evaluation available natively in FiftyOne, you can quickly evaluate your models and compute mAP and PR curves.

While metrics like mAP are often used to compare models, the best way to improve your model’s performance isn’t to look at aggregate metrics but instead to get hands-on with your evaluation and visualize how your model performs on individual samples. All of this is made easy with FiftyOne!

Setup

If you haven’t already, install FiftyOne:

!pip install fiftyone

In this tutorial, we’ll use some TensorFlow models and PyTorch to generate predictions and embeddings, and we’ll use the UMAP method to reduce the dimensionality of embeddings, so we need to install the corresponding packages:

!pip install tensorflow torch torchvision umap-learn

This tutorial also includes some of FiftyOne’s interactive plotting capabilities.

The recommended way to work with FiftyOne’s interactive plots is in Jupyter notebooks or JupyterLab. In these environments, you can leverage the full power of plots by attaching them to the FiftyOne App and bidirectionally interacting with the plots and the App to identify interesting subsets of your data.

To use interactive plots in Jupyter notebooks, ensure that you have the ipywidgets package installed:

!pip install ipywidgets>=7.5

If you’re working in JupyterLab, refer to these instructions to get setup.

Support for interactive plots in non-notebook contexts and Google Colab is coming soon! In the meantime, you can still use FiftyOne’s plotting features in those environments, but you must manually call plot.show() to update the state of a plot to match the state of a connected session, and any callbacks that would normally be triggered in response to interacting with a plot will not be triggered.

Loading Open Images

In this section, we’ll load various subsets of Open Images from the FiftyOne Dataset Zoo and visualize them using FiftyOne.

Let’s start by downloading a small sample of 100 randomly chosen images + annotations:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset(
    "open-images-v6",
    split="validation",
    max_samples=100,
    seed=51,
    shuffle=True,
)

Downloading split 'validation' to '~/fiftyone/open-images/validation'
Downloading validation samples
 100% |█████████████████| 100/100 [20.3s elapsed, 0s remaining, 1.8 samples/s]
Downloading relevant segmentation masks
Found 100 samples
Dataset info written to '~/fiftyone/open-images/info.json'
Loading 'open-images' split 'validation'
 100% |█████████████████| 100/100 [4.1s elapsed, 0s remaining, 19.6 samples/s]
Dataset 'open-images-validation-100' created

Now let’s launch the FiftyOne App so we can explore the dataset we just downloaded.

session = fo.launch_app(dataset)

Activate