Running Inference

FiftyOne lets you add model predictions to any dataset or view via apply_model() and compute_embeddings(). Whatever your model produces — classifications, detections, instance or semantic segmentations, keypoints, heatmaps, and more — is stored directly on your samples using FiftyOne’s native Label types, ready to explore in the App.

These methods work identically whether the model comes from the Model Zoo or is entirely your own.

Inference with zoo models

The Model Zoo provides hundreds of pre-trained models, spanning detection, classification, segmentation, keypoints, embeddings, and more, that you can apply to your data in a couple of lines:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

model = foz.load_zoo_model("faster-rcnn-resnet50-fpn-coco-torch")
dataset.apply_model(model, label_field="predictions")

session = fo.launch_app(dataset)

See the full recipe for zoo model inference

Note

Browse or search the Model Zoo catalog to find a model for your task, or work through the Model Dataset Zoo Guide for a complete, guided walkthrough.

Inference with custom models

If your model isn’t in the zoo, you have two options.

The simplest is to iterate over your dataset and construct the appropriate Label instances yourself. This is the most direct path and requires no FiftyOne-specific model code.

The more reusable option is to wrap your model so that it implements the Model interface, at which point it works with apply_model() and compute_embeddings() exactly like a zoo model. FiftyOne’s TorchImageModel class makes this easy for most PyTorch models.

Grok the Model interface and custom models

Note

Did you know? You can also register your custom model under a name of your choice so that it can be loaded and shared just like a built-in zoo model.