Run in Google Colab

View source on GitHub

Download notebook

# Step 4: Evaluating Detections This step demonstrates how to use FiftyOne to perform hands-on evaluation of your detection model. It covers the following concepts: - Evaluating your model using FiftyOne’s evaluation API - Viewing the best and worst performing samples in your dataset ## Load a Detection Dataset In this example, we’ll load the [quickstart](https://docs.voxel51.com/user_guide/dataset_zoo/datasets.html#dataset-zoo-quickstart) dataset again from the FiftyOne Dataset Zoo, which has ground truth annotations and predictions from a PyTorch Faster-RCNN model for a few samples from the COCO dataset. ## Evaluate Detections Now that we have samples with ground truth and predicted objects, let’s use FiftyOne to evaluate the quality of the detections. FiftyOne provides a powerful [evaluation API](https://docs.voxel51.com/user_guide/evaluation.html) that contains a collection of methods for performing evaluation of model predictions. Since we’re working with object detections here, we’ll use [detection evaluation](https://docs.voxel51.com/user_guide/evaluation.html#detections). We can run evaluation on our samples via evaluate_detections(). Note that this method is available on both the `Dataset` and `DatasetView` classes, which means that we can run evaluation on our high_conf_view to assess the quality of only the high confidence predictions in our dataset. By default, this method will use the [COCO evaluation](https://cocodataset.org/#detection-eval) protocol, plus some extra goodies that we will use later. The `results` object returned by the evaluation routine provides a number of convenient methods for analyzing our predictions. For example, let’s print a classification report for the top-10 most common classes in the dataset, as well as the mAP score: ## Model Evaluation Panel You can observe all of your model evaluation results in the FiftyOne app with the [Model Evaluation Panel](https://docs.voxel51.com/user_guide/evaluation.html#model-evaluation-panel-sub-new)! When you load a dataset in the App that contains one or more evaluations, you can open the [Model Evaluation panel](https://docs.voxel51.com/user_guide/app.html#app-model-evaluation-panel) to visualize and interactively explore the evaluation results in the App:  You can even click into individual classes to see model performance on filtered samples as well!  Follow to [Model Evaluation Doc Page](https://docs.voxel51.com/user_guide/app.html#app-model-evaluation-panel) to learn more about how you can perform model eval, compare multiple models, and more!