<table class="fo-notebook-links" align="left">
    <td>
        <a target="_blank" href="https://colab.research.google.com/github/voxel51/fiftyone/blob/main/docs/source/getting_started/model_evaluation/01_basic_evaluation.ipynb">
            <img src="https://cdn.voxel51.com/colab-logo-256px.png"> &nbsp; Run in Google Colab
        </a>
    </td>
    <td>
        <a target="_blank" href="https://github.com/voxel51/fiftyone/blob/main/docs/source/getting_started/model_evaluation/01_basic_evaluation.ipynb">
            <img src="https://cdn.voxel51.com/github-logo-256px.png"> &nbsp; View source on GitHub
        </a>
    </td>
    <td>
        <a target="_blank" href="https://raw.githubusercontent.com/voxel51/fiftyone/main/docs/source/getting_started/model_evaluation/01_basic_evaluation.ipynb" download>
            <img src="https://cdn.voxel51.com/cloud-icon-256px.png"> &nbsp; Download notebook
        </a>
    </td>
</table>

# Step 1: Basic Evaluation

In our first step, we will be covering how we can perform basic evaluation. One of the great parts of FiftyOne is that once your data and model predictions are in FiftyOne, evaluation becomes easy, no matter if you are coming from different formats. Gone are the days of converting your YOLO styled predictions to COCO styled evaluation, FiftyOne handles all the conversions for you so you can focus on the task at hand.

Let’s take a look first at loading a common a dataset with predictions.

## Installation

Here are some packages that are needed to help run some of our demo code:

### Loading a Zoo Dataset for Evaluation

We will be loading the [quickstart](https://docs.voxel51.com/api/fiftyone.utils.quickstart.html) dataset from the [Dataset Zoo](https://docs.voxel51.com/dataset_zoo/index.html). This dataset is a slice of MSCOCO and contains some preloaded predictions. If you are unsure how to load your own detection dataset, be sure to checkout our [Getting Started with Detections](../object_detection/index.html)

Once our dataset is loaded, we can start getting ready for model eval!

Before we go further, let’s launch the [FiftyOne App](https://docs.voxel51.com/user_guide/app.html) and use the GUI to explore the dataset visually:

### Evaluate Detections

Now that we have samples with ground truth and predicted objects, let’s use FiftyOne to evaluate the quality of the detections.

FiftyOne provides a powerful [evaluation API](https://docs.voxel51.com/user_guide/evaluation.html) that contains a collection of methods for performing evaluation of model predictions. Since we’re working with object detections here, we’ll use [detection evaluation](https://docs.voxel51.com/api/fiftyone.core.collections.html#fiftyone.core.collections.SampleCollection.evaluate_detections)

We can run evaluation on our samples via `evaluate_detections()`. Note that this method is available on both the `Dataset` and `DatasetView` classes, which means that we can run evaluation on subsets of our dataset as well.

By default, this method will use the COCO evaluation protocol, plus some extra goodies that we will use later.

### Analyzing Results

The `results` object returned by the evaluation routine provides a number of convenient methods for analyzing our predictions.

For example, let’s print a classification report for the top-10 most common classes in the dataset:

We can also grab the mean average-precision (mAP) of our model as well:

### Evaluate Subsets

As mentioned before, we can evaluate `DatasetViews` as well! Let’s evaluate only where our model is highly confident. First we will create a high confidence view, then evaluate with `evaluate_detections()` again. See using [Dataset Views](https://docs.voxel51.com/user_guide/using_views.html) for full details on matching, filtering, or sorting detections.

We can check out our new view in the session before we run evaluation:

Just like before, lets run evaluation. Be sure to change the eval_key to a new name this time!

### Evaluate for Classification

Evaluation is just as easy for classification tasks. Once you have loaded up your dataset and model predictions, you can start with `dataset.evaluate_classifications()`

If you need a refresher on how to work with classification datasets, head over to Getting Started with Classifications!

### Evaluate for Segmentation

The last basic form of evaluation we will cover is evaluating segmentations!

There are two popular forms of segmentations, Instance Segmentation and Semantic Segmentation. In FiftyOne, instance segmentation is stored in `fo.Detections` and semantic segmentations are stored in `fo.Segmentations`. We will cover how to evaluate both.

If you need a refresher on how to work with segmentation datasets, head over to Getting Started with Segmentations!

Once your dataset is prepped and ready with ground_truth and predicted segmentations, you can start evaluation!

## Instance Segmentation Example

With our model and predictions loaded, lets run evaluation with [detection.evaluate_detection()](https://docs.voxel51.com/api/fiftyone.utils.eval.detection.html).

## Semantic Segmentation Evaluation

Now let’s look at an example of semantic segmentation. We can easily convert our instance segmentation dataset to semantic masks using [to_segmentation()](https://docs.voxel51.com/api/fiftyone.core.labels.html#fiftyone.core.labels.Detection.to_segmentation). After we convert our `ground_truth` and `instances` fields, we can evaluate our new masks! Let’s convert now:

Finally, we can evaluate our semantic segmentations with `dataset.evaluate_segmentations()`:

### Conclusion

This covers the basic of model eval in FiftyOne. In the next step, we will learn how to dive even deeper with the [Model Evaluation Panel](https://docs.voxel51.com/plugins/api/plugins.panels.model_evaluation.html), a interactive tool that allows you find exactly where your model is doing the best and the worst all in the FiftyOne app. Learn more by continuing!
