Run in Google Colab

View source on GitHub

Download notebook

# Step 2: Adding Object Detections to a FiftyOne Dataset In our first step, we will be covering how you can add object detections to your dataset. First we will go through how to add predictions using the FiftyOne Model Zoo and apply_model. In the second part, we will demonstrate how to add your detection predictions from your own custom model or labels. Feel free to skip ahead if you are interested in only adding object detections with your own model or labels! ## Using the Model Zoo Let’s kick things off by loading in the [MSCOCO 2017](https://cocodataset.org/#home) validation split from the [FiftyOne Dataset Zoo](https://docs.voxel51.com/dataset_zoo/datasets.html). We will cap it to a max of 1000 samples: With FiftyOne, you have tons of pretrained models at your disposal to use via the [FiftyOne Model Zoo](https://docs.voxel51.com/model_zoo/index.html) or using one of our [integrations](https://docs.voxel51.com/integrations/index.html) such as [HuggingFace](https://docs.voxel51.com/integrations/huggingface.html)! To get started using them, first load the model in and pass it into the apply_model function. We will use [retinanet-resnet50-fpn-coco-torch](https://docs.voxel51.com/model_zoo/models.html#retinanet-resnet50-fpn-coco-torch) from the model zoo first! Let’s visualize our results!  ## Adding Predictions using Ultralytics Thanks to [FiftyOne’s integration](https://docs.voxel51.com/integrations/ultralytics.html) with [Ultralytics](https://github.com/ultralytics/ultralytics), we can pass any Ultralytics YOLO model into apply_model as well!  ## Adding Predictions from Custom Model When bringing your own model to add predictions to your dataset, you can add [detection labels](https://docs.voxel51.com/user_guide/using_datasets.html#object-detection) directly to each sample! The **most** important part to remember is that FiftyOne uses `[nx, ny, nw, nh]` bounding box format, or normalized x,y,w,h notation. This means that each value in the bounding box is between (0,1). Below is a sample function that converts an `xyxy` box to `nxywh`. For our custom model in this example, we will be using torchvision [FasterRCNN_Resnet50](https://docs.voxel51.com/user_guide/using_datasets.html#object-detection). The pattern for adding custom labels looks like this: 1. Load the sample image 2. Perform any necessary preprocessing 3. Inference on the image 4. Grab the prediction and confidence of the model_output 5. Adjust the bounding box if needed 6. Add the values as a label to your sample Let’s walkthrough them below! Finally, we can see all of our results in the FiftyOne App!  ## Summary You’ve added object detections using Model Zoo models, Ultralytics YOLO, and custom models. Remember: FiftyOne uses normalized `[nx, ny, nw, nh]` bounding box format. Next up: **Step 3 covers finding detection mistakes**