Ultralytics Integration¶
FiftyOne integrates natively with Ultralytics, so you can load, fine-tune, and run inference with your favorite Ultralytics models on your FiftyOne datasets with just a few lines of code!
Setup¶
To get started with Ultralytics, just install the following packages:
pip install ultralytics "torch>=1.8"
Inference¶
The examples below show how to run inference with various Ultralytics models on the following sample dataset:
1 2 3 4 5 6 7 8 9 10 11 12 | # Suppress Ultralytics logging import os; os.environ["YOLO_VERBOSE"] = "False" import fiftyone as fo import fiftyone.zoo as foz import fiftyone.utils.ultralytics as fou from ultralytics import YOLO # Load an example dataset dataset = foz.load_zoo_dataset("quickstart", max_samples=25) dataset.select_fields().keep_fields() |
Object detection¶
You can use the builtin
to_detections()
utility to
convert Ultralytics bounding boxes to
FiftyOne format:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | # YOLOv8 model = YOLO("yolov8s.pt") # model = YOLO("yolov8m.pt") # model = YOLO("yolov8l.pt") # model = YOLO("yolov8x.pt") # YOLOv5 # model = YOLO("yolov5s.pt") # model = YOLO("yolov5m.pt") # model = YOLO("yolov5l.pt") # model = YOLO("yolov5x.pt") for sample in dataset.iter_samples(progress=True): result = model(sample.filepath)[0] sample["boxes"] = fou.to_detections(result) sample.save() session = fo.launch_app(dataset) |

Instance segmentation¶
You can use the builtin
to_instances()
and
to_polylines()
utilities to
convert Ultralytics instance segmentations to
FiftyOne format:
You can use the builtin
to_detections()
utility to
convert YOLO boxes to FiftyOne format:
1 2 3 4 5 6 7 8 9 10 11 12 13 | model = YOLO("yolov8s-seg.pt") # model = YOLO("yolov8m-seg.pt") # model = YOLO("yolov8l-seg.pt") # model = YOLO("yolov8x-seg.pt") for sample in dataset.iter_samples(progress=True): result = model(sample.filepath)[0] sample["detections"] = fou.to_detections(result) sample["instances"] = fou.to_instances(result) sample["polylines"] = fou.to_polylines(result) sample.save() session = fo.launch_app(dataset) |

Keypoints¶
You can use the builtin
to_keypoints()
utility to
convert Ultralytics keypoints to FiftyOne format:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | model = YOLO("yolov8s-pose.pt") # model = YOLO("yolov8m-pose.pt") # model = YOLO("yolov8l-pose.pt") # model = YOLO("yolov8x-pose.pt") for sample in dataset.iter_samples(progress=True): result = model(sample.filepath)[0] sample["keypoints"] = fou.to_keypoints(result) sample.save() # Store the COCO-pose keypoint skeleton so the App can render it dataset.default_skeleton = fo.KeypointSkeleton( labels=[ "nose", "left eye", "right eye", "left ear", "right ear", "left shoulder", "right shoulder", "left elbow", "right elbow", "left wrist", "right wrist", "left hip", "right hip", "left knee", "right knee", "left ankle", "right ankle", ], edges=[ [11, 5, 3, 1, 0, 2, 4, 6, 12], [9, 7, 5, 6, 8, 10], [15, 13, 11, 12, 14, 16], ], ) session = fo.launch_app(dataset) |

Batch inference¶
Any of the above loops can be executed using batch inference using the pattern below:
1 2 3 4 5 6 7 8 9 10 11 | from fiftyone.core.utils import iter_batches # The inference batch size batch_size = 4 predictions = [] for filepaths in iter_batches(dataset.values("filepath"), batch_size): results = model(filepaths) predictions.extend(fou.to_detections(results)) dataset.set_values("predictions", predictions) |
Note
See this section for more information about performing batch updates to your FiftyOne dataset.
Training¶
You can use FiftyOne’s builtin YOLOv5 exporter to export your FiftyOne datasets for use with Ultralytics models.
For example, the code below prepares a random subset of the Open Images v7 dataset for fine-tuning:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | import fiftyone as fo import fiftyone.utils.ultralytics as fou import fiftyone.zoo as foz # The path to export the dataset EXPORT_DIR = "/tmp/oiv7-yolo" # Prepare train split train = foz.load_zoo_dataset( "open-images-v7", split="train", label_types=["detections"], max_samples=100, ) # YOLO format requires a common classes list classes = train.default_classes train.export( export_dir=EXPORT_DIR, dataset_type=fo.types.YOLOv5Dataset, label_field="ground_truth", split="train", classes=classes, ) # Prepare validation split validation = foz.load_zoo_dataset( "open-images-v7", split="validation", label_types=["detections"], max_samples=10, ) validation.export( export_dir=EXPORT_DIR, dataset_type=fo.types.YOLOv5Dataset, label_field="ground_truth", split="val", # Ultralytics uses 'val' classes=classes, ) |
From here, training an Ultralytics model is as simple as passing the path to the dataset YAML file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | from ultralytics import YOLO # The path to the `dataset.yaml` file we created above YAML_FILE = "/tmp/oiv7-yolo/dataset.yaml" # Load a model model = YOLO("yolov8s.pt") # load a pretrained model # model = YOLO("yolov8s.yaml") # build a model from scratch # Train the model model.train(data=YAML_FILE, epochs=3) # Evaluate model on the validation set metrics = model.val() # Export the model path = model.export(format="onnx") |