<table class="fo-notebook-links" align="left">
    <td>
        <a target="_blank" href="https://colab.research.google.com/github/voxel51/fiftyone/blob/main/docs/source/getting_started/model_dataset_zoo/03_remote_models.ipynb">
            <img src="https://cdn.voxel51.com/colab-logo-256px.png"> &nbsp; Run in Google Colab
        </a>
    </td>
    <td>
        <a target="_blank" href="https://github.com/voxel51/fiftyone/blob/main/docs/source/getting_started/model_dataset_zoo/03_remote_models.ipynb">
            <img src="https://cdn.voxel51.com/github-logo-256px.png"> &nbsp; View source on GitHub
        </a>
    </td>
    <td>
        <a target="_blank" href="https://raw.githubusercontent.com/voxel51/fiftyone/main/docs/source/getting_started/model_dataset_zoo/03_remote_models.ipynb" download>
            <img src="https://cdn.voxel51.com/cloud-icon-256px.png"> &nbsp; Download notebook
        </a>
    </td>
</table>

# Exploring Remote Zoo Models

This section walks you through the process of using **remotely-sourced models** from the FiftyOne Model Zoo—models whose definitions are hosted on public GitHub repositories or accessible via external URLs.

With this approach, you can:

- Seamlessly integrate custom models hosted on GitHub or cloud archives
- Reproduce and share models across teams and projects using standardized links
- Apply these models to your datasets within FiftyOne just like built-in zoo models

FiftyOne’s flexible zoo API supports both built-in and remote models. That means whether you’re pulling a model from `voxel51/openai-clip`, `ultralytics/yolov5`, or your own repository, the workflow remains consistent and intuitive.

## In this notebook, you’ll learn how to:

- Specify remote model sources using GitHub repos, branches, or commit references
- Load and apply these models to your datasets
- Visualize the predictions directly in the FiftyOne App

> 💡 To use private GitHub repositories, be sure to set the `GITHUB_TOKEN` environment variable with your personal access token.
> 💡 Before starting, ensure you’ve installed the required packages:

### Using Florence2 as Remotely Sourced Zoo Model

Original documentation of this Remotely Sourced Zoo Model is [here](https://github.com/harpreetsahota204/florence2)

For context, here is the first image:

# Setup Zoo Model

The three captioning operations require no additional arguments beyond selecting the operation type.

Supported `detail_level` values:

- `basic`
- `detailed`
- `more_detailed`

To change the caption detail level:

The operations for `detection`, `dense_region_caption`, `region_proposal` don’t require additional parameters for general use.

However, `open_vocabulary_detection` requires a `text_prompt` parameter to guide the detection towards specific objects.

The results are stored as Detections objects containing bounding boxes and labels:

Or you can use the caption field:

For dense detections, this runs without a prompt and returns all detectable objects.

Phrase grounding requires either a direct caption or a reference to a caption field. You can provide this in two ways:

When you want to use a Field of a Sample for grounding, you use the following pattern:

Segmentation requires either a direct expression or a reference to a field containing expressions.

Similar to phrase grounding, you can provide this in two ways:

When you want to use a Field of a Sample for grounding, you use the following pattern:

Basic OCR (“ocr”) requires no additional parameters and returns text strings. For OCR with region information (`ocr_with_region`), you can set `store_region_info=True` to include bounding boxes for each text region:

Remotely-sourced models expand the power and flexibility of the FiftyOne Model Zoo by allowing you to access and deploy models from external GitHub repositories or public URLs. Whether you’re leveraging a popular open-source model or integrating one from your own team, this workflow makes it easy to apply and visualize predictions in your datasets.
