FiftyOne Agent#

The FiftyOne Agent is an AI-powered assistant built into the FiftyOne Enterprise App. It lets you work with your datasets using natural language. You can import data, run model inference, find duplicates, evaluate predictions, and more, all from a conversational interface.

fiftyone-agent-demo

Setup#

  1. Contact your Customer Success representative to enable the FiftyOne Agent for your deployment.

  2. Open any dataset in the FiftyOne Enterprise App. You will see a new Agent button in the upper-right corner of the App.

fiftyone-agent-button-location

Configuring model providers#

The first time you open the Agent, you will be prompted to configure a model provider. The Agent supports over 100 providers, including Anthropic, OpenAI, Google, and more.

fiftyone-agent-settings

To add a provider, fill in the following fields:

  • Name: a label for this provider configuration

  • Provider: select from the list of supported providers

  • Endpoint (optional): use this if your model is hosted at a custom URL

  • API key: your provider’s API key

  • Models: select one or more models to make available

  • Default: mark this provider as the default

fiftyone-agent-provider-details

You can click Test connection to verify your credentials before saving.

Note

Need help configuring a provider? Contact your Customer Success representative, or see Secrets for how to store API keys securely in your deployment.

Using the agent#

Once a provider is configured, you can start a conversation with the Agent. Type any task in plain language and the Agent will execute it against your dataset.

fiftyone-agent-chat

Some examples of what you can ask:

  • “Find and remove duplicate images from this dataset”

  • “Run object detection and show me low-confidence predictions”

  • “Export this dataset to COCO format”

To start a new conversation, click the + button.

fiftyone-agent-new-conversation

To return to a previous conversation, click History.

fiftyone-agent-conversation-history

Skills#

The Agent ships with a set of built-in skills that cover the most common computer vision workflows. Skills are structured instructions that tell the agent exactly how to perform a task, step by step.

fiftyone-agent-skills

Note

You can also build and add your own custom skills to extend the agent’s capabilities. See Developing skills for full instructions.