Using Agents#

What Are Skills and MCP?#

Skills and MCP solve different parts of the same problem. MCP is about connection: it lets an agent talk to real systems, run real operations, and get real results instead of just generating text. Skills are about guidance: they teach the agent how to use those capabilities correctly for a specific task. MCP exposes what the system can do, while skills explain how and when to do it.

On their own, tools are powerful but ambiguous. Skills turn those tools into repeatable workflows. They encode experience, decisions, and guardrails. Together, they let agents move from “I can call functions” to “I know how to complete this task end to end.”

Two Parts, One System#

FiftyOne MCP Server connects agents to FiftyOne’s 80+ operators, dataset management, model inference, brain computations, and the App. It implements the open Model Context Protocol standard, giving any compatible agent direct access to your data and tools.

FiftyOne Skills are expert workflows built on top. Each skill teaches the agent how to complete a specific task: import data, find duplicates, visualize embeddings. Skills handle the nuances so you don’t have to.

Quick Start#

Step 1: Install the MCP Server#

pip install fiftyone-mcp-server

Warning

Install into the same Python environment where FiftyOne is installed. If you use a virtual environment or conda environment, activate it first or use the full path to the executable in your AI tool config.

To verify the installation:

fiftyone-mcp

You should see:

Starting fiftyone-mcp server...
fiftyone-mcp server initialized successfully

Press Ctrl+C to stop it. Your AI tool manages the server lifecycle automatically once configured.

Step 2: Configure Your AI Tool#

claude mcp add fiftyone -- fiftyone-mcp

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project-scoped):

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

Or use the one-click install: Install FiftyOne MCP in Cursor

Restart Cursor after saving.

Add to .vscode/mcp.json:

{
  "servers": {
    "fiftyone": {
      "command": "fiftyone-mcp",
      "type": "stdio"
    }
  }
}

Or use the one-click install: Install FiftyOne MCP in VS Code

The Gemini CLI extension registers the MCP server automatically. Install skills and MCP together:

gemini extensions install https://github.com/voxel51/fiftyone-skills.git --consent

If fiftyone-mcp is not on your PATH, edit the extension config to use the full path to the executable.

Create or edit .antigravity/mcp.json in your project directory:

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

Restart Antigravity after saving.

If you have uv installed, skip the pip install step entirely:

{
  "mcpServers": {
    "fiftyone": {
      "command": "uvx",
      "args": ["fiftyone-mcp-server"]
    }
  }
}

Step 3: Install Skills#

Skills teach your agent how to complete complex FiftyOne workflows end to end. Run the command below to download the universal installer and install the default skills pack from voxel51/fiftyone-skills:

curl -sL skil.sh | sh -s -- voxel51/fiftyone-skills

You can also develop and install your own skills to teach agents custom workflows specific to your project.

See the Agent Ecosystem for the full list of available skills and per-agent install instructions.

Step 4: Use It#

"List all my datasets"
"Load quickstart dataset and show summary"
"Open the map panel and show me the embeddings"
"Select samples with confidence above 0.9"
"What plugins are available? Install the brain plugin"
"Find near-duplicate images in my dataset"

Your agent automatically discovers operators and executes the appropriate tools.

Available Tools#

	Category	Tools	Description
📊	Dataset Management	3	List, load, and summarize datasets
🎯	App Operations	29	Control the App UI (views, panels, selection, …)
⚡	Operator System	3	Discover and execute any FiftyOne operator
🔄	Pipelines	2	Run pipelines and manage delegated operations
🔌	Plugin Management	5	Discover, install, and manage plugins
🖥️	Session	1	Launch the FiftyOne App server
📈	Aggregations	8	Count, distinct, bounds, mean, histogram, …
🧬	Samples	5	Add, tag, untag, and set values on samples
🗂️	Schema	2	Inspect and modify dataset field schemas
🎨	App Config	6	Color scheme, sidebar groups, active fields

Tool Modes#

Which tools are available depends on your integration:

Integration	Modes	Use case
Voxel-Agent	`app` + `sdk`	Agent in FiftyOne Enterprise (full UI control + data operations)
Terminal / CLI	`session` + `sdk`	Headless agent (launch the App, query data, execute operators)

Resources#

Resource	Description
FiftyOne MCP Server	Source code and contributing guide
FiftyOne Skills	Expert workflows for AI assistants
FiftyOne Plugins	Official plugin collection
Model Context Protocol	MCP specification
PyPI Package	MCP server on PyPI
Discord Community	Get help and share ideas