Table of Contents
Contents

FiftyOne Plugins

FiftyOne provides a plugin system that you can use to customize and extend its behavior!

This page describes how to develop, publish, and install custom plugins.

Configuring your plugin directory

First, create a directory where you want to store your plugins.

Subsequently, when running FiftyOne, you must specify the location of this directory by setting the FIFTYONE_PLUGINS_DIR environment variable:

export FIFTYONE_PLUGINS_DIR=/path/to/your/plugins

You can also permanently configure this directory by adding it to your FiftyOne config.

{
    "plugins_dir": "/path/to/your/plugins"
}

If you are running an instance of your FiftyOne App server, you will need to restart it to pick up this new setting.

Note

Your plugins directory must be readable by the FiftyOne server.

Installing plugins manually

Fiftyone will try and find your plugin’s package.json file within the plugin directory described above. Below is an example of a typical plugin directory.

/my-fiftyone-plugins
  /my-plugin-1
    /package.json
    /dist
      /my-plugin.min.js
  /my-plugin-2
    /package.json
    /dist
      /my-plugin2.min.js

In order to manually install a plugin, you must copy the plugin’s directory into your plugin directory so that it matches the structure above.

If your FiftyOne App server is already running, you should restart the server and refresh any connected browser clients to see the plugins show up.

Note

If you do not see your plugin, make sure the package.json file is present and has the appropriate fiftyone.script setting described below.

Installing plugins with a package manager or git

In order to install plugins with a package manager, you must have one of the following tools available in the environment you are running FiftyOne:

  • npm

  • yarn

  • git - if installing from a git repo

Once you have a plugins directory setup, you must create a shell package to version your plugins:

cd $FIFTYONE_PLUGINS_DIR

yarn init

# or with npm
npm init

Now you can install a node package that contains a plugin:

# if it is avaialable on plubic/private npm registry
yarn add my-fiftyone-plugin

# or with npm
npm install my-fiftyone-plugin --save

If your plugin is only available in a git repository, you can still install via git, although the environment must be configured to allow reading from that git repository:

# install via a github http url
yarn add my-plugin@https://github.com/user/my-plugin.git#my-branch-name

# install via ssh/repo
yarn add ssh://github.com/user/my-plugin#my-branch

Configuring plugins

You can store system-wide plugin configurations under the plugins key of your App config or dataset. This allows for configuring plugins at the application wide level or per dataset.

See the configuring plugins page for more information on changing a plugin’s configuration.

Note

You can see an example of dataset level plugin configuration on the map panel page.

Developing plugins

In order to develop and test your plugin you will need the following:

  • A development install of FiftyOne

  • The FiftyOne App setup for development

  • A plugin skeleton (start with one of the plugins in voxel51/fiftyone-plugins)

  • npm link / symlink to the @fiftyone/plugins package

  • npm link / symlink to the @fiftyone/aggregations package (optional)

  • npm link / symlink to the @fiftyone/components package (optional)

Note

You cannot use relative paths to load these modules. They must be loaded using the from '@fiftyone/$PKG_NAME' syntax. This allows the build to externalize them, so they are loaded at runtime by the parent application.

For local testing, follow these basic steps:

First ensure your plugin’s package.json includes the path to the plugin script:

{
    "fiftyone": {
        "script": "dist/index.umd.js"
    }
}

Then follow the steps below, in separate terminal sessions as needed.

# tell FiftyOne where you want to load plugins from
# this should be the parent directory to all your plugins
FIFTYONE_PLUGINS_DIR=/path/to/your/plugins

# start the FiftyOne App in dev mode
cd $FIFTYONE/app/packages/app
yarn dev

# start the FiftyOne python server (in a separate session)
cd $FIFTYONE
python fiftyone/server/main.py

# ensure your plugin has a symlink to the @fiftyone/plugins package
cd $FIFTYONE/app/packages/plugins
npm link
cd $MY_PLUGIN
npm link @fiftyone/plugins

# note: if you are using other @fiftyone/* packages
# you will need to follow the same linking steps for those packages

# now you can build your plugin for development
yarn build

You should now have a running FiftyOne server and App, including your plugin.

Note

Each time you change you plugin’s source you must rebuild using yarn build. You can setup a watcher to do this automatically. See nodemon.

Publishing your plugin

You can publish your plugin to either a public/private npm registry or a git repository. Including your package.json and built (dist) files is required for both. No other files are required to be published with your plugin.

Before publishing make sure you do the following:

  • Login to the registry you are trying to publish to

  • OR use a .npmrc to include private registry credentials

  • Have the correct name, version, etc in your package.json

  • Have a built plugin dist directory

  • Your package.json points to the plugin entry point

Then to publish your latest plugin to an npm registry:

yarn publish

# or with npm
npm publish

If you are using a git repository to publish your plugins, you must ensure that you include the dist directory when pushing to the remote repo.

How to write plugins

Below are introductory examples to the FiftyOne plugin API.

Hello world

A simple hello world plugin, that renders “hello world” in a panel, would look like this:

import { registerComponent, PluginComponentTypes } from "@fiftyone/plugins";

function HelloWorld() {
    return <h1>Hello World</h1>;
}

registerComponent({
    copmponent: HelloWorld,
    type: PluginComponentTypes.Panel,
});

Installing the plugin above would require building a bundle JS file and placing in a directory with a package.json file.

The FiftyOne python server will then detect this as an installed plugin and the App will load it and render it.

Adding a custom FiftyOne Visualizer

import * as fop from "@fiftyone/plugins";
import * as fos from "@fiftyone/visualizer";

function PointCloud({ src }) {
    // TODO: implement your visualizer using React
}

// this separate components shows where the FiftyOne plugin
// dependent code ends and the pure react code begins
function CustomVisualizer({ sample }) {
    const src = fos.getSampleSrc(sample.filepath);
    // now that we have all the data we need
    // we can delegate to code that doesn't depend
    // on the FiftyOne plugin API
    return <PointCloud src={src} />;
}

fop.registerComponent({
    // component to delegate to
    copmponent: CustomVisualizer,
    // tell FiftyOne you want to provide a Visualizer
    type: PluginComponentTypes.Visualizer,
    // activate this plugin when the mediaType is PointCloud
    activator: ({ dataset }) => dataset.mediaType === "PointCloud",
});

Adding a custom Plot

import * as fop from "@fiftyone/plugins";
import * as fos from "@fiftyone/state";
import * as foa from "@fiftyone/aggregations";
import AwesomeMap from "react-mapping-library";

function CustomPlot() {
    const dataset = useRecoilValue(fos.dataset);
    const view = useRecoilValue(fos.view);
    const filters = useRecoilValue(fos.filters);
    const [aggregate, points, loading] = foa.useAggregation({
        dataset,
        filters,
        view,
    });

    React.useEffect(() => {
        aggregate(
            [
                new foa.aggregations.Values({
                    fieldOrExpr: "id",
                }),
                new foa.aggregations.Values({
                    fieldOrExpr: "location.point.coordinates",
                }),
            ],
            dataset.name
        );
    }, [dataset, filters, view]);

    if (loading) return <h1>Loading</h1>;

    return <MyMap geoPoints={points} />;
}

fop.registerComponent({
    // component to delegate to
    copmponent: CustomPlot,
    // tell FiftyOne you want to provide a custom Panel
    type: PluginComponentTypes.Panel,
    // used for the plot selector button
    label: "Map",
    // only show the Map plot when the dataset has Geo data
    activator: ({ dataset }) => dataset.sampleFields.location,
});

Note

The PluginComponentType.Plot type is deprecated. Use PluginComponentType.Panel instead.

Reacting to state changes

import * as fos from '@fiftyone/state'
import * as recoil from 'recoil'

// this example demonstrates handling updates to
// filters/sidebar, but applies to everything
// listed under "state" below
function MyPlugin() {
  const activeFields = recoil.useRecoilValue(fos.activeFields)

  return <ul>{activeFields.map(f => <li>{f.name}</li>)}
}

Interactivity and state

If your plugin only has internal state, you can use existing state management to achieve your desired ux. For example, in a 3D visualizer, you might want to use Thee.js and its object model, events, and state management. Or just use your own React hooks to maintain your plugin components internal state.

If you want to allow users to interact with other aspects of FiftyOne through your plugin, you can use the @fiftyone/state package:

// note: similar to react hooks, these must be used in the context
// of a React component

// select a dataset
const selectLabel = fos.useOnSelectLabel();

// in a callback
selectLabel({ id: "labelId", field: "fieldName" });

Panel state

Plugins that provide PluginComponentTypes.Panel components should use the @fiftyone/spaces package to manage their state. This package provides hooks to allow plugins to manage the state of individual panel instances.

import { usePanelStatePartial, usePanelTitle } from "@fiftyone/spaces";
import { Button } from '@fiftyone/components';

// in your panel component, you can use the usePanelStatePartial hook
// to read and write to the panel state
function MyPanel() {
    const [state, setState] = usePanelStatePartial('choice');
    const setTitle = usePanelTitle();

    React.useEffect(() => {
      setTitle(`My Panel: ${state}`);
    }, [state]);

    return (
      <div>
        <h1>Choice: {state}</h1>
        <Button onClick={() => setState('A')}>A</Button>
        <Button onClick={() => setState('B')}>B</Button>
      </div>
    );
}

Reading settings in your plugin

Some plugins use libraries or tools that require credentials such as API keys or tokens. Below is an example for how to provide and read these credentials.

The same mechanism can be used to expose configuration to plugin users, such as color choices, and default values.

You can store a setting in either the App config or an individual dataset. Here’s an example of both.

// app_config.json
{
  "plugins": {
    "my-plugin": {
      "mysetting": "foo"
    }
  }
}

Now lets take that setting and change it for the quickstart dataset.

import fiftyone as fo
dataset = fo.load_dataset("quickstart")

# Modify the dataset's App config
dataset.app_config.plugins["my-plugin"] = {
  "mysetting": "bar"
}

dataset.save()

Then in your plugin implementation, you can read settings with the useSettings hook:

const { mysetting } = fop.useSettings("my-plugin");

Querying FiftyOne

The typical use case for a plugin is to provide a unique way of visualizing FiftyOne data. However some plugins may need to also fetch data in a unique way to efficiently visualize it.

For example, a PluginComponentType.Plot plugin rendering a map of geo points may need to fetch data relative to where the user is currently viewing. In MongoDB, such a query would look like this:

{
  $geoNear: {
    near: { type: "Point", coordinates: [ -73.99279 , 40.719296 ] },
    maxDistance: 2,
    query: { category: "Parks" },
  }
}

In a FiftyOne plugin this same query can be performed using the useAggregation() method of the plugin SDK:

import * as fop from "@fiftyone/plugins";
import * as fos from "@fiftyone/state";
import * as foa from "@fiftyone/aggregations";
import * as recoil from "recoil";

function useGeoDataNear() {
    const dataset = useRecoilValue(fos.dataset);
    const view = useRecoilValue(fos.view);
    const filters = useRecoilValue(fos.filters);
    const [aggregate, points, isLoading] = foa.useAggregation({
        dataset,
        filters,
        view,
    });
    const availableFields = findAvailableFields(dataset.sampleFields);
    const [selectedField, setField] = React.useState(availableFields[0]);

    React.useEffect(() => {
        aggregate([
            new foa.aggregations.Values({
                fieldOrExpr: "location.point.coordinates",
            }),
        ]);
    }, []);

    return {
        points,
        isLoading,
        setField,
        availableFields,
        selectedField,
    };
}

function MapPlugin() {
    const { points, isLoading, setField, availableFields, selectedField } =
        useGeoDataNear();

    return (
        <Map
            points={points}
            onSelectField={(f) => setField(f)}
            selectedField={selectedField}
            locationFields={availableFields}
        />
    );
}

fop.registerComponent({
    name: "MapPlugin",
    label: "Map",
    activator: ({ dataset }) => findAvailableFields(dataset.fields).length > 0,
});