Migrations#

This page describes how to migrate between FiftyOne Enterprise versions, both for admins migrating the core Enterprise App infrastructure and individual users who need to install a new version of the Enterprise Python SDK.

Refer to this section to see how to migrate existing datasets from open source to Enterprise.

Upgrading your Python SDK#

Users can upgrade their FiftyOne Enterprise Python client to the latest version as follows:

pip install --index-url https://${TOKEN}@pypi.fiftyone.ai --upgrade fiftyone

A specific FiftyOne Enterprise client version can be installed like so:

pip install --index-url https://${TOKEN}@pypi.fiftyone.ai fiftyone==${VERSION}

Note

You can find your TOKEN by logging into the FiftyOne Enterprise App and clicking on the account icon in the upper right.

Upgrading your deployment#

The basic admin workflow for upgrading a FiftyOne Enterprise deployment is:

  • Upgrade all automated services and individual user workflows that use the Enterprise Python SDK to an appropriate SDK version

  • Upgrade your core Enterprise App infrastructure (via Kubernetes, Docker, etc)

  • Upgrade your database’s version, as described below

Note

Contact your Voxel51 CS Engineer for all relevant upgrade information, including compatible SDK versions, deployment assets, and upgrade assistance.

New FiftyOne Enterprise versions occasionally introduce data model changes that require database migrations when upgrading your deployment.

Admins can check a deployment’s current version via the Python SDK as shown below:

$ fiftyone migrate --info
FiftyOne Enterprise version: 0.7.1
FiftyOne compatibility version: 0.15.1
Database version: 0.15.1

...

Note

Individual datasets have versions as well. They are lazily upgraded the first time they are loaded under a new database version. Often there is no migration required, but there could be.

Unlike open source FiftyOne, a Enterprise database is not automatically upgraded when a user connects to the database with a newer Python client version. Instead, an admin must manually upgrade your Enterprise database by installing the newest version of the Enterprise SDK locally, assuming admin privileges, and running the command shown below:

export FIFTYONE_DATABASE_ADMIN=true

# Option 1: update the database version only (datasets lazily migrated on load)
fiftyone migrate

# Option 2: migrate the database and all datasets
fiftyone migrate --all

Note

Once the database is upgraded, all users must upgrade their Python SDK to a compatible version. Any connections from incompatible Python clients will be refused and an informative error message will be displayed.

Downgrading your deployment#

Admins can also downgrade their FiftyOne Enterprise deployment to an older version if necessary.

The steps are the same as when upgrading, except that you’ll need to know the appropriate database version to migrate down to. Each version of Enterprise corresponds to a version of open source FiftyOne called its “open source compatibility version”, and this versioning system is used to set the database version.

For example, you can downgrade to Enterprise v0.10 like so:

OS_COMPAT_VERSION=0.18.0  # OS compatibility version for Enterprise v0.10.0

export FIFTYONE_DATABASE_ADMIN=true
fiftyone migrate --all -v ${OS_COMPAT_VERSION}

Note

The above command must be run with the newer SDK version installed.

Note

Contact your Voxel51 CS engineer if you need to know the open source compatibility version for a particular Enterprise version that you wish to downgrade to.

Migrating datasets to Enterprise#

Any datasets that you have created via open source FiftyOne can be migrated to your Enterprise deployment by exporting them in FiftyOneDataset format:

 1# Open source SDK
 2import fiftyone as fo
 3
 4dataset = fo.load_dataset(...)
 5
 6dataset.export(
 7    export_dir="/tmp/dataset",
 8    dataset_type=fo.types.FiftyOneDataset,
 9    export_media=False,
10)

and then re-importing them with the Enterprise SDK connected to your Enterprise deployment:

1# Enterprise SDK
2import fiftyone as fo
3
4dataset = fo.Dataset.from_dir(
5    dataset_dir="/tmp/dataset",
6    dataset_type=fo.types.FiftyOneDataset,
7    persistent=True,
8)

Note that you’ll need to update any local filepaths to cloud paths in order to use the dataset in Enterprise.

If you need to upload the local media to the cloud, the Enterprise SDK provides a builtin utility for this:

1import fiftyone.core.storage as fos
2
3fos.upload_media(
4    dataset,
5    "s3://path/for/media",
6    update_filepaths=True,
7    progress=True,
8)

Note

By default, the above method only uploads the media in the filepath field of your samples. If your dataset contains other media fields (e.g. thumbnails, segmentations, or heatmaps) simply run the above command multiple times, using the media_field argument to specify the appropriate fields to upload.

If any media fields use the same filenames as other fields, be sure to provide different remote_dir paths each time you call the above method to avoid overwriting existing media.

If the files already exist in cloud buckets, you can manually update the filepaths on the dataset:

1cloud_paths = []
2for filepath in dataset.values("filepath"):
3    cloud_path = get_cloud_path(filepath)  # your function
4    cloud_paths.append(cloud_path)
5
6dataset.set_values("filepath", cloud_paths)

When you’re finished, delete the local export of the dataset:

1shutil.rmtree("/tmp/dataset")