Configuring FiftyOne¶
FiftyOne can be configured in various ways. This guide covers the various options that exist, how to view your current config, and how to customize your config as desired.
Configuration options¶
FiftyOne supports the configuration options described below:
Config field |
Environment variable |
Default value |
Description |
---|---|---|---|
|
|
|
Whether the client is allowed to trigger database migrations. See this section for more information. |
|
|
|
The directory in which to store FiftyOne’s backing database. Only applicable if
|
|
|
|
A name to use for FiftyOne’s backing database in your MongoDB instance. The database is automatically created if necessary. |
|
|
|
A MongoDB URI to specifying a custom MongoDB database to which to connect. See this section for more information. |
|
|
|
Whether to validate the compatibility of database before connecting to it. See this section for more information. |
|
|
|
The default directory in which to store datasets that are downloaded from the FiftyOne Dataset Zoo. |
|
|
|
A list of manifest JSON files specifying additional zoo datasets. See adding datasets to the zoo for more information. |
|
|
|
The default directory to use when performing FiftyOne operations that
require writing dataset contents to disk, such as ingesting datasets via
|
|
|
|
The default ML backend to use when performing operations such as
downloading datasets from the FiftyOne Dataset Zoo that support multiple ML
backends. Supported values are |
|
|
|
A default batch size to use when applying models to datasets. |
|
|
|
Batching implementation to use in some batched database operations such as
|
|
|
|
Fixed size of batches. Only used when |
|
|
|
Target content size of batches, in bytes. Only used when |
|
|
|
Target latency between batches, in seconds. Only used when |
|
|
|
The default numeric string pattern to use when writing sequential lists of files. |
|
|
|
The default image format to use when writing images to disk. |
|
|
|
The default video format to use when writing videos to disk. |
|
|
|
The default port to use to serve the FiftyOne App. |
|
|
|
The default address to use to serve the FiftyOne App. This may
be either an IP address or hostname. If it’s a hostname, the App will listen to all
IP addresses associated with the name. The default is |
|
|
|
Controls whether UUID based import and App usage events are tracked. |
|
|
|
Controls FiftyOne’s package-wide logging level. Can be any valid |
|
|
|
An optional maximum number of workers to use when creating thread pools |
|
|
|
An optional maximum number of workers to use when creating process pools |
|
|
|
The default directory in which to store models that are downloaded from the FiftyOne Model Zoo. |
|
|
|
A list of manifest JSON files specifying additional zoo models. See adding models to the zoo for more information. |
|
|
|
A list of modules that should be automatically imported whenever FiftyOne is imported. See this page for an example usage. |
|
|
|
The timeout for execution of an operator. See this page for more information. |
|
|
|
Whether to allow delegated operations to be scheduled locally. See this page for more information. |
|
|
|
A directory containing custom App plugins. See this page for more information. |
|
|
|
When set to |
|
|
|
Controls whether UUID based import and App usage events are tracked. |
|
|
|
Controls whether progress bars are printed to the terminal when performing operations such reading/writing large datasets or activating FiftyOne Brain methods on datasets. |
|
|
|
An optional timezone string. If provided, all datetimes read from FiftyOne datasets will be expressed in this timezone. See this section for more information. |
Viewing your config¶
You can print your current FiftyOne config at any time via the Python library and the CLI:
import fiftyone as fo
# Print your current config
print(fo.config)
# Print a specific config field
print(fo.config.default_ml_backend)
{
"batcher_static_size": 100,
"batcher_target_latency": 0.2,
"batcher_target_size_bytes": 1048576,
"database_admin": true,
"database_dir": "~/.fiftyone/var/lib/mongo",
"database_name": "fiftyone",
"database_uri": null,
"database_validation": true,
"dataset_zoo_dir": "~/fiftyone",
"dataset_zoo_manifest_paths": null,
"default_app_address": null,
"default_app_port": 5151,
"default_batch_size": null,
"default_batcher": "latency",
"default_dataset_dir": "~/fiftyone",
"default_image_ext": ".jpg",
"default_ml_backend": "torch",
"default_sequence_idx": "%06d",
"default_video_ext": ".mp4",
"do_not_track": false,
"logging_level": "INFO",
"max_process_pool_workers": null,
"max_thread_pool_workers": null,
"model_zoo_dir": "~/fiftyone/__models__",
"model_zoo_manifest_paths": null,
"module_path": null,
"operator_timeout": 600,
"allow_legacy_orchestrators": false,
"plugins_cache_enabled": false,
"plugins_dir": null,
"requirement_error_level": 0,
"show_progress_bars": true,
"timezone": null
}
torch
# Print your current config
fiftyone config
# Print a specific config field
fiftyone config default_ml_backend
{
"batcher_static_size": 100,
"batcher_target_latency": 0.2,
"batcher_target_size_bytes": 1048576,
"database_admin": true,
"database_dir": "~/.fiftyone/var/lib/mongo",
"database_name": "fiftyone",
"database_uri": null,
"database_validation": true,
"dataset_zoo_dir": "~/fiftyone",
"dataset_zoo_manifest_paths": null,
"default_app_address": null,
"default_app_port": 5151,
"default_batch_size": null,
"default_batcher": "latency",
"default_dataset_dir": "~/fiftyone",
"default_image_ext": ".jpg",
"default_ml_backend": "torch",
"default_sequence_idx": "%06d",
"default_video_ext": ".mp4",
"do_not_track": false,
"logging_level": "INFO",
"max_process_pool_workers": null,
"max_thread_pool_workers": null,
"model_zoo_dir": "~/fiftyone/__models__",
"model_zoo_manifest_paths": null,
"module_path": null,
"operator_timeout": 600,
"allow_legacy_orchestrators": false,
"plugins_cache_enabled": false,
"plugins_dir": null,
"requirement_error_level": 0,
"show_progress_bars": true,
"timezone": null
}
torch
Note
If you have customized your FiftyOne config via any of the methods described below, printing your config is a convenient way to ensure that the changes you made have taken effect as you expected.
Modifying your config¶
You can modify your FiftyOne config in a variety of ways. The following sections describe these options in detail.
Order of precedence¶
The following order of precedence is used to assign values to your FiftyOne config settings at runtime:
Config changes applied at runtime by directly editing
fiftyone.config
FIFTYONE_XXX
environment variablesSettings in your JSON config (
~/.fiftyone/config.json
)The default config values
Editing your JSON config¶
You can permanently customize your FiftyOne config by creating a
~/.fiftyone/config.json
file on your machine. The JSON file may contain any
desired subset of config fields that you wish to customize.
For example, a valid config JSON file is:
{
"default_ml_backend": "tensorflow",
"show_progress_bars": true
}
When fiftyone
is imported, any options from your JSON config are applied,
as per the order of precedence described above.
Note
You can customize the location from which your JSON config is read by
setting the FIFTYONE_CONFIG_PATH
environment variable.
Setting environment variables¶
FiftyOne config settings may be customized on a per-session basis by setting
the FIFTYONE_XXX
environment variable(s) for the desired config settings.
When fiftyone
is imported, all config environment variables are applied, as
per the order of precedence described above.
For example, you can customize your FiftyOne config in a Terminal session by issuing the following commands prior to launching your Python interpreter:
export FIFTYONE_DEFAULT_ML_BACKEND=tensorflow
export FIFTYONE_SHOW_PROGRESS_BARS=true
Modifying your config in code¶
You can dynamically modify your FiftyOne config at runtime by editing the
fiftyone.config
object.
Any changes to your FiftyOne config applied via this manner will immediately
take effect for all subsequent calls to fiftyone.config
during your current
session.
1 2 3 4 | import fiftyone as fo fo.config.default_ml_backend = "tensorflow" fo.config.show_progress_bars = True |
Configuring a MongoDB connection¶
By default, FiftyOne is installed with its own MongoDB database distribution. This database is managed by FiftyOne automatically as a service that runs whenever at least one FiftyOne Python client is alive.
Alternatively, you can configure FiftyOne to connect to your own self-managed
MongoDB instance. To do so, simply set the database_uri
property of your
FiftyOne config to any valid
MongoDB connection string URI.
You can achieve this by adding the following entry to your
~/.fiftyone/config.json
file:
{
"database_uri": "mongodb://[username:password@]host[:port]"
}
or you can set the following environment variable:
export FIFTYONE_DATABASE_URI=mongodb://[username:password@]host[:port]
If you are running MongoDB with authentication enabled (the --auth
flag),
FiftyOne must connect as a root user.
You can create a root user with the Mongo shell as follows:
mongo --shell
> use admin
> db.createUser({user: "username", pwd: passwordPrompt(), roles: ["root"]})
You must also add ?authSource=admin
to your database URI:
mongodb://[username:password@]host[:port]/?authSource=admin
Using a different MongoDB version¶
FiftyOne is designed for MongoDB v4.4 or later.
If you wish to connect FiftyOne to a MongoDB database whose version is not
explicitly supported, you will also need to set the database_validation
property of your FiftyOne config to False
to suppress a runtime error that
will otherwise occur.
You can achieve this by adding the following entry to your
~/.fiftyone/config.json
file:
{
"database_validation": false
}
or you can set the following environment variable:
export FIFTYONE_DATABASE_VALIDATION=false
Controlling database migrations¶
If you are working with a shared MongoDB database, you can use database admin privileges to control which clients are allowed to migrate the shared database.
Example custom database usage¶
In order to use a custom MongoDB database with FiftyOne, you must manually start the database before importing FiftyOne. MongoDB provides a variety of options for this, including running the database as a daemon automatically.
In the simplest case, you can just run mongod
in one shell:
mkdir -p /path/for/db
mongod --dbpath /path/for/db
Then, in another shell, configure the database URI and launch FiftyOne:
export FIFTYONE_DATABASE_URI=mongodb://localhost
import fiftyone as fo
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset("quickstart")
session = fo.launch_app(dataset)
Database migrations¶
New FiftyOne versions occasionally introduce data model changes that require database migrations when you upgrade or downgrade.
By default, database upgrades happen automatically in two steps:
Database: when you import FiftyOne for the first time using a newer version of the Python package, the database’s version is automatically updated to match your client version
Datasets are lazily migrated to the current database version on a per-dataset basis whenever you load the dataset for the first time using a newer version of the FiftyOne package
Database downgrades must be manually performed. See this page for instructions.
You can use the fiftyone migrate command to view the current versions of your client, database, and datasets:
# View your client, database, and dataset versions
fiftyone migrate --info
Client version: 0.16.6
Compatible versions: >=0.16.3,<0.17
Database version: 0.16.6
dataset version
--------------------------- ---------
bdd100k-validation 0.16.5
quickstart 0.16.5
...
Restricting migrations¶
You can use the database_admin
config setting to control whether a client is
allowed to upgrade/downgrade your FiftyOne database. The default is True
,
which means that upgrades are automatically performed when you connect to your
database with newer Python client versions.
If you set database_admin
to False
, your client will never cause the
database to be migrated to a new version. Instead, you’ll see the following
behavior:
If your client is compatible with the current database version, you will be allowed to connect to the database and use FiftyOne
If your client is not compatible with the current database version, you will see an informative error message when you import the library
You can restrict migrations by adding the following entry to your
~/.fiftyone/config.json
file:
{
"database_admin": false
}
or by setting the following environment variable:
export FIFTYONE_DATABASE_ADMIN=false
Note
A common pattern when working with
custom/shared MongoDB databases is
to adopt a convention that all non-administrators set their
database_admin
config setting to False
to ensure that they cannot
trigger automatic database upgrades by connecting to the database with
newer Python client versions.
Coordinating a migration¶
If you are working in an environment where multiple services are connecting to your MongoDB database at any given time, use this strategy to upgrade your deployment:
Ensure that all clients are running without database admin privileges, e.g., by adding this to their
~/.fiftyone/config.json
:
{
"database_admin": false
}
Perform a test upgrade of one client and ensure that it is compatible with your current database version:
# In a test environment
pip install --upgrade fiftyone
# View client's compatibility info
fiftyone migrate --info
import fiftyone as fo
# Convince yourself that the new client can load a dataset
dataset = fo.load_dataset(...)
Now upgrade the client version used by all services:
# In all client environments
pip install --upgrade fiftyone
Once all services are running the new client version, upgrade the database with admin privileges:
export FIFTYONE_DATABASE_ADMIN=true
pip install --upgrade fiftyone
fiftyone migrate --all
Note
Newly created datasets will always bear the
version
of the Python
client that created them, which may differ from your database’s version
if you are undergoing a migration.
If the new client’s version is not in the compatibility range for the old clients that are still in use, the old clients will not be able to load the new datasets.
Therefore, it is recommended to upgrade all clients as soon as possible!
Configuring a timezone¶
By default, FiftyOne loads all datetimes in FiftyOne datasets as naive
datetime
objects expressed in UTC time.
However, you can configure FiftyOne to express datetimes in a specific timezone
by setting the timezone
property of your FiftyOne config.
The timezone
property can be set to any timezone string supported by
pytz.timezone()
, or "local"
to use your current local timezone.
For example, you could set the FIFTYONE_TIMEZONE
environment variable:
# Local timezone
export FIFTYONE_TIMEZONE=local
# US Eastern timezone
export FIFTYONE_TIMEZONE=US/Eastern
Or, you can even dynamically change the timezone while you work in Python:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | from datetime import datetime import fiftyone as fo sample = fo.Sample(filepath="image.png", created_at=datetime.utcnow()) dataset = fo.Dataset() dataset.add_sample(sample) print(sample.created_at) # 2021-08-24 20:24:09.723021 fo.config.timezone = "local" dataset.reload() print(sample.created_at) # 2021-08-24 16:24:09.723000-04:00 |
Note
The timezone
setting does not affect the internal database representation
of datetimes, which are always stored as UTC timestamps.
Configuring the App¶
The FiftyOne App can also be configured in various ways.
A new copy of your App config is applied to each Session
object that is
created when you launch the App. A session’s config can be inspected and
modified via the session.config
property.
1 2 3 4 5 6 7 8 | import fiftyone as fo import fiftyone.zoo as foz dataset = foz.load_zoo_dataset("quickstart") print(fo.app_config) session = fo.launch_app(dataset) print(session.config) |
Note
For changes to a live session’s config to take effect in the App, you must
call session.refresh()
or
invoke another state-updating action such as session.view = my_view
.
The FiftyOne App can be configured in the ways described below:
Config field |
Environment variable |
Default value |
Description |
---|---|---|---|
|
|
|
Whether to color labels by their field name ( |
|
|
See below |
A list of browser supported color strings from which the App should draw from when drawing labels (e.g., object bounding boxes). |
|
|
|
The colorscale to use when rendering heatmaps in the App. See this section for more details. |
|
|
|
Default if a user hasn’t selected a query performance mode in their current session. See this section for more details. |
|
|
|
Whether to disable frame filtering for video datasets in the App’s grid view. See this section for more details. |
|
|
|
Whether to show the query performance toggle in the UI for users to select. See this section for more details. |
|
|
|
The zoom level of the App’s sample grid. Larger values result in larger samples (and thus
fewer samples in the grid). Supported values are |
|
|
|
Whether to loop videos by default in the expanded sample view. |
|
|
|
Whether to fall back to the default media field ( |
|
|
|
Whether to independently coloy keypoint points by their index |
|
|
|
The height of App instances displayed in notebook cells. |
|
|
|
A URL string to override the default server URL. Useful for configuring the session through a reverse proxy in notebook environments. |
|
|
|
Whether to show confidences when rendering labels in the App’s expanded sample view. |
|
|
|
Whether to show indexes when rendering labels in the App’s expanded sample view. |
|
|
|
Whether to show the label value when rendering detection labels in the App’s expanded sample view. |
|
|
|
Whether to show keypoint skeletons, if available. |
|
|
|
Whether to show the tooltip when hovering over labels in the App’s expanded sample view. |
|
|
|
The default theme to use in the App. Supported values are |
|
|
|
Whether to use the frame number instead of a timestamp in the expanded sample view. Only applicable to video samples. |
|
N/A |
|
A dict of plugin configurations. See this section for details. |
Viewing your App config¶
You can print your App config at any time via the Python library and the CLI:
import fiftyone as fo
# Print your current App config
print(fo.app_config)
# Print a specific App config field
print(fo.app_config.show_label)
{
"color_by": "field",
"color_pool": [
"#ee0000",
"#ee6600",
"#993300",
"#996633",
"#999900",
"#009900",
"#003300",
"#009999",
"#000099",
"#0066ff",
"#6600ff",
"#cc33cc",
"#777799"
],
"colorscale": "viridis",
"frame_stream_size": 1000,
"grid_zoom": 5,
"loop_videos": false,
"media_fallback": false,
"default_query_performance": true,
"disable_frame_filtering": false,
"enable_query_performance": true,
"multicolor_keypoints": false,
"notebook_height": 800,
"proxy_url": None,
"show_confidence": true,
"show_index": true,
"show_label": true,
"show_skeletons": true,
"show_tooltip": true,
"sidebar_mode": "fast",
"theme": "browser",
"use_frame_number": false,
"plugins": {},
}
True
# Print your current App config
fiftyone app config
# Print a specific App config field
fiftyone app config show_label
{
"color_by": "field",
"color_pool": [
"#ee0000",
"#ee6600",
"#993300",
"#996633",
"#999900",
"#009900",
"#003300",
"#009999",
"#000099",
"#0066ff",
"#6600ff",
"#cc33cc",
"#777799"
],
"colorscale": "viridis",
"frame_stream_size": 1000,
"grid_zoom": 5,
"default_query_performance": true,
"disable_frame_filtering": false,
"enable_query_performance": true,
"loop_videos": false,
"media_fallback": false,
"multicolor_keypoints": false,
"notebook_height": 800,
"proxy_url": None,
"show_confidence": true,
"show_index": true,
"show_label": true,
"show_skeletons": true,
"show_tooltip": true,
"sidebar_mode": "fast",
"theme": "browser",
"use_frame_number": false,
"plugins": {},
}
True
Note
If you have customized your App config via any of the methods described below, printing your config is a convenient way to ensure that the changes you made have taken effect as you expected.
Modifying your App config¶
You can modify your App config in a variety of ways. The following sections describe these options in detail.
Note
Did you know? You can also configure the behavior of the App on a per-dataset basis by customizing your dataset’s App config.
Order of precedence¶
The following order of precedence is used to assign values to your App config settings at runtime:
Config settings of a
Session
instance in questionApp config settings applied at runtime by directly editing
fiftyone.app_config
FIFTYONE_APP_XXX
environment variablesSettings in your JSON App config (
~/.fiftyone/app_config.json
)The default App config values
Launching the App with a custom config¶
You can launch the FiftyOne App with a customized App config on a one-off basis via the following pattern:
1 2 3 4 5 6 7 8 9 10 11 | import fiftyone as fo import fiftyone.zoo as foz dataset = foz.load_zoo_dataset("quickstart") # Create a custom App config app_config = fo.app_config.copy() app_config.show_confidence = False app_config.show_label = False session = fo.launch_app(dataset, config=app_config) |
You can also configure a live Session
by editing its
session.config
property and
calling session.refresh()
to
apply the changes:
1 2 3 4 | # Customize the config of a live session session.config.show_confidence = True session.config.show_label = True session.refresh() # must refresh after edits |
Editing your JSON App config¶
You can permanently customize your App config by creating a
~/.fiftyone/app_config.json
file on your machine. The JSON file may contain
any desired subset of config fields that you wish to customize.
For example, a valid App config JSON file is:
{
"show_confidence": false,
"show_label": false
}
When fiftyone
is imported, any options from your JSON App config are applied,
as per the order of precedence described above.
Note
You can customize the location from which your JSON App config is read by
setting the FIFTYONE_APP_CONFIG_PATH
environment variable.
Setting App environment variables¶
App config settings may be customized on a per-session basis by setting the
FIFTYONE_APP_XXX
environment variable(s) for the desired App config settings.
When fiftyone
is imported, all App config environment variables are applied,
as per the order of precedence described above.
For example, you can customize your App config in a Terminal session by issuing the following commands prior to launching your Python interpreter:
export FIFTYONE_APP_SHOW_CONFIDENCE=false
export FIFTYONE_APP_SHOW_LABEL=false
Modifying your App config in code¶
You can dynamically modify your App config at runtime by editing the
fiftyone.app_config
object.
Any changes to your App config applied via this manner will immediately
take effect for all subsequent calls to fiftyone.app_config
during your
current session.
1 2 3 4 | import fiftyone as fo fo.app_config.show_confidence = False fo.app_config.show_label = False |
Configuring plugins¶
You can store system-wide plugin configurations under the plugins
key of your
App config.
Builtin plugins that you can configure include:
The builtin Map panel
The builtin 3D visualizer
Any custom plugins that you’ve registered
For example, you may add the following to your JSON App config
(~/.fiftyone/app_config.json
) to register a Mapbox token globally on your
system:
{
"plugins": {
"map": {
"mapboxAccessToken": "XXXXXXXX"
}
}
}
Note
You can also store dataset-specific plugin settings by storing any subset of the above values on a dataset’s App config.
Configuring a proxy URL¶
When running FiftyOne in a cloud machine, such as a
SageMaker Notebook, a
proxy_url
should be set in your
FiftyOne App config before launching the App
in order for browser windows or notebook cells to point to a correct App URL.
For SageMaker Notebooks, the
below code snippet shows how to configure the proxy based on your instance.
import fiftyone as fo
# before launching the App, configure a proxy_url
fo.app_config.proxy_url = "https://<myinstance>.notebook.<region>.sagemaker.aws/proxy/<port>/"
session = fo.launch_app(port=<port>)