Brain Plugin#
A plugin that contains utilities for working with the FiftyOne Brain.
Installation#
fiftyone plugins download \
https://github.com/voxel51/fiftyone-plugins \
--plugin-names @voxel51/brain
Refer to the main README for more information about managing downloaded plugins and developing plugins locally.
Configuration#
You can optionally configure which license(s) models must be distributed under in order to be made available by this plugin’s operators by setting the following plugin secret:
export FIFTYONE_ZOO_ALLOWED_MODEL_LICENSES="MIT,Apache 2.0"
Run the following command to see the available models and their licenses:
fiftyone zoo models list
You can also provide a specific list of models that should be made available by this plugin’s operators by setting the following plugin secret:
export FIFTYONE_ZOO_ALLOWED_MODEL_NAMES="clip-vit-base32-torch,zero-shot-classification-transformer-torch"
Usage#
Launch the App:
import fiftyone as fo
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset("quickstart")
session = fo.launch_app(dataset)
Press
`
or click theBrowse operations
action to open the Operators listSelect any of the operators listed below!
Operators#
compute_visualization#
You can use this operator to create embeddings visualizations for your datasets.
This operator is essentially a wrapper around compute_visualization():
import fiftyone.brain as fob
fob.compute_visualization(dataset_or_view, brain_key=brain_key, ...)
where the operator’s form allows you to configure the brain key and all other relevant parameters.
compute_similarity#
You can use this operator to create similarity indexes for your datasets.
This operator is essentially a wrapper around compute_similarity():
import fiftyone.brain as fob
fob.compute_similarity(dataset_or_view, brain_key=brain_key, ...)
where the operator’s form allows you to configure the brain key and all other relevant parameters.
sort_by_similarity#
You can use this operator to perform image/text similarity queries against any similarity indexes on your dataset.
This operator is essentially a wrapper around sort_by_similarity():
view = dataset.sort_by_similarity(query, brain_key=brain_key, k=k, ...)
This operator supports all of the following similarity queries:
Searching by image similarity based on the currently selected samples in the App
Searching by image similarity to an uploaded image
Searching by text similarity
add_similar_samples#
You can use this operator to retrieve similar samples from another dataset to add to your current dataset.
This operator is essentially a wrapper around the following pseudocode:
view = src_dataset.sort_by_similarity(query, brain_key=brain_key, k=k)
dataset.add_samples(view)
This operator supports both image and text similarity queries, depending on whether you have samples currently selected in the App when you launch the operator:
If one or more samples currently selected, an image similarity query will be performed to retrieve similar images from
src_dataset
to your currently selected samples indataset
Otherwise, a text similarity query will be performed to retrieve new images from
src_dataset
that match the provided text prompt
compute_uniqueness#
You can use this operator to compute uniqueness for your datasets.
This operator is essentially a wrapper around compute_uniqueness():
import fiftyone.brain as fob
fob.compute_uniqueness(dataset_or_view, uniqueness_field, ...)
where the operator’s form allows you to configure all relevant parameters.
compute_mistakenness#
You can use this operator to compute mistakenness for your datasets.
This operator is essentially a wrapper around compute_mistakenness():
import fiftyone.brain as fob
fob.compute_mistakenness(dataset_or_view, pred_field, label_field, ...)
where the operator’s form allows you to configure all relevant parameters.
compute_hardness#
You can use this operator to compute hardness for your datasets.
This operator is essentially a wrapper around compute_hardness():
import fiftyone.brain as fob
fob.compute_hardness(dataset_or_view, label_field, ...)
where the operator’s form allows you to configure all relevant parameters.
find_exact_duplicates + deduplicate_exact_duplicates#
You can use these operators to detect and delete samples with exact duplicate media in a dataset.
The find_exact_duplicates
operator is essentially a wrapper around
compute_exact_duplicates():
import fiftyone.brain as fob
results = fob.compute_exact_duplicates(dataset_or_view)
print(results)
Executing this operator will create two saved views on your dataset:
exact duplicates
: a view that contains all samples whose media is an exact duplicate of one or more other samplesrepresentatives of exact duplicates
: a view that contains one representative sample from each group of exact duplicates
Executing the deduplicate_exact_duplicates
operator will delete all of the
exact duplicate samples from the exact duplicates
view except the one
representiatve from each group in the representatives of exact duplicates
view.
find_near_duplicates + deduplicate_near_duplicates#
You can use these operators to detect and delete near duplicate samples in a dataset.
The find_near_duplicates
operator is essentially a wrapper around
compute_near_duplicates():
import fiftyone.brain as fob
results = fob.compute_near_duplicates(dataset_or_view, threshold=threshold, ...)
print(results)
where the operator’s form allows you to configure the embeddings to use to compute near duplicates along with an appropriate distance threshold to use to detect duplicates in embeddings space.
Executing this operator will create two saved views on your dataset:
near duplicates
: a view that contains all samples that are near duplicates of one or more other samplesrepresentatives of near duplicates
: a view that contains one representative sample from each group of near duplicates
Executing the deduplicate_near_duplicates
operator will delete all of the
near duplicate samples from the near duplicates
view except the one
representiatve from each group in the representatives of near duplicates
view.
get_brain_info#
You can use this operator to get information about brain runs.
This operator is essentially a wrapper around get_brain_info():
info = dataset_or_view.get_brain_info(brain_key)
print(info)
load_brain_view#
You can use this operator to load the view on which a brain run was performed.
This operator is essentially a wrapper around load_brain_view():
view = dataset.load_brain_view(brain_key)
rename_brain_run#
You can use this operator to rename brain runs.
This operator is essentially a wrapper around rename_brain_run():
dataset_or_view.rename_brain_run(brain_key, new_brain_key)
delete_brain_run#
You can use this operator to delete brain runs.
This operator is essentially a wrapper around delete_brain_run():
dataset_or_view.delete_brain_run(brain_key)