fiftyone.zoo¶
Module contents¶
The FiftyOne Zoo.
Copyright 2017-2023, Voxel51, Inc. voxel51.com
Classes:
Dictionary that remembers insertion order |
|
|
Class containing info about a dataset in the FiftyOne Dataset Zoo. |
|
Class containing info about a split of a dataset in the FiftyOne Dataset Zoo. |
Base class for datasets made available in the FiftyOne Dataset Zoo. |
|
Class representing a zoo dataset that no longer exists in the FiftyOne Dataset Zoo. |
|
defaultdict(default_factory[, …]) –> dict with default factory |
|
Mixin class for Config classes of |
|
|
Class describing a model in the FiftyOne Model Zoo. |
|
Class that describes the collection of models in the FiftyOne Model Zoo. |
Functions:
|
Returns the list of available datasets in the FiftyOne Dataset Zoo. |
|
Returns information about the zoo datasets that have been downloaded. |
|
Downloads the dataset of the given name from the FiftyOne Dataset Zoo. |
|
Loads the dataset of the given name from the FiftyOne Dataset Zoo as a |
|
Returns the directory containing the given zoo dataset. |
|
Loads the |
|
Returns the |
|
Deletes the zoo dataset from local disk, if necessary. |
|
Deep copy operation on arbitrary Python objects. |
|
Returns the list of available models in the FiftyOne Model Zoo. |
Returns information about the zoo models that have been downloaded. |
|
|
Determines whether the zoo model of the given name is downloaded. |
|
Downloads the model of the given name from the FiftyOne Dataset Zoo. |
|
Installs any package requirements for the zoo model with the given name. |
|
Ensures that the package requirements for the zoo model with the given name are satisfied. |
|
Loads the model of the given name from the FiftyOne Model Zoo. |
|
Returns the path to the zoo model on disk. |
|
Returns the |
|
Deletes the zoo model from local disk, if necessary. |
Exceptions:
Exception raised when an invalid Config instance is encountered. |
-
class
fiftyone.zoo.
OrderedDict
¶ Bases:
dict
Dictionary that remembers insertion order
Methods:
clear
()copy
()Create a new ordered dictionary with keys from iterable and values set to value.
Return the value for key if key is in the dictionary, else default.
items
()keys
()Move an existing element to the end (or beginning if last is false).
pop
(k[,d])value.
Remove and return a (key, value) pair from the dictionary.
Insert key with a value of default if key is not in the dictionary.
update
([E, ]**F)If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values
()-
clear
() → None. Remove all items from od.¶
-
popitem
()¶ Remove and return a (key, value) pair from the dictionary.
Pairs are returned in LIFO order if last is true or FIFO order if false.
-
move_to_end
()¶ Move an existing element to the end (or beginning if last is false).
Raise KeyError if the element does not exist.
-
update
([E, ]**F) → None. Update D from dict/iterable E and F.¶ If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
-
keys
() → a set-like object providing a view on D's keys¶
-
items
() → a set-like object providing a view on D's items¶
-
values
() → an object providing a view on D's values¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding¶ value. If key is not found, d is returned if given, otherwise KeyError is raised.
-
setdefault
()¶ Insert key with a value of default if key is not in the dictionary.
Return the value for key if key is in the dictionary, else default.
-
copy
() → a shallow copy of od¶
-
fromkeys
()¶ Create a new ordered dictionary with keys from iterable and values set to value.
-
get
()¶ Return the value for key if key is in the dictionary, else default.
-
-
fiftyone.zoo.
list_zoo_datasets
(tags=None, source=None)¶ Returns the list of available datasets in the FiftyOne Dataset Zoo.
Example usage:
import fiftyone as fo import fiftyone.zoo as foz # # List all zoo datasets # names = foz.list_zoo_datasets() print(names) # # List all zoo datasets with (both of) the specified tags # names = foz.list_zoo_datasets(tags=["image", "detection"]) print(names) # # List all zoo datasets available via the given source # names = foz.list_zoo_datasets(source="torch") print(names)
- Parameters
tags (None) – only include datasets that have the specified tag or list of tags
source (None) – only include datasets available via the given source or list of sources
- Returns
a sorted list of dataset names
-
fiftyone.zoo.
list_downloaded_zoo_datasets
(base_dir=None)¶ Returns information about the zoo datasets that have been downloaded.
- Parameters
base_dir (None) – the base directory to search for downloaded datasets. By default,
fo.config.dataset_zoo_dir
is used- Returns
a dict mapping dataset names to (dataset dir,
ZooDatasetInfo
) tuples
-
fiftyone.zoo.
download_zoo_dataset
(name, split=None, splits=None, dataset_dir=None, overwrite=False, cleanup=True, **kwargs)¶ Downloads the dataset of the given name from the FiftyOne Dataset Zoo.
Any dataset splits that already exist in the specified directory are not re-downloaded, unless
overwrite == True
is specified.- Parameters
name – the name of the zoo dataset to download. Call
list_zoo_datasets()
to see the available datasetssplit (None) –
("train", "validation", "test")
. If neithersplit
norsplits
are provided, all available splits are downloaded. Consult the documentation for theZooDataset
you specified to see the supported splitssplits (None) – a list of splits to download, if applicable. Typical values are
("train", "validation", "test")
. If neithersplit
norsplits
are provided, all available splits are downloaded. Consult the documentation for theZooDataset
you specified to see the supported splitsdataset_dir (None) – the directory into which to download the dataset. By default, it is downloaded to a subdirectory of
fiftyone.config.dataset_zoo_dir
overwrite (False) – whether to overwrite any existing files
cleanup (True) – whether to cleanup any temporary files generated during download
**kwargs – optional arguments for the
ZooDataset
constructor
- Returns
tuple of
info: the
ZooDatasetInfo
for the datasetdataset_dir: the directory containing the dataset
-
fiftyone.zoo.
load_zoo_dataset
(name, split=None, splits=None, label_field=None, dataset_name=None, dataset_dir=None, download_if_necessary=True, drop_existing_dataset=False, overwrite=False, cleanup=True, **kwargs)¶ Loads the dataset of the given name from the FiftyOne Dataset Zoo as a
fiftyone.core.dataset.Dataset
.By default, the dataset will be downloaded if it does not already exist in the specified directory.
If you do not specify a custom
dataset_name
and you have previously loaded the same zoo dataset and split(s) into FiftyOne, the existingfiftyone.core.dataset.Dataset
will be returned.- Parameters
name – the name of the zoo dataset to load. Call
list_zoo_datasets()
to see the available datasetssplit (None) –
("train", "validation", "test")
. If neithersplit
norsplits
are provided, all available splits are loaded. Consult the documentation for theZooDataset
you specified to see the supported splitssplits (None) – a list of splits to load, if applicable. Typical values are
("train", "validation", "test")
. If neithersplit
norsplits
are provided, all available splits are loaded. Consult the documentation for theZooDataset
you specified to see the supported splitslabel_field (None) – the label field (or prefix, if the dataset contains multiple label fields) in which to store the dataset’s labels. By default, this is
"ground_truth"
if the dataset contains a single label field. If the dataset contains multiple label fields and this value is not provided, the labels will be stored under dataset-specific field namesdataset_name (None) – an optional name to give the returned
fiftyone.core.dataset.Dataset
. By default, a name will be constructed based on the dataset and split(s) you are loadingdataset_dir (None) – the directory in which the dataset is stored or will be downloaded. By default, the dataset will be located in
fiftyone.config.dataset_zoo_dir
download_if_necessary (True) – whether to download the dataset if it is not found in the specified dataset directory
drop_existing_dataset (False) – whether to drop an existing dataset with the same name if it exists
overwrite (False) – whether to overwrite any existing files if the dataset is to be downloaded
cleanup (True) – whether to cleanup any temporary files generated during download
**kwargs – optional arguments to pass to the
fiftyone.utils.data.importers.DatasetImporter
constructor. Ifdownload_if_necessary == True
, thenkwargs
can also contain arguments fordownload_zoo_dataset()
- Returns
-
fiftyone.zoo.
find_zoo_dataset
(name, split=None)¶ Returns the directory containing the given zoo dataset.
If a
split
is provided, the path to the dataset split is returned; otherwise, the path to the root directory is returned.The dataset must be downloaded. Use
download_zoo_dataset()
to download datasets.- Parameters
name – the name of the zoo dataset
split (None) –
- Returns
the directory containing the dataset
- Raises
ValueError – if the dataset or split does not exist or has not been downloaded
-
fiftyone.zoo.
load_zoo_dataset_info
(name, dataset_dir=None)¶ Loads the
ZooDatasetInfo
for the specified zoo dataset.The dataset must be downloaded. Use
download_zoo_dataset()
to download datasets.- Parameters
name – the name of the zoo dataset
dataset_dir (None) – the directory in which the dataset is stored. By default, the dataset is located in
fiftyone.config.dataset_zoo_dir
- Returns
the
ZooDatasetInfo
for the dataset- Raises
ValueError – if the dataset has not been downloaded
-
fiftyone.zoo.
get_zoo_dataset
(name, **kwargs)¶ Returns the
ZooDataset
instance for the dataset with the given name.If the dataset is available from multiple sources, the default source is used.
- Parameters
name – the name of the zoo dataset
**kwargs – optional arguments for
ZooDataset
- Returns
the
ZooDataset
instance
-
fiftyone.zoo.
delete_zoo_dataset
(name, split=None)¶ Deletes the zoo dataset from local disk, if necessary.
If a
split
is provided, only that split is deleted.- Parameters
name – the name of the zoo dataset
split (None) –
-
class
fiftyone.zoo.
ZooDatasetInfo
(zoo_dataset, dataset_type, num_samples, downloaded_splits=None, parameters=None, classes=None)¶ Bases:
eta.core.serial.Serializable
Class containing info about a dataset in the FiftyOne Dataset Zoo.
- Parameters
zoo_dataset – the
ZooDataset
instance for the datasetdataset_type – the
fiftyone.types.Dataset
type of the datasetnum_samples – the total number of samples in all downloaded splits of the dataset
downloaded_splits (None) – a dict of
ZooDatasetSplitInfo
instances describing the downloaded splits of the dataset, if applicableparameters (None) – a dict of parameters for the dataset
classes (None) – a list of class label strings
Methods:
add_split
(split_info)Adds the split to the dataset.
Returns a list of class attributes to be serialized.
copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
from_dict
(d)Loads a
ZooDatasetInfo
from a JSON dictionary.from_json
(json_path[, upgrade, warn_deprecated])Loads a
ZooDatasetInfo
from a JSON file on disk.from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
Returns the
fiftyone.types.Dataset
type instance for the dataset.Returns the
ZooDataset
instance for the dataset.is_split_downloaded
(split)Whether the given dataset split is downloaded.
remove_split
(split)Removes the split from the dataset.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
Attributes:
The fully-qualified class string of the
fiftyone.types.Dataset
type.The name of the dataset.
A tuple of supported splits for the dataset, or None if the dataset does not have splits.
The fully-qualified class string for the
ZooDataset
of the dataset.-
property
name
¶ The name of the dataset.
-
property
zoo_dataset
¶ The fully-qualified class string for the
ZooDataset
of the dataset.
-
property
dataset_type
¶ The fully-qualified class string of the
fiftyone.types.Dataset
type.
-
property
supported_splits
¶ A tuple of supported splits for the dataset, or None if the dataset does not have splits.
-
get_zoo_dataset
()¶ Returns the
ZooDataset
instance for the dataset.- Returns
a
ZooDataset
instance
-
get_dataset_type
()¶ Returns the
fiftyone.types.Dataset
type instance for the dataset.- Returns
a
fiftyone.types.Dataset
instance
-
is_split_downloaded
(split)¶ Whether the given dataset split is downloaded.
- Parameters
split – the dataset split
- Returns
True/False
-
add_split
(split_info)¶ Adds the split to the dataset.
- Parameters
split_info – a
ZooDatasetSplitInfo
-
remove_split
(split)¶ Removes the split from the dataset.
- Parameters
split – the name of the split
-
attributes
()¶ Returns a list of class attributes to be serialized.
- Returns
a list of class attributes
-
classmethod
from_dict
(d)¶ Loads a
ZooDatasetInfo
from a JSON dictionary.- Parameters
d – a JSON dictionary
- Returns
-
classmethod
from_json
(json_path, upgrade=False, warn_deprecated=False)¶ Loads a
ZooDatasetInfo
from a JSON file on disk.- Parameters
json_path – path to JSON file
upgrade (False) – whether to upgrade the JSON file on disk if any migrations were necessary
warn_deprecated (False) – whether to issue a warning if the dataset has a deprecated format
- Returns
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
class
fiftyone.zoo.
ZooDatasetSplitInfo
(split, num_samples)¶ Bases:
eta.core.serial.Serializable
Class containing info about a split of a dataset in the FiftyOne Dataset Zoo.
- Parameters
split – the name of the split
num_samples – the number of samples in the split
Methods:
Returns a list of class attributes to be serialized.
copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
from_dict
(d)Loads a
ZooDatasetSplitInfo
from a JSON dictionary.from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
-
attributes
()¶ Returns a list of class attributes to be serialized.
- Returns
a list of class attributes
-
classmethod
from_dict
(d)¶ Loads a
ZooDatasetSplitInfo
from a JSON dictionary.- Parameters
d – a JSON dictionary
- Returns
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
class
fiftyone.zoo.
ZooDataset
¶ Bases:
object
Base class for datasets made available in the FiftyOne Dataset Zoo.
Methods:
download_and_prepare
([dataset_dir, split, …])Downloads the dataset and prepares it for use.
get_info_path
(dataset_dir)Returns the path to the
ZooDatasetInfo
for the dataset.get_split_dir
(dataset_dir, split)Returns the directory for the given split of the dataset.
has_split
(split)Whether the dataset has the given split.
has_tag
(tag)Whether the dataset has the given tag.
load_info
(dataset_dir[, upgrade, …])Loads the
ZooDatasetInfo
from the given dataset directory.Attributes:
Whether the dataset has patches that may need to be applied to already downloaded files.
Whether the dataset has splits.
Whether the dataset has tags.
A dict of default kwargs to pass to this dataset’s
fiftyone.utils.data.importers.DatasetImporter
.The name of the dataset.
An optional dict of parameters describing the configuration of the zoo dataset when it was downloaded.
Whether this dataset requires some files to be manually downloaded by the user before the dataset can be loaded.
A tuple of supported splits for the dataset, or None if the dataset does not have splits.
Whether the dataset supports downloading partial subsets of its splits.
A tuple of tags for the dataset.
-
property
name
¶ The name of the dataset.
A tuple of tags for the dataset.
Whether the dataset has tags.
-
property
parameters
¶ An optional dict of parameters describing the configuration of the zoo dataset when it was downloaded.
-
property
supported_splits
¶ A tuple of supported splits for the dataset, or None if the dataset does not have splits.
-
property
has_splits
¶ Whether the dataset has splits.
-
property
has_patches
¶ Whether the dataset has patches that may need to be applied to already downloaded files.
-
property
supports_partial_downloads
¶ Whether the dataset supports downloading partial subsets of its splits.
-
property
requires_manual_download
¶ Whether this dataset requires some files to be manually downloaded by the user before the dataset can be loaded.
-
property
importer_kwargs
¶ A dict of default kwargs to pass to this dataset’s
fiftyone.utils.data.importers.DatasetImporter
.
-
has_tag
(tag)¶ Whether the dataset has the given tag.
- Parameters
tag – the tag
- Returns
True/False
-
has_split
(split)¶ Whether the dataset has the given split.
- Parameters
split – the dataset split
- Returns
True/False
-
get_split_dir
(dataset_dir, split)¶ Returns the directory for the given split of the dataset.
- Parameters
dataset_dir – the dataset directory
split – the dataset split
- Returns
the directory that will/does hold the specified split
-
static
load_info
(dataset_dir, upgrade=True, warn_deprecated=False)¶ Loads the
ZooDatasetInfo
from the given dataset directory.- Parameters
dataset_dir – the directory in which to construct the dataset
upgrade (True) – whether to upgrade the JSON file on disk if any migrations were necessary
warn_deprecated (False) – whether to issue a warning if the dataset has a deprecated format
- Returns
the
ZooDatasetInfo
for the dataset
-
static
get_info_path
(dataset_dir)¶ Returns the path to the
ZooDatasetInfo
for the dataset.- Parameters
dataset_dir – the dataset directory
- Returns
the path to the
ZooDatasetInfo
-
download_and_prepare
(dataset_dir=None, split=None, splits=None, overwrite=False, cleanup=True)¶ Downloads the dataset and prepares it for use.
If the requested splits have already been downloaded, they are not re-downloaded unless
overwrite
is True.- Parameters
dataset_dir (None) – the directory in which to construct the dataset. By default, it is written to a subdirectory of
fiftyone.config.dataset_zoo_dir
split (None) –
split
norsplits
are provided, the full dataset is downloadedsplits (None) – a list of splits to download, if applicable. If neither
split
norsplits
are provided, the full dataset is downloadedoverwrite (False) – whether to overwrite any existing files
cleanup (True) – whether to cleanup any temporary files generated during download
- Returns
tuple of
info: the
ZooDatasetInfo
for the datasetdataset_dir: the directory containing the dataset
-
property
-
class
fiftyone.zoo.
DeprecatedZooDataset
¶ Bases:
fiftyone.zoo.datasets.ZooDataset
Class representing a zoo dataset that no longer exists in the FiftyOne Dataset Zoo.
Methods:
download_and_prepare
([dataset_dir, split, …])Downloads the dataset and prepares it for use.
get_info_path
(dataset_dir)Returns the path to the
ZooDatasetInfo
for the dataset.get_split_dir
(dataset_dir, split)Returns the directory for the given split of the dataset.
has_split
(split)Whether the dataset has the given split.
has_tag
(tag)Whether the dataset has the given tag.
load_info
(dataset_dir[, upgrade, …])Loads the
ZooDatasetInfo
from the given dataset directory.Attributes:
Whether the dataset has patches that may need to be applied to already downloaded files.
Whether the dataset has splits.
Whether the dataset has tags.
A dict of default kwargs to pass to this dataset’s
fiftyone.utils.data.importers.DatasetImporter
.The name of the dataset.
An optional dict of parameters describing the configuration of the zoo dataset when it was downloaded.
Whether this dataset requires some files to be manually downloaded by the user before the dataset can be loaded.
A tuple of supported splits for the dataset, or None if the dataset does not have splits.
Whether the dataset supports downloading partial subsets of its splits.
A tuple of tags for the dataset.
-
property
name
¶ The name of the dataset.
-
property
supported_splits
¶ A tuple of supported splits for the dataset, or None if the dataset does not have splits.
-
download_and_prepare
(dataset_dir=None, split=None, splits=None, overwrite=False, cleanup=True)¶ Downloads the dataset and prepares it for use.
If the requested splits have already been downloaded, they are not re-downloaded unless
overwrite
is True.- Parameters
dataset_dir (None) – the directory in which to construct the dataset. By default, it is written to a subdirectory of
fiftyone.config.dataset_zoo_dir
split (None) –
split
norsplits
are provided, the full dataset is downloadedsplits (None) – a list of splits to download, if applicable. If neither
split
norsplits
are provided, the full dataset is downloadedoverwrite (False) – whether to overwrite any existing files
cleanup (True) – whether to cleanup any temporary files generated during download
- Returns
tuple of
info: the
ZooDatasetInfo
for the datasetdataset_dir: the directory containing the dataset
-
static
get_info_path
(dataset_dir)¶ Returns the path to the
ZooDatasetInfo
for the dataset.- Parameters
dataset_dir – the dataset directory
- Returns
the path to the
ZooDatasetInfo
-
get_split_dir
(dataset_dir, split)¶ Returns the directory for the given split of the dataset.
- Parameters
dataset_dir – the dataset directory
split – the dataset split
- Returns
the directory that will/does hold the specified split
-
property
has_patches
¶ Whether the dataset has patches that may need to be applied to already downloaded files.
-
has_split
(split)¶ Whether the dataset has the given split.
- Parameters
split – the dataset split
- Returns
True/False
-
property
has_splits
¶ Whether the dataset has splits.
-
has_tag
(tag)¶ Whether the dataset has the given tag.
- Parameters
tag – the tag
- Returns
True/False
Whether the dataset has tags.
-
property
importer_kwargs
¶ A dict of default kwargs to pass to this dataset’s
fiftyone.utils.data.importers.DatasetImporter
.
-
static
load_info
(dataset_dir, upgrade=True, warn_deprecated=False)¶ Loads the
ZooDatasetInfo
from the given dataset directory.- Parameters
dataset_dir – the directory in which to construct the dataset
upgrade (True) – whether to upgrade the JSON file on disk if any migrations were necessary
warn_deprecated (False) – whether to issue a warning if the dataset has a deprecated format
- Returns
the
ZooDatasetInfo
for the dataset
-
property
parameters
¶ An optional dict of parameters describing the configuration of the zoo dataset when it was downloaded.
-
property
requires_manual_download
¶ Whether this dataset requires some files to be manually downloaded by the user before the dataset can be loaded.
-
property
supports_partial_downloads
¶ Whether the dataset supports downloading partial subsets of its splits.
A tuple of tags for the dataset.
-
property
-
class
fiftyone.zoo.
defaultdict
¶ Bases:
dict
defaultdict(default_factory[, …]) –> dict with default factory
The default factory is called without arguments to produce a new value when a key is not present, in __getitem__ only. A defaultdict compares equal to a dict with the same items. All remaining arguments are treated the same as if they were passed to the dict constructor, including keyword arguments.
Methods:
clear
()copy
()Create a new dictionary with keys from iterable and values set to value.
Return the value for key if key is in the dictionary, else default.
items
()keys
()pop
(k[,d])If key is not found, d is returned if given, otherwise KeyError is raised
Remove and return a (key, value) pair as a 2-tuple.
Insert key with a value of default if key is not in the dictionary.
update
([E, ]**F)If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values
()Attributes:
Factory for default value called by __missing__().
-
clear
() → None. Remove all items from D.¶
-
copy
() → a shallow copy of D.¶
-
default_factory
¶ Factory for default value called by __missing__().
-
fromkeys
()¶ Create a new dictionary with keys from iterable and values set to value.
-
get
()¶ Return the value for key if key is in the dictionary, else default.
-
items
() → a set-like object providing a view on D's items¶
-
keys
() → a set-like object providing a view on D's keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised
-
popitem
()¶ Remove and return a (key, value) pair as a 2-tuple.
Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.
-
setdefault
()¶ Insert key with a value of default if key is not in the dictionary.
Return the value for key if key is in the dictionary, else default.
-
update
([E, ]**F) → None. Update D from dict/iterable E and F.¶ If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
-
values
() → an object providing a view on D's values¶
-
-
fiftyone.zoo.
deepcopy
(x, memo=None, _nil=[])¶ Deep copy operation on arbitrary Python objects.
See the module’s __doc__ string for more info.
-
exception
fiftyone.zoo.
ConfigError
¶ Bases:
Exception
Exception raised when an invalid Config instance is encountered.
-
args
¶
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
fiftyone.zoo.
list_zoo_models
(tags=None)¶ Returns the list of available models in the FiftyOne Model Zoo.
Example usage:
import fiftyone as fo import fiftyone.zoo as foz # # List all zoo models # names = foz.list_zoo_models() print(names) # # List all zoo models with the specified tag(s) # names = foz.list_zoo_models(tags="torch") print(names)
- Parameters
tags (None) – only include models that have the specified tag or list of tags
- Returns
a list of model names
-
fiftyone.zoo.
list_downloaded_zoo_models
()¶ Returns information about the zoo models that have been downloaded.
- Returns
a dict mapping model names to (model path,
ZooModel
) tuples
-
fiftyone.zoo.
is_zoo_model_downloaded
(name)¶ Determines whether the zoo model of the given name is downloaded.
- Parameters
name – the name of the zoo model, which can have
@<ver>
appended to refer to a specific version of the model. If no version is specified, the latest version of the model is used- Returns
True/False
-
fiftyone.zoo.
download_zoo_model
(name, overwrite=False)¶ Downloads the model of the given name from the FiftyOne Dataset Zoo.
If the model is already downloaded, it is not re-downloaded unless
overwrite == True
is specified.- Parameters
name – the name of the zoo model, which can have
@<ver>
appended to refer to a specific version of the model. If no version is specified, the latest version of the model is used. Calllist_zoo_models()
to see the available modelsoverwrite (False) – whether to overwrite any existing files
- Returns
tuple of
model: the
ZooModel
for the modelmodel_path: the path to the downloaded model on disk
-
fiftyone.zoo.
install_zoo_model_requirements
(name, error_level=None)¶ Installs any package requirements for the zoo model with the given name.
- Parameters
name – the name of the zoo model, which can have
@<ver>
appended to refer to a specific version of the model. If no version is specified, the latest version of the model is used. Calllist_zoo_models()
to see the available modelserror_level (None) –
the error level to use, defined as:
0: raise error if a requirement install fails
1: log warning if a requirement install fails
2: ignore install fails requirements
By default,
fo.config.requirement_error_level
is used
-
fiftyone.zoo.
ensure_zoo_model_requirements
(name, error_level=None, log_success=True)¶ Ensures that the package requirements for the zoo model with the given name are satisfied.
- Parameters
name – the name of the zoo model, which can have
@<ver>
appended to refer to a specific version of the model. If no version is specified, the latest version of the model is used. Calllist_zoo_models()
to see the available modelserror_level (None) –
the error level to use when installing/ensuring requirements, defined as:
0: raise error if a requirement is not satisfied
1: log warning if a requirement is not satisifed
2: ignore unsatisifed requirements
By default,
fo.config.requirement_error_level
is usedlog_success (True) – whether to generate a log message when a requirement is satisifed
-
fiftyone.zoo.
load_zoo_model
(name, download_if_necessary=True, ensure_requirements=True, install_requirements=False, error_level=None, cache=True, **kwargs)¶ Loads the model of the given name from the FiftyOne Model Zoo.
By default, the model will be downloaded if necessary, and any documented package requirements will be checked to ensure that they are installed.
- Parameters
name – the name of the zoo model, which can have
@<ver>
appended to refer to a specific version of the model. If no version is specified, the latest version of the model is downloaded. Calllist_zoo_models()
to see the available modelsdownload_if_necessary (True) – whether to download the model if it is not found in the specified directory
ensure_requirements (True) – whether to ensure any requirements are installed before loading the model. By default, this is True
install_requirements – whether to install any requirements before loading the model. By default, this is False
error_level (None) –
the error level to use when installing/ensuring requirements, defined as:
0: raise error if a requirement is not satisfied
1: log warning if a requirement is not satisifed
2: ignore unsatisifed requirements
By default,
fo.config.requirement_error_level
is usedcache (True) – whether to store a weak reference to the model so that running this method again will return the same instance while the model is still in use. If False, no weak reference is stored/used
**kwargs – keyword arguments to inject into the model’s
Config
instance
- Returns
-
fiftyone.zoo.
find_zoo_model
(name)¶ Returns the path to the zoo model on disk.
The model must be downloaded. Use
download_zoo_model()
to download models.- Parameters
name – the name of the zoo model, which can have
@<ver>
appended to refer to a specific version of the model. If no version is specified, the latest version of the model is used- Returns
the path to the model on disk
- Raises
ValueError – if the model does not exist or has not been downloaded
-
fiftyone.zoo.
get_zoo_model
(name)¶ Returns the
ZooModel
instance for the model with the given name.- Parameters
name – the name of the zoo model
- Returns
a
ZooModel
-
fiftyone.zoo.
delete_zoo_model
(name)¶ Deletes the zoo model from local disk, if necessary.
- Parameters
name – the name of the zoo model, which can have
@<ver>
appended to refer to a specific version of the model. If no version is specified, the latest version of the model is used
-
class
fiftyone.zoo.
HasZooModel
¶ Bases:
eta.core.learning.HasPublishedModel
Mixin class for Config classes of
fiftyone.core.models.Model
instances whose models are stored in the FiftyOne Model Zoo.This class provides the following functionality:
The model to load can be specified either by:
providing a model_name, which specifies the zoo model to load. The model will be downloaded, if necessary
providing a model_path, which directly specifies the path to the model to load
fiftyone.core.models.ModelConfig
definitions that use zoo models with default deployments will have default values for any unspecified parameters loaded and applied at runtime
- Parameters
model_name – the name of the zoo model to load. If this value is provided, model_path does not need to be
model_path – the path to an already downloaded zoo model on disk to load. If this value is provided, model_name does not need to be
Methods:
Downloads the published model specified by the config, if necessary.
init
(d)Initializes the published model config.
-
download_model_if_necessary
()¶ Downloads the published model specified by the config, if necessary.
After this method is called, the model_path attribute will always contain the path to the model on disk.
-
init
(d)¶ Initializes the published model config.
This method should be called by ModelConfig.__init__(), and it performs the following tasks:
Parses the model_name and model_path parameters
Populates any default parameters in the provided ModelConfig dict
- Parameters
d – a ModelConfig dict
- Returns
a ModelConfig dict with any default parameters populated
-
class
fiftyone.zoo.
ZooModel
(base_name, base_filename=None, manager=None, version=None, description=None, source=None, size_bytes=None, default_deployment_config_dict=None, requirements=None, tags=None, date_added=None)¶ Bases:
eta.core.models.Model
Class describing a model in the FiftyOne Model Zoo.
- Parameters
base_name – the base name of the model (no version info)
base_filename (None) – the base filename of the model (no version info), if applicable
manager (None) – the
fiftyone.core.models.ModelManager
instance that describes the remote storage location of the model, if applicableversion (None) – the version of the model
description (None) – the description of the model
source (None) – the source of the model
size_bytes (None) – the size of the model on disk
default_deployment_config_dict (None) – a
fiftyone.core.models.ModelConfig
dict describing the recommended settings for deploying the modelrequirements (None) – the
eta.core.models.ModelRequirements
for the modeltags (None) – a list of tags for the model
date_added (None) – the datetime that the model was added to the zoo
Methods:
Returns a list of class attributes to be serialized.
copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
download_model
(model_path[, force])Downloads the model to the given local path.
ensure_requirements
([error_level, log_success])Ensures that any requirement(s) for this model are satisfied.
flush_model
(model_path)Flushes the copy of the model at the given local path, if necessary.
flush_model_from_dir
(models_dir)Flushes the copy of the model in the given models directory, if necessary.
from_dict
(d)Constructs a Model from a JSON dictionary.
from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
get_path_in_dir
(models_dir)Gets the model path for the model in the given models directory.
has_tag
(tag)Whether this model has the given tag.
has_version_str
(name)Determines whether the given model name has a version string.
install_requirements
([error_level])Installs any necessary requirement(s) for this model.
is_in_dir
(models_dir)Determines whether a copy of the model exists in the given models directory.
is_model_downloaded
(model_path)Determines whether the model is downloaded to the given location.
parse_name
(name)Parses the model name, returning the base name and the version, if any.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
Attributes:
The version of this model expressed as a distutils.version.LooseVersion intended for comparison operations.
The version-aware filename of the model.
Whether this model has a manager instance.
Whether this model has requirements in order to be used.
Whether this model has tags.
Determines whether the model has a version.
The version-aware name of the model.
Whether the model supports CPU (True), or not (False), or unknown (None).
Whether the model supports GPU (True), or not (False), or unknown (None).
-
attributes
()¶ Returns a list of class attributes to be serialized.
- Returns
a list of class attributes
-
property
comp_version
¶ The version of this model expressed as a distutils.version.LooseVersion intended for comparison operations.
Models with no version are given a version of 0.0.0.
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
download_model
(model_path, force=False)¶ Downloads the model to the given local path.
If the download is forced, any existing model is overwritten. If the download is not forced, the model will only be downloaded if it does not already exist locally.
If the model has no manager, nothing is downloaded.
- Parameters
model_path – the path to which to download the model
force – whether to force download the model. If True, the model is always downloaded. If False, the model is only downloaded if necessary. The default is False
- Raises
ModelError – if model downloading is not currently allowed
-
ensure_requirements
(error_level=0, log_success=False)¶ Ensures that any requirement(s) for this model are satisfied.
- Parameters
error_level –
the error level to use, defined as:
0: raise error if a requirement is not satisfied 1: log warning if a requirement is not satisifed 2: ignore unsatisifed requirements
log_success – whether to generate a log message when a requirement is satisifed
-
property
filename
¶ The version-aware filename of the model.
-
flush_model
(model_path)¶ Flushes the copy of the model at the given local path, if necessary.
- Parameters
model_path – the path on disk for the model
-
flush_model_from_dir
(models_dir)¶ Flushes the copy of the model in the given models directory, if necessary.
- Parameters
models_dir – the models directory
-
classmethod
from_dict
(d)¶ Constructs a Model from a JSON dictionary.
- Parameters
d – a JSON dictionary
- Returns
a Model instance
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
get_path_in_dir
(models_dir)¶ Gets the model path for the model in the given models directory.
- Parameters
models_dir – the models directory
- Returns
the model path, or None if the model has no manager
-
property
has_manager
¶ Whether this model has a manager instance.
-
property
has_requirements
¶ Whether this model has requirements in order to be used.
-
has_tag
(tag)¶ Whether this model has the given tag.
- Parameters
tag – a tag
- Returns
True/False
Whether this model has tags.
-
property
has_version
¶ Determines whether the model has a version.
-
static
has_version_str
(name)¶ Determines whether the given model name has a version string.
- Parameters
name – the model name
- Returns
True/False
-
install_requirements
(error_level=0)¶ Installs any necessary requirement(s) for this model.
- Parameters
error_level –
the error level to use, defined as:
0: raise error if an install fails 1: log warning if an install fails 2: ignore install fails
-
is_in_dir
(models_dir)¶ Determines whether a copy of the model exists in the given models directory.
- Parameters
models_dir – the models directory
- Returns
True/False, or None if the model has no manager
-
is_model_downloaded
(model_path)¶ Determines whether the model is downloaded to the given location.
If model_path is an archive, this method will also return True if a directory with the same basename as model_path exists.
- Parameters
model_path – the path on disk for the model
- Returns
True/False, or None if the model has no manager
-
property
name
¶ The version-aware name of the model.
-
static
parse_name
(name)¶ Parses the model name, returning the base name and the version, if any.
- Parameters
name – the name of the model, which can have “@<ver>” appended to refer to a specific version of the model
- Returns
the base name of the model version: the version of the model, or None if no version was found
- Return type
base_name
- Raises
ModelError – if the model name was invalid
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
property
supports_cpu
¶ Whether the model supports CPU (True), or not (False), or unknown (None).
-
property
supports_gpu
¶ Whether the model supports GPU (True), or not (False), or unknown (None).
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
class
fiftyone.zoo.
ZooModelsManifest
(models=None)¶ Bases:
eta.core.models.ModelsManifest
Class that describes the collection of models in the FiftyOne Model Zoo.
- Parameters
models – a list of
ZooModel
instances
Methods:
add_model
(model)Adds the given model to the manifest.
Returns a list of class attributes to be serialized.
copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
dir_has_manifest
(models_dir)Determines whether the given directory has a models manifest.
from_dict
(d)Constructs a ModelsManifest from a JSON dictionary.
from_dir
(models_dir)Loads the ModelsManifest from the given models directory.
from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
get_latest_model_with_base_name
(base_name)Gets the Model instance for the latest version of the model with the given base name.
get_model_with_name
(name)Gets the model with the given name.
has_model_with_filename
(filename)Determines whether this manifest contains a model with the given filename.
has_model_with_name
(name)Determines whether this manifest contains the model with the given name.
make_manifest_path
(models_dir)Makes the manifest path for the given models directory.
merge
(models_manifest)Merges the models manifest into this one.
remove_model
(name)Removes the model with the given name from the ModelsManifest.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
write_to_dir
(models_dir)Writes the ModelsManifest to the given models directory.
-
add_model
(model)¶ Adds the given model to the manifest.
- Parameters
model – a Model instance
- Raises
ModelError – if the model conflicts with an existing model in the manifest
-
attributes
()¶ Returns a list of class attributes to be serialized.
This method is called internally by serialize() to determine the class attributes to serialize.
Subclasses can override this method, but, by default, all attributes in vars(self) are returned, minus private attributes, i.e., those starting with “_”. The order of the attributes in this list is preserved when serializing objects, so a common pattern is for subclasses to override this method if they want their JSON files to be organized in a particular way.
- Returns
a list of class attributes to be serialized
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
static
dir_has_manifest
(models_dir)¶ Determines whether the given directory has a models manifest.
- Parameters
models_dir – the models directory
- Returns
True/False
-
classmethod
from_dict
(d)¶ Constructs a ModelsManifest from a JSON dictionary.
- Parameters
d – a JSON dictionary
- Returns
a ModelsManifest
-
classmethod
from_dir
(models_dir)¶ Loads the ModelsManifest from the given models directory.
- Parameters
models_dir – the models directory
- Returns
a ModelsManifest
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
get_latest_model_with_base_name
(base_name)¶ Gets the Model instance for the latest version of the model with the given base name.
- Parameters
base_name – the base name of the model
- Returns
the Model instance
- Raises
ModelError – if the model was not found
-
get_model_with_name
(name)¶ Gets the model with the given name.
- Parameters
name – the name of the model
- Returns
the Model instance
- Raises
ModelError – if the model was not found
-
has_model_with_filename
(filename)¶ Determines whether this manifest contains a model with the given filename.
- Parameters
filename – the filename
- Returns
True/False
-
has_model_with_name
(name)¶ Determines whether this manifest contains the model with the given name.
- Parameters
name – the model name
- Returns
True/False
-
static
make_manifest_path
(models_dir)¶ Makes the manifest path for the given models directory.
- Parameters
models_dir – the models directory
- Returns
the manifest path
-
merge
(models_manifest)¶ Merges the models manifest into this one.
- Parameters
models_manifest – a ModelsManifest
-
remove_model
(name)¶ Removes the model with the given name from the ModelsManifest.
- Parameters
name – the name of the model
- Raises
ModelError – if the model was not found
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
write_to_dir
(models_dir)¶ Writes the ModelsManifest to the given models directory.
- Parameters
models_dir – the models directory