fiftyone.factory.repos.execution_store¶
Execution store repository interface and implementations.
Classes:
|
Abstract base class for execution store repositories. |
|
MongoDB implementation of the execution store repository. |
|
In-memory implementation of execution store repository. |
-
class
fiftyone.factory.repos.execution_store.ExecutionStoreRepo(dataset_id: Optional[bson.objectid.ObjectId] = None, is_cache=False)¶ Bases:
abc.ABCAbstract base class for execution store repositories.
Each instance operates in a context: - If a dataset_id is provided, it operates on stores associated with that dataset. - If no dataset_id is provided, it operates on stores not associated with any dataset.
To operate on all stores across all contexts, use the
XXX_global()methods that this class provides.Methods:
create_store(store_name[, metadata, policy])Create a store in the store collection.
clear_cache([store_name])Clear all keys with either a
ttlorpolicy="eviction".get_store(store_name)Get a store from the store collection.
has_store(store_name)Check if a store exists in the store collection.
List all stores in the store collection.
Count the number of stores in the store collection.
delete_store(store_name)Delete a store.
set_key(store_name, key, value[, ttl, policy])Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key will expire and be automatically removed. :type ttl: Optional[int] :param policy: The eviction policy for the key. One of: -
"persist"(default): Key is persistent until deleted. -"evict": Key is eligible for eviction or cache clearing. :type policy: str.set_cache_key(store_name, key, value[, ttl])Set a cache key in a store.
has_key(store_name, key)Check if a key exists in a store.
get_key(store_name, key)Get a key from a store.
update_ttl(store_name, key, ttl)Update the TTL of a key.
delete_key(store_name, key)Delete a key from a store.
list_keys(store_name)List all keys in a store.
count_keys(store_name)Count the number of keys in a store.
cleanup()Delete all stores in the global store collection.
has_store_global(store_name)Check if a store exists in the global store collection.
List all stores in the global store collection.
Count the number of stores in the global store collection.
delete_store_global(store_name)Delete a store from the global store collection.
-
abstract
create_store(store_name: str, metadata: Optional[Dict[str, Any]] = None, policy: str = 'persist') → fiftyone.operators.store.models.StoreDocument¶ Create a store in the store collection.
- Parameters
store_name (str) – the name of the store to create
metadata (Optional[Dict[str, Any]]) – the metadata to store with the store
- Returns
the created store document
- Return type
-
abstract
clear_cache(store_name=None) → None¶ Clear all keys with either a
ttlorpolicy="eviction".- Parameters
store_name (str, optional) – the name of the store to clear. If None, all stores will be queried for deletion.
-
abstract
get_store(store_name: str) → Optional[fiftyone.operators.store.models.StoreDocument]¶ Get a store from the store collection.
- Parameters
store_name (str) – the name of the store to get
- Returns
the store document, or None if the store does not exist
- Return type
Optional[StoreDocument]
-
abstract
has_store(store_name: str) → bool¶ Check if a store exists in the store collection.
- Parameters
store_name (str) – the name of the store to check
- Returns
True if the store exists, False otherwise
- Return type
bool
-
abstract
list_stores() → List[str]¶ List all stores in the store collection.
- Returns
a list of store names
- Return type
List[str]
-
abstract
count_stores() → int¶ Count the number of stores in the store collection.
- Returns
the number of stores
- Return type
int
-
abstract
delete_store(store_name: str) → int¶ Delete a store.
- Parameters
store_name (str) – the name of the store to delete
- Returns
the number of documents deleted
- Return type
int
-
abstract
set_key(store_name: str, key: str, value: Any, ttl: Optional[int] = None, policy: str = 'persist') → fiftyone.operators.store.models.KeyDocument¶ Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key
will expire and be automatically removed.
- Parameters
policy (str) – The eviction policy for the key. One of: -
"persist"(default): Key is persistent until deleted. -"evict": Key is eligible for eviction or cache clearing.- Returns
The created or updated key document.
- Return type
-
abstract
set_cache_key(store_name: str, key: str, value: Any, ttl: Optional[int] = None) → fiftyone.operators.store.models.KeyDocument¶ Set a cache key in a store.
- Parameters
store_name (str) – the name of the store to set the cache key in
key (str) – the cache key to set
value (Any) – the value to set
ttl (Optional[int]) – the TTL of the cache key
- Returns
the created or updated cache key document
- Return type
-
abstract
has_key(store_name: str, key: str) → bool¶ Check if a key exists in a store.
- Parameters
store_name (str) – the name of the store to check
key (str) – the key to check
- Returns
True if the key exists, False otherwise
- Return type
bool
-
abstract
get_key(store_name: str, key: str) → Optional[fiftyone.operators.store.models.KeyDocument]¶ Get a key from a store.
- Parameters
store_name (str) – the name of the store to get the key from
key (str) – the key to get
- Returns
the key document, or None if the key does not exist
- Return type
Optional[KeyDocument]
-
abstract
update_ttl(store_name: str, key: str, ttl: int) → bool¶ Update the TTL of a key.
- Parameters
store_name (str) – the name of the store to update the TTL for
key (str) – the key to update the TTL for
ttl (int) – the new TTL
- Returns
True if the TTL was updated, False otherwise
- Return type
bool
-
abstract
delete_key(store_name: str, key: str) → bool¶ Delete a key from a store.
- Parameters
store_name (str) – the name of the store to delete the key from
key (str) – the key to delete
- Returns
True if the key was deleted, False otherwise
- Return type
bool
-
abstract
list_keys(store_name: str) → List[str]¶ List all keys in a store.
- Parameters
store_name (str) – the name of the store to list keys for
- Returns
a list of keys in the store
- Return type
List[str]
-
abstract
count_keys(store_name: str) → int¶ Count the number of keys in a store.
- Parameters
store_name (str) – the name of the store to count keys for
- Returns
the number of keys in the store
- Return type
int
-
abstract
cleanup() → int¶ Delete all stores in the global store collection.
- Returns
the number of documents deleted
- Return type
int
-
abstract
has_store_global(store_name: str) → bool¶ Check if a store exists in the global store collection.
- Parameters
store_name (str) – the name of the store to check
- Returns
True if the store exists, False otherwise
- Return type
bool
-
abstract
list_stores_global() → List[fiftyone.operators.store.models.StoreDocument]¶ List all stores in the global store collection.
- Returns
a list of store documents
- Return type
-
abstract
count_stores_global() → int¶ Count the number of stores in the global store collection.
- Returns
the number of stores
- Return type
int
-
abstract
delete_store_global(store_name: str) → int¶ Delete a store from the global store collection.
- Parameters
store_name (str) – the name of the store to delete
- Returns
the number of documents deleted
- Return type
int
-
abstract
-
class
fiftyone.factory.repos.execution_store.MongoExecutionStoreRepo(collection, dataset_id: Optional[bson.objectid.ObjectId] = None, is_cache=False)¶ Bases:
fiftyone.factory.repos.execution_store.ExecutionStoreRepoMongoDB implementation of the execution store repository.
Attributes:
Methods:
create_store(store_name[, metadata, policy])Create a store in the store collection.
clear_cache([store_name])Clear all keys with either a
ttlorpolicy="eviction".get_store(store_name)Get a store from the store collection.
has_store(store_name)Check if a store exists in the store collection.
List all stores in the store collection.
Count the number of stores in the store collection.
delete_store(store_name)Delete a store.
set_key(store_name, key, value[, ttl, policy])Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key will expire and be automatically removed. :type ttl: Optional[int] :param policy: The eviction policy for the key. One of: -
"persist"(default): Key is persistent until deleted. -"evict": Key is eligible for eviction or cache clearing. :type policy: str.set_cache_key(store_name, key, value[, ttl])Set a cache key in a store.
has_key(store_name, key)Check if a key exists in a store.
get_key(store_name, key)Get a key from a store.
update_ttl(store_name, key, ttl)Update the TTL of a key.
delete_key(store_name, key)Delete a key from a store.
list_keys(store_name)List all keys in a store.
count_keys(store_name)Count the number of keys in a store.
cleanup()Delete all stores in the global store collection.
has_store_global(store_name)Check if a store exists in the global store collection.
List all stores in the global store collection.
Count the number of stores in the global store collection.
delete_store_global(store_name)Delete a store from the global store collection.
-
COLLECTION_NAME= 'execution_store'¶
-
create_store(store_name: str, metadata: Optional[Dict[str, Any]] = None, policy: str = 'persist') → fiftyone.operators.store.models.StoreDocument¶ Create a store in the store collection.
- Parameters
store_name (str) – the name of the store to create
metadata (Optional[Dict[str, Any]]) – the metadata to store with the store
- Returns
the created store document
- Return type
-
clear_cache(store_name=None) → int¶ Clear all keys with either a
ttlorpolicy="eviction".- Parameters
store_name (str, optional) – the name of the store to clear. If None, all stores will be queried for deletion.
-
get_store(store_name: str) → Optional[fiftyone.operators.store.models.StoreDocument]¶ Get a store from the store collection.
- Parameters
store_name (str) – the name of the store to get
- Returns
the store document, or None if the store does not exist
- Return type
Optional[StoreDocument]
-
has_store(store_name: str) → bool¶ Check if a store exists in the store collection.
- Parameters
store_name (str) – the name of the store to check
- Returns
True if the store exists, False otherwise
- Return type
bool
-
list_stores() → List[str]¶ List all stores in the store collection.
- Returns
a list of store names
- Return type
List[str]
-
count_stores() → int¶ Count the number of stores in the store collection.
- Returns
the number of stores
- Return type
int
-
delete_store(store_name: str) → int¶ Delete a store.
- Parameters
store_name (str) – the name of the store to delete
- Returns
the number of documents deleted
- Return type
int
-
set_key(store_name: str, key: str, value: Any, ttl: Optional[int] = None, policy: str = 'persist') → fiftyone.operators.store.models.KeyDocument¶ Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key
will expire and be automatically removed.
- Parameters
policy (str) – The eviction policy for the key. One of: -
"persist"(default): Key is persistent until deleted. -"evict": Key is eligible for eviction or cache clearing.- Returns
The created or updated key document.
- Return type
-
set_cache_key(store_name, key, value, ttl=None)¶ Set a cache key in a store.
- Parameters
store_name (str) – the name of the store to set the cache key in
key (str) – the cache key to set
value (Any) – the value to set
ttl (Optional[int]) – the TTL of the cache key
- Returns
the created or updated cache key document
- Return type
-
has_key(store_name: str, key: str) → bool¶ Check if a key exists in a store.
- Parameters
store_name (str) – the name of the store to check
key (str) – the key to check
- Returns
True if the key exists, False otherwise
- Return type
bool
-
get_key(store_name: str, key: str) → Optional[fiftyone.operators.store.models.KeyDocument]¶ Get a key from a store.
- Parameters
store_name (str) – the name of the store to get the key from
key (str) – the key to get
- Returns
the key document, or None if the key does not exist
- Return type
Optional[KeyDocument]
-
update_ttl(store_name: str, key: str, ttl: int) → bool¶ Update the TTL of a key.
- Parameters
store_name (str) – the name of the store to update the TTL for
key (str) – the key to update the TTL for
ttl (int) – the new TTL
- Returns
True if the TTL was updated, False otherwise
- Return type
bool
-
delete_key(store_name: str, key: str) → bool¶ Delete a key from a store.
- Parameters
store_name (str) – the name of the store to delete the key from
key (str) – the key to delete
- Returns
True if the key was deleted, False otherwise
- Return type
bool
-
list_keys(store_name: str) → List[str]¶ List all keys in a store.
- Parameters
store_name (str) – the name of the store to list keys for
- Returns
a list of keys in the store
- Return type
List[str]
-
count_keys(store_name: str) → int¶ Count the number of keys in a store.
- Parameters
store_name (str) – the name of the store to count keys for
- Returns
the number of keys in the store
- Return type
int
-
cleanup() → int¶ Delete all stores in the global store collection.
- Returns
the number of documents deleted
- Return type
int
-
has_store_global(store_name: str) → bool¶ Check if a store exists in the global store collection.
- Parameters
store_name (str) – the name of the store to check
- Returns
True if the store exists, False otherwise
- Return type
bool
-
list_stores_global() → List[fiftyone.operators.store.models.StoreDocument]¶ List all stores in the global store collection.
- Returns
a list of store documents
- Return type
-
count_stores_global() → int¶ Count the number of stores in the global store collection.
- Returns
the number of stores
- Return type
int
-
delete_store_global(store_name: str) → int¶ Delete a store from the global store collection.
- Parameters
store_name (str) – the name of the store to delete
- Returns
the number of documents deleted
- Return type
int
-
-
class
fiftyone.factory.repos.execution_store.InMemoryExecutionStoreRepo(dataset_id: Optional[bson.objectid.ObjectId] = None)¶ Bases:
fiftyone.factory.repos.execution_store.ExecutionStoreRepoIn-memory implementation of execution store repository.
Methods:
create_store(store_name[, metadata, policy])Create a store in the store collection.
clear_cache([store_name])Clear all keys with either a
ttlorpolicy="eviction".get_store(store_name)Get a store from the store collection.
has_store(store_name)Check if a store exists in the store collection.
List all stores in the store collection.
Count the number of stores in the store collection.
delete_store(store_name)Delete a store.
set_key(store_name, key, value[, ttl, policy])Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key will expire and be automatically removed. :type ttl: Optional[int] :param policy: The eviction policy for the key. One of: -
"persist"(default): Key is persistent until deleted. -"evict": Key is eligible for eviction or cache clearing. :type policy: str.set_cache_key(store_name, key, value[, ttl])Set a cache key in a store.
has_key(store_name, key)Check if a key exists in a store.
get_key(store_name, key)Get a key from a store.
update_ttl(store_name, key, ttl)Update the TTL of a key.
update_policy(store_name, key, policy)delete_key(store_name, key)Delete a key from a store.
list_keys(store_name)List all keys in a store.
count_keys(store_name)Count the number of keys in a store.
cleanup()Delete all stores in the global store collection.
has_store_global(store_name)Check if a store exists in the global store collection.
List all stores in the global store collection.
Count the number of stores in the global store collection.
delete_store_global(store_name)Delete a store from the global store collection.
-
create_store(store_name: str, metadata: Optional[Dict[str, Any]] = None, policy: str = 'persist') → fiftyone.operators.store.models.StoreDocument¶ Create a store in the store collection.
- Parameters
store_name (str) – the name of the store to create
metadata (Optional[Dict[str, Any]]) – the metadata to store with the store
- Returns
the created store document
- Return type
-
clear_cache(store_name=None) → int¶ Clear all keys with either a
ttlorpolicy="eviction".- Parameters
store_name (str, optional) – the name of the store to clear. If None, all stores will be queried for deletion.
-
get_store(store_name: str) → Optional[fiftyone.operators.store.models.StoreDocument]¶ Get a store from the store collection.
- Parameters
store_name (str) – the name of the store to get
- Returns
the store document, or None if the store does not exist
- Return type
Optional[StoreDocument]
-
has_store(store_name: str) → bool¶ Check if a store exists in the store collection.
- Parameters
store_name (str) – the name of the store to check
- Returns
True if the store exists, False otherwise
- Return type
bool
-
list_stores() → List[str]¶ List all stores in the store collection.
- Returns
a list of store names
- Return type
List[str]
-
count_stores() → int¶ Count the number of stores in the store collection.
- Returns
the number of stores
- Return type
int
-
delete_store(store_name: str) → int¶ Delete a store.
- Parameters
store_name (str) – the name of the store to delete
- Returns
the number of documents deleted
- Return type
int
-
set_key(store_name: str, key: str, value: Any, ttl: Optional[int] = None, policy: str = 'persist') → fiftyone.operators.store.models.KeyDocument¶ Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key
will expire and be automatically removed.
- Parameters
policy (str) – The eviction policy for the key. One of: -
"persist"(default): Key is persistent until deleted. -"evict": Key is eligible for eviction or cache clearing.- Returns
The created or updated key document.
- Return type
-
set_cache_key(store_name: str, key: str, value: Any, ttl: Optional[int] = None) → None¶ Set a cache key in a store.
- Parameters
store_name (str) – the name of the store to set the cache key in
key (str) – the cache key to set
value (Any) – the value to set
ttl (Optional[int]) – the TTL of the cache key
- Returns
the created or updated cache key document
- Return type
-
has_key(store_name: str, key: str) → bool¶ Check if a key exists in a store.
- Parameters
store_name (str) – the name of the store to check
key (str) – the key to check
- Returns
True if the key exists, False otherwise
- Return type
bool
-
get_key(store_name: str, key: str) → Optional[fiftyone.operators.store.models.KeyDocument]¶ Get a key from a store.
- Parameters
store_name (str) – the name of the store to get the key from
key (str) – the key to get
- Returns
the key document, or None if the key does not exist
- Return type
Optional[KeyDocument]
-
update_ttl(store_name: str, key: str, ttl: int) → bool¶ Update the TTL of a key.
- Parameters
store_name (str) – the name of the store to update the TTL for
key (str) – the key to update the TTL for
ttl (int) – the new TTL
- Returns
True if the TTL was updated, False otherwise
- Return type
bool
-
update_policy(store_name: str, key: str, policy: str) → bool¶
-
delete_key(store_name: str, key: str) → bool¶ Delete a key from a store.
- Parameters
store_name (str) – the name of the store to delete the key from
key (str) – the key to delete
- Returns
True if the key was deleted, False otherwise
- Return type
bool
-
list_keys(store_name: str) → List[str]¶ List all keys in a store.
- Parameters
store_name (str) – the name of the store to list keys for
- Returns
a list of keys in the store
- Return type
List[str]
-
count_keys(store_name: str) → int¶ Count the number of keys in a store.
- Parameters
store_name (str) – the name of the store to count keys for
- Returns
the number of keys in the store
- Return type
int
-
cleanup() → int¶ Delete all stores in the global store collection.
- Returns
the number of documents deleted
- Return type
int
-
has_store_global(store_name: str) → bool¶ Check if a store exists in the global store collection.
- Parameters
store_name (str) – the name of the store to check
- Returns
True if the store exists, False otherwise
- Return type
bool
-
list_stores_global() → List[fiftyone.operators.store.models.StoreDocument]¶ List all stores in the global store collection.
- Returns
a list of store documents
- Return type
-
count_stores_global() → int¶ Count the number of stores in the global store collection.
- Returns
the number of stores
- Return type
int
-
delete_store_global(store_name: str) → int¶ Delete a store from the global store collection.
- Parameters
store_name (str) – the name of the store to delete
- Returns
the number of documents deleted
- Return type
int
-