fiftyone.factory.repos.execution_store

Execution store repository interface and implementations.

Copyright 2017-2025, Voxel51, Inc.

voxel51.com

Classes:

ExecutionStoreRepo([dataset_id, is_cache])	Abstract base class for execution store repositories.
MongoExecutionStoreRepo(collection[, ...])	MongoDB implementation of the execution store repository.
InMemoryExecutionStoreRepo([dataset_id])	In-memory implementation of execution store repository.

class fiftyone.factory.repos.execution_store.ExecutionStoreRepo(dataset_id: ObjectId | None = None, is_cache=False)

Bases: ABC

Abstract base class for execution store repositories.

Each instance operates in a context: - If a dataset_id is provided, it operates on stores associated with that dataset. - If no dataset_id is provided, it operates on stores not associated with any dataset.

To operate on all stores across all contexts, use the XXX_global() methods that this class provides.

Methods:

create_store(store_name[, metadata, policy])	Create a store in the store collection.
clear_cache([store_name])	Clear all keys with either a ttl or policy="eviction".
get_store(store_name)	Get a store from the store collection.
has_store(store_name)	Check if a store exists in the store collection.
list_stores()	List all stores in the store collection.
count_stores()	Count the number of stores in the store collection.
delete_store(store_name)	Delete a store.
set_key(store_name, key, value[, ttl, policy])	Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key will expire and be automatically removed. :type ttl: Optional[int] :param policy: The eviction policy for the key. One of: - "persist" (default): Key is persistent until deleted. - "evict": Key is eligible for eviction or cache clearing. :type policy: str.
set_cache_key(store_name, key, value[, ttl])	Set a cache key in a store.
has_key(store_name, key)	Check if a key exists in a store.
get_key(store_name, key)	Get a key from a store.
update_ttl(store_name, key, ttl)	Update the TTL of a key.
delete_key(store_name, key)	Delete a key from a store.
list_keys(store_name)	List all keys in a store.
count_keys(store_name)	Count the number of keys in a store.
cleanup()	Delete all stores in the global store collection.
has_store_global(store_name)	Check if a store exists in the global store collection.
list_stores_global()	List all stores in the global store collection.
count_stores_global()	Count the number of stores in the global store collection.
delete_store_global(store_name)	Delete a store from the global store collection.

abstract create_store(store_name: str, metadata: Dict[str, Any] | None = None, policy: str = 'persist') → StoreDocument

Create a store in the store collection.

Parameters:

• store_name (str) – the name of the store to create

• metadata (Optional[Dict[str, Any]]) – the metadata to store with the store

Returns:

the created store document

Return type:

StoreDocument

abstract clear_cache(store_name=None) → None

Clear all keys with either a ttl or policy="eviction".

Parameters:

store_name (str, optional) – the name of the store to clear. If None, all stores will be queried for deletion.

abstract get_store(store_name: str) → StoreDocument | None

Get a store from the store collection.

Parameters:

store_name (str) – the name of the store to get

Returns:

the store document, or None if the store does not exist

Return type:

Optional[StoreDocument]

abstract has_store(store_name: str) → bool

Check if a store exists in the store collection.

Parameters:

store_name (str) – the name of the store to check

Returns:

True if the store exists, False otherwise

Return type:

bool

abstract list_stores() → List[str]

List all stores in the store collection.

Returns:

a list of store names

Return type:

List[str]

abstract count_stores() → int

Count the number of stores in the store collection.

Returns:

the number of stores

Return type:

int

abstract delete_store(store_name: str) → int

Delete a store.

Parameters:

store_name (str) – the name of the store to delete

Returns:

the number of documents deleted

Return type:

int

abstract set_key(store_name: str, key: str, value: Any, ttl: int | None = None, policy: str = 'persist') → KeyDocument

Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key

will expire and be automatically removed.

Parameters:

policy (str) – The eviction policy for the key. One of: - "persist" (default): Key is persistent until deleted. - "evict": Key is eligible for eviction or cache clearing.

Returns:

The created or updated key document.

Return type:

KeyDocument

abstract set_cache_key(store_name: str, key: str, value: Any, ttl: int | None = None) → KeyDocument

Set a cache key in a store.

Parameters:

• store_name (str) – the name of the store to set the cache key in

• key (str) – the cache key to set

• value (Any) – the value to set

• ttl (Optional[int]) – the TTL of the cache key

Returns:

the created or updated cache key document

Return type:

KeyDocument

abstract has_key(store_name: str, key: str) → bool

Check if a key exists in a store.

Parameters:

• store_name (str) – the name of the store to check

• key (str) – the key to check

Returns:

True if the key exists, False otherwise

Return type:

bool

abstract get_key(store_name: str, key: str) → KeyDocument | None

Get a key from a store.

Parameters:

• store_name (str) – the name of the store to get the key from

• key (str) – the key to get

Returns:

the key document, or None if the key does not exist

Return type:

Optional[KeyDocument]

abstract update_ttl(store_name: str, key: str, ttl: int) → bool

Update the TTL of a key.

Parameters:

• store_name (str) – the name of the store to update the TTL for

• key (str) – the key to update the TTL for

• ttl (int) – the new TTL

Returns:

True if the TTL was updated, False otherwise

Return type:

bool

abstract delete_key(store_name: str, key: str) → bool

Delete a key from a store.

Parameters:

• store_name (str) – the name of the store to delete the key from

• key (str) – the key to delete

Returns:

True if the key was deleted, False otherwise

Return type:

bool

abstract list_keys(store_name: str) → List[str]

List all keys in a store.

Parameters:

store_name (str) – the name of the store to list keys for

Returns:

a list of keys in the store

Return type:

List[str]

abstract count_keys(store_name: str) → int

Count the number of keys in a store.

Parameters:

store_name (str) – the name of the store to count keys for

Returns:

the number of keys in the store

Return type:

int

abstract cleanup() → int

Delete all stores in the global store collection.

Returns:

the number of documents deleted

Return type:

int

abstract has_store_global(store_name: str) → bool

Check if a store exists in the global store collection.

Parameters:

store_name (str) – the name of the store to check

Returns:

True if the store exists, False otherwise

Return type:

bool

abstract list_stores_global() → List[StoreDocument]

List all stores in the global store collection.

Returns:

a list of store documents

Return type:

List[StoreDocument]

abstract count_stores_global() → int

Count the number of stores in the global store collection.

Returns:

the number of stores

Return type:

int

abstract delete_store_global(store_name: str) → int

Delete a store from the global store collection.

Parameters:

store_name (str) – the name of the store to delete

Returns:

the number of documents deleted

Return type:

int

class fiftyone.factory.repos.execution_store.MongoExecutionStoreRepo(collection, dataset_id: ObjectId | None = None, is_cache=False)

Bases: ExecutionStoreRepo

MongoDB implementation of the execution store repository.

Attributes:

COLLECTION_NAME

Methods:

create_store(store_name[, metadata, policy])	Create a store in the store collection.
clear_cache([store_name])	Clear all keys with either a ttl or policy="eviction".
get_store(store_name)	Get a store from the store collection.
has_store(store_name)	Check if a store exists in the store collection.
list_stores()	List all stores in the store collection.
count_stores()	Count the number of stores in the store collection.
delete_store(store_name)	Delete a store.
set_key(store_name, key, value[, ttl, policy])	Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key will expire and be automatically removed. :type ttl: Optional[int] :param policy: The eviction policy for the key. One of: - "persist" (default): Key is persistent until deleted. - "evict": Key is eligible for eviction or cache clearing. :type policy: str.
set_cache_key(store_name, key, value[, ttl])	Set a cache key in a store.
has_key(store_name, key)	Check if a key exists in a store.
get_key(store_name, key)	Get a key from a store.
update_ttl(store_name, key, ttl)	Update the TTL of a key.
delete_key(store_name, key)	Delete a key from a store.
list_keys(store_name)	List all keys in a store.
count_keys(store_name)	Count the number of keys in a store.
cleanup()	Delete all stores in the global store collection.
has_store_global(store_name)	Check if a store exists in the global store collection.
list_stores_global()	List all stores in the global store collection.
count_stores_global()	Count the number of stores in the global store collection.
delete_store_global(store_name)	Delete a store from the global store collection.

COLLECTION_NAME = 'execution_store'

create_store(store_name: str, metadata: Dict[str, Any] | None = None, policy: str = 'persist') → StoreDocument

Create a store in the store collection.

Parameters:

• store_name (str) – the name of the store to create

• metadata (Optional[Dict[str, Any]]) – the metadata to store with the store

Returns:

the created store document

Return type:

StoreDocument

clear_cache(store_name=None) → int

Clear all keys with either a ttl or policy="eviction".

Parameters:

store_name (str, optional) – the name of the store to clear. If None, all stores will be queried for deletion.

get_store(store_name: str) → StoreDocument | None

Get a store from the store collection.

Parameters:

store_name (str) – the name of the store to get

Returns:

the store document, or None if the store does not exist

Return type:

Optional[StoreDocument]

has_store(store_name: str) → bool

Check if a store exists in the store collection.

Parameters:

store_name (str) – the name of the store to check

Returns:

True if the store exists, False otherwise

Return type:

bool

list_stores() → List[str]

List all stores in the store collection.

Returns:

a list of store names

Return type:

List[str]

count_stores() → int

Count the number of stores in the store collection.

Returns:

the number of stores

Return type:

int

delete_store(store_name: str) → int

Delete a store.

Parameters:

store_name (str) – the name of the store to delete

Returns:

the number of documents deleted

Return type:

int

set_key(store_name: str, key: str, value: Any, ttl: int | None = None, policy: str = 'persist') → KeyDocument

Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key

will expire and be automatically removed.

Parameters:

policy (str) – The eviction policy for the key. One of: - "persist" (default): Key is persistent until deleted. - "evict": Key is eligible for eviction or cache clearing.

Returns:

The created or updated key document.

Return type:

KeyDocument

set_cache_key(store_name, key, value, ttl=None)

Set a cache key in a store.

Parameters:

• store_name (str) – the name of the store to set the cache key in

• key (str) – the cache key to set

• value (Any) – the value to set

• ttl (Optional[int]) – the TTL of the cache key

Returns:

the created or updated cache key document

Return type:

KeyDocument

has_key(store_name: str, key: str) → bool

Check if a key exists in a store.

Parameters:

• store_name (str) – the name of the store to check

• key (str) – the key to check

Returns:

True if the key exists, False otherwise

Return type:

bool

get_key(store_name: str, key: str) → KeyDocument | None

Get a key from a store.

Parameters:

• store_name (str) – the name of the store to get the key from

• key (str) – the key to get

Returns:

the key document, or None if the key does not exist

Return type:

Optional[KeyDocument]

update_ttl(store_name: str, key: str, ttl: int) → bool

Update the TTL of a key.

Parameters:

• store_name (str) – the name of the store to update the TTL for

• key (str) – the key to update the TTL for

• ttl (int) – the new TTL

Returns:

True if the TTL was updated, False otherwise

Return type:

bool

delete_key(store_name: str, key: str) → bool

Delete a key from a store.

Parameters:

• store_name (str) – the name of the store to delete the key from

• key (str) – the key to delete

Returns:

True if the key was deleted, False otherwise

Return type:

bool

list_keys(store_name: str) → List[str]

List all keys in a store.

Parameters:

store_name (str) – the name of the store to list keys for

Returns:

a list of keys in the store

Return type:

List[str]

count_keys(store_name: str) → int

Count the number of keys in a store.

Parameters:

store_name (str) – the name of the store to count keys for

Returns:

the number of keys in the store

Return type:

int

cleanup() → int

Delete all stores in the global store collection.

Returns:

the number of documents deleted

Return type:

int

has_store_global(store_name: str) → bool

Check if a store exists in the global store collection.

Parameters:

store_name (str) – the name of the store to check

Returns:

True if the store exists, False otherwise

Return type:

bool

list_stores_global() → List[StoreDocument]

List all stores in the global store collection.

Returns:

a list of store documents

Return type:

List[StoreDocument]

count_stores_global() → int

Count the number of stores in the global store collection.

Returns:

the number of stores

Return type:

int

delete_store_global(store_name: str) → int

Delete a store from the global store collection.

Parameters:

store_name (str) – the name of the store to delete

Returns:

the number of documents deleted

Return type:

int

class fiftyone.factory.repos.execution_store.InMemoryExecutionStoreRepo(dataset_id: ObjectId | None = None)

Bases: ExecutionStoreRepo

In-memory implementation of execution store repository.

Methods:

create_store(store_name[, metadata, policy])	Create a store in the store collection.
clear_cache([store_name])	Clear all keys with either a ttl or policy="eviction".
get_store(store_name)	Get a store from the store collection.
has_store(store_name)	Check if a store exists in the store collection.
list_stores()	List all stores in the store collection.
count_stores()	Count the number of stores in the store collection.
delete_store(store_name)	Delete a store.
set_key(store_name, key, value[, ttl, policy])	Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key will expire and be automatically removed. :type ttl: Optional[int] :param policy: The eviction policy for the key. One of: - "persist" (default): Key is persistent until deleted. - "evict": Key is eligible for eviction or cache clearing. :type policy: str.
set_cache_key(store_name, key, value[, ttl])	Set a cache key in a store.
has_key(store_name, key)	Check if a key exists in a store.
get_key(store_name, key)	Get a key from a store.
update_ttl(store_name, key, ttl)	Update the TTL of a key.
update_policy(store_name, key, policy)
delete_key(store_name, key)	Delete a key from a store.
list_keys(store_name)	List all keys in a store.
count_keys(store_name)	Count the number of keys in a store.
cleanup()	Delete all stores in the global store collection.
has_store_global(store_name)	Check if a store exists in the global store collection.
list_stores_global()	List all stores in the global store collection.
count_stores_global()	Count the number of stores in the global store collection.
delete_store_global(store_name)	Delete a store from the global store collection.

create_store(store_name: str, metadata: Dict[str, Any] | None = None, policy: str = 'persist') → StoreDocument

Create a store in the store collection.

Parameters:

• store_name (str) – the name of the store to create

• metadata (Optional[Dict[str, Any]]) – the metadata to store with the store

Returns:

the created store document

Return type:

StoreDocument

clear_cache(store_name=None) → int

Clear all keys with either a ttl or policy="eviction".

Parameters:

store_name (str, optional) – the name of the store to clear. If None, all stores will be queried for deletion.

get_store(store_name: str) → StoreDocument | None

Get a store from the store collection.

Parameters:

store_name (str) – the name of the store to get

Returns:

the store document, or None if the store does not exist

Return type:

Optional[StoreDocument]

has_store(store_name: str) → bool

Check if a store exists in the store collection.

Parameters:

store_name (str) – the name of the store to check

Returns:

True if the store exists, False otherwise

Return type:

bool

list_stores() → List[str]

List all stores in the store collection.

Returns:

a list of store names

Return type:

List[str]

count_stores() → int

Count the number of stores in the store collection.

Returns:

the number of stores

Return type:

int

delete_store(store_name: str) → int

Delete a store.

Parameters:

store_name (str) – the name of the store to delete

Returns:

the number of documents deleted

Return type:

int

set_key(store_name: str, key: str, value: Any, ttl: int | None = None, policy: str = 'persist') → KeyDocument

Set a key in a store. :param store_name: The name of the store to set the key in. :type store_name: str :param key: The key to set. :type key: str :param value: The value to associate with the key. :type value: Any :param ttl: Optional TTL (in seconds) after which the key

will expire and be automatically removed.

Parameters:

policy (str) – The eviction policy for the key. One of: - "persist" (default): Key is persistent until deleted. - "evict": Key is eligible for eviction or cache clearing.

Returns:

The created or updated key document.

Return type:

KeyDocument

set_cache_key(store_name: str, key: str, value: Any, ttl: int | None = None) → None

Set a cache key in a store.

Parameters:

• store_name (str) – the name of the store to set the cache key in

• key (str) – the cache key to set

• value (Any) – the value to set

• ttl (Optional[int]) – the TTL of the cache key

Returns:

the created or updated cache key document

Return type:

KeyDocument

has_key(store_name: str, key: str) → bool

Check if a key exists in a store.

Parameters:

• store_name (str) – the name of the store to check

• key (str) – the key to check

Returns:

True if the key exists, False otherwise

Return type:

bool

get_key(store_name: str, key: str) → KeyDocument | None

Get a key from a store.

Parameters:

• store_name (str) – the name of the store to get the key from

• key (str) – the key to get

Returns:

the key document, or None if the key does not exist

Return type:

Optional[KeyDocument]

update_ttl(store_name: str, key: str, ttl: int) → bool

Update the TTL of a key.

Parameters:

• store_name (str) – the name of the store to update the TTL for

• key (str) – the key to update the TTL for

• ttl (int) – the new TTL

Returns:

True if the TTL was updated, False otherwise

Return type:

bool

update_policy(store_name: str, key: str, policy: str) → bool

delete_key(store_name: str, key: str) → bool

Delete a key from a store.

Parameters:

• store_name (str) – the name of the store to delete the key from

• key (str) – the key to delete

Returns:

True if the key was deleted, False otherwise

Return type:

bool

list_keys(store_name: str) → List[str]

List all keys in a store.

Parameters:

store_name (str) – the name of the store to list keys for

Returns:

a list of keys in the store

Return type:

List[str]

count_keys(store_name: str) → int

Count the number of keys in a store.

Parameters:

store_name (str) – the name of the store to count keys for

Returns:

the number of keys in the store

Return type:

int

cleanup() → int

Delete all stores in the global store collection.

Returns:

the number of documents deleted

Return type:

int

has_store_global(store_name: str) → bool

Check if a store exists in the global store collection.

Parameters:

store_name (str) – the name of the store to check

Returns:

True if the store exists, False otherwise

Return type:

bool

list_stores_global() → List[StoreDocument]

List all stores in the global store collection.

Returns:

a list of store documents

Return type:

List[StoreDocument]

count_stores_global() → int

Count the number of stores in the global store collection.

Returns:

the number of stores

Return type:

int

delete_store_global(store_name: str) → int

Delete a store from the global store collection.

Parameters:

store_name (str) – the name of the store to delete

Returns:

the number of documents deleted

Return type:

int