fiftyone.operators.store#

Module contents#

Execution store.

Copyright 2017-2025, Voxel51, Inc.
voxel51.com

Classes:

ExecutionStoreService([repo, dataset_id, ...])

Service for managing execution store operations.

ExecutionStore(store_name, store_service[, ...])

Execution store.

StoreDocument(store_name, key, value, ...)

Model representing a Store.

KeyDocument(store_name, key, value, _id, ...)

Model representing a key in the store.

KeyPolicy(value[, names, module, qualname, ...])

Defines the eviction policy for a key in the execution store.
class fiftyone.operators.store.ExecutionStoreService(repo: ExecutionStoreRepo | None = None, dataset_id: ObjectId | None = None, collection_name: str = None)#

Bases: object

Service for managing execution store operations.

Note that each instance of this service has a context:

  • If a dataset_id is provided (or a repo associated with one), this instance operates on stores associated with that dataset

  • If no dataset_id is provided (or a repo is provided that is not associated with one), this instance operates on stores that are not associated with a dataset

To operate on all stores across all contexts, use the XXX_global() methods that this class provides.

Parameters:

Methods:

create_store(store_name[, metadata, policy])

Creates a new store with the specified name.

clear_cache([store_name])

Clears all cache entries in the execution stores.

get_store(store_name)

Gets the specified store for the current context.

list_stores()

Lists all stores for the current context.

count_stores()

Counts the stores for the current context.

has_store(store_name)

Determines whether the specified store exists in the current context.

delete_store(store_name)

Deletes the specified store.

set_key(store_name, key, value[, ttl, policy])

Sets the value of a key in the specified store.

set_cache_key(store_name, key, value[, ttl])

Sets the value of a cache key in the specified store.

has_key(store_name, key)

Determines whether the specified key exists in the specified store.

get_key(store_name, key)

Retrieves the value of a key from the specified store.

delete_key(store_name, key)

Deletes the specified key from the store.

update_ttl(store_name, key, new_ttl)

Updates the TTL of the specified key in the store.

list_keys(store_name)

Lists all keys in the specified store.

count_keys(store_name)

Counts the keys in the specified store.

cleanup()

Deletes all stores associated with the current context.

has_store_global(store_name)

Determines whether a store with the given name exists across all datasets and the global context.

list_stores_global()

Lists the stores across all datasets and the global context.

count_stores_global()

Counts the stores across all datasets and the global context.

delete_store_global(store_name)

Deletes the specified store across all datasets and the global context.
create_store(store_name: str, metadata: dict[str, Any] | None = None, policy: str = 'persist') → StoreDocument#

Creates a new store with the specified name.

Parameters:

store_name – the name of the store

Returns:

a fiftyone.store.models.StoreDocument

clear_cache(store_name=None) → None#

Clears all cache entries in the execution stores.

get_store(store_name: str) → StoreDocument#

Gets the specified store for the current context.

Parameters:

store_name – the name of the store

Returns:

a fiftyone.store.models.StoreDocument

list_stores() → list[str]#

Lists all stores for the current context.

Returns:

a list of store names

count_stores() → int#

Counts the stores for the current context.

Returns:

the number of stores

has_store(store_name) → bool#

Determines whether the specified store exists in the current context.

Parameters:

store_name – the name of the store

Returns:

True/False

delete_store(store_name: str) → StoreDocument#

Deletes the specified store.

Parameters:

store_name – the name of the store

Returns:

a fiftyone.store.models.StoreDocument

set_key(store_name: str, key: str, value: Any, ttl: int | None = None, policy: str = 'persist') → KeyDocument#

Sets the value of a key in the specified store.

Keys can be either persistent or cacheable, depending on the provided policy or whether a TTL (time-to-live) is set.

  • If policy="persist" (default), the key will remain in the store until explicitly deleted.

  • If policy="evict", the key may be evicted by the system or manually removed using clear_cache().

  • If a TTL is provided, the key is always treated as policy="evict".

Parameters:

  • store_name – the name of the store

  • key – the key to set

  • value – the value to set

  • ttl (None) – an optional TTL in seconds

  • policy (persist) – the eviction policy for the key. Can be “persist” or “evict”. If “persist”, the key will never be automatically removed. If “evict”, the key may be removed automatically if a TTL is set, or manually via clear_cache().

Returns:

The created or updated key document.

Return type:

KeyDocument

set_cache_key(store_name: str, key: str, value: Any, ttl: int | None = None) → KeyDocument#

Sets the value of a cache key in the specified store.

Parameters:

  • store_name – the name of the store

  • key – the key to set

  • value – the value to set

  • ttl (None) – an optional TTL in seconds

has_key(store_name: str, key: str) → bool#

Determines whether the specified key exists in the specified store.

Parameters:

  • store_name – the name of the store

  • key – the key to check

get_key(store_name: str, key: str) → KeyDocument#

Retrieves the value of a key from the specified store.

Parameters:

  • store_name – the name of the store

  • key – the key to retrieve

Returns:

a fiftyone.store.models.KeyDocument

delete_key(store_name: str, key: str) → bool#

Deletes the specified key from the store.

Parameters:

  • store_name – the name of the store

  • key – the key to delete

Returns:

True if the key was deleted, False otherwise

update_ttl(store_name: str, key: str, new_ttl: int) → KeyDocument#

Updates the TTL of the specified key in the store.

Parameters:

  • store_name – the name of the store

  • key – the key to update the TTL for

  • new_ttl – the new TTL in seconds

Returns:

a fiftyone.store.models.KeyDocument

list_keys(store_name: str) → list[str]#

Lists all keys in the specified store.

Parameters:

store_name – the name of the store

Returns:

a list of keys in the store

count_keys(store_name: str) → int#

Counts the keys in the specified store.

Parameters:

store_name – the name of the store

Returns:

the number of keys in the store

cleanup() → None#

Deletes all stores associated with the current context.

has_store_global(store_name) → bool#

Determines whether a store with the given name exists across all datasets and the global context.

Parameters:

store_name – the name of the store

Returns:

True/False

list_stores_global() → list[StoreDocument]#

Lists the stores across all datasets and the global context.

Returns:

a list of fiftyone.store.models.StoreDocument

count_stores_global() → int#

Counts the stores across all datasets and the global context.

Returns:

the number of stores

delete_store_global(store_name) → int#

Deletes the specified store across all datasets and the global context.

Parameters:

store_name – the name of the store

Returns:

the number of stores deleted

class fiftyone.operators.store.ExecutionStore(store_name: str, store_service: ExecutionStoreService, default_policy: str = 'persist')#

Bases: object

Execution store.

Parameters:

Methods:

create(store_name[, dataset_id, ...])

list_stores()

Lists all stores in the execution store.

get(key)

Retrieves a value from the store by its key.

set(key, value[, ttl, policy])

Sets the value of a key in the specified store.

set_cache(key, value[, ttl])

Sets a value in the store with the eviction policy set to "evict".

delete(key)

Deletes a key from the store.

has(key)

Checks if the store has a specific key.

clear()

Clears all the data in the store.

clear_cache()

Clears the cache for the store.

update_ttl(key, new_ttl)

Updates the TTL for a specific key.

update_policy(key, policy)

Updates the eviction policy for a specific key.

get_metadata(key)

Retrieves the metadata for the given key.

list_keys()

Lists all keys in the store.
static create(store_name: str, dataset_id: ObjectId | None = None, default_policy: str = 'persist', collection_name: str | None = None) → ExecutionStore#
list_stores() → list[str]#

Lists all stores in the execution store.

Returns:

a list of store names

Return type:

list

get(key: str) → Any | None#

Retrieves a value from the store by its key.

Parameters:

key – the key to retrieve the value for

Returns:

the value stored under the given key, or None if not found

set(key: str, value: Any, ttl: int | None = None, policy=None) → None#

Sets the value of a key in the specified store.

Parameters:

  • key – the key to set

  • value – the value to set

  • ttl (None) – an optional TTL in seconds

  • policy (persist) – the eviction policy for the key. Can be “persist” or “evict”. If “persist”, the key will never be automatically removed. If “evict”, the key may be removed automatically if a TTL is set, or manually via clear_cache().

Returns:

a fiftyone.store.models.KeyDocument

set_cache(key: str, value: Any, ttl: int | None = None) → None#

Sets a value in the store with the eviction policy set to “evict”.

Parameters:

  • key – the key to store the value under

  • value – the value to store

  • ttl (None) – the time-to-live in seconds

delete(key: str) → bool#

Deletes a key from the store.

Parameters:

key – the key to delete.

Returns:

True/False whether the key was deleted

has(key: str) → bool#

Checks if the store has a specific key.

Parameters:

key – the key to check

Returns:

True/False whether the key exists

clear() → None#

Clears all the data in the store.

clear_cache() → None#

Clears the cache for the store.

This will remove all keys that are eligible for eviction.

update_ttl(key: str, new_ttl: int) → None#

Updates the TTL for a specific key.

Parameters:

  • key – the key to update the TTL for

  • new_ttl – the new TTL in seconds

update_policy(key: str, policy: str) → None#

Updates the eviction policy for a specific key.

Parameters:

  • key – the key to update the policy for

  • policy – the new policy, either “persist” or “evict”

get_metadata(key: str) → dict | None#

Retrieves the metadata for the given key.

Parameters:

key – the key to check

Returns:

a dict of metadata about the key

list_keys() → list[str]#

Lists all keys in the store.

Returns:

a list of keys in the store

class fiftyone.operators.store.StoreDocument(store_name: str, key: str = '__store__', value: dict[str, ~typing.Any] | None = None, _id: ~typing.Any | None = None, dataset_id: ~bson.objectid.ObjectId | None = None, created_at: ~datetime.datetime = <factory>, updated_at: ~datetime.datetime | None = None, expires_at: ~datetime.datetime | None = None, policy: ~fiftyone.operators.store.models.KeyPolicy = KeyPolicy.PERSIST)#

Bases: KeyDocument

Model representing a Store.

Attributes:

key

value

metadata

The metadata associated with the store.

dataset_id

expires_at

policy

updated_at

store_name

created_at

Methods:

from_dict(doc)

Creates a KeyDocument from a dictionary.

get_expiration(ttl)

Gets the expiration date for a key with the given TTL.

to_mongo_dict([exclude_id])

Serializes the document to a MongoDB dictionary.
key: str = '__store__'#
value: dict[str, Any] | None = None#
property metadata: dict[str, Any]#

The metadata associated with the store.

dataset_id: ObjectId | None = None#
expires_at: datetime | None = None#
classmethod from_dict(doc: dict[str, Any]) → KeyDocument#

Creates a KeyDocument from a dictionary.

static get_expiration(ttl: int | None) → datetime | None#

Gets the expiration date for a key with the given TTL.

policy: KeyPolicy = 'persist'#
to_mongo_dict(exclude_id: bool = True) → dict[str, Any]#

Serializes the document to a MongoDB dictionary.

updated_at: datetime | None = None#
store_name: str#
created_at: datetime#
class fiftyone.operators.store.KeyDocument(store_name: str, key: str, value: ~typing.Any, _id: ~typing.Any | None = None, dataset_id: ~bson.objectid.ObjectId | None = None, created_at: ~datetime.datetime = <factory>, updated_at: ~datetime.datetime | None = None, expires_at: ~datetime.datetime | None = None, policy: ~fiftyone.operators.store.models.KeyPolicy = KeyPolicy.PERSIST)#

Bases: object

Model representing a key in the store.

Attributes:

store_name

key

value

dataset_id

created_at

updated_at

expires_at

policy

Methods:

get_expiration(ttl)

Gets the expiration date for a key with the given TTL.

from_dict(doc)

Creates a KeyDocument from a dictionary.

to_mongo_dict([exclude_id])

Serializes the document to a MongoDB dictionary.
store_name: str#
key: str#
value: Any#
dataset_id: ObjectId | None = None#
created_at: datetime#
updated_at: datetime | None = None#
expires_at: datetime | None = None#
policy: KeyPolicy = 'persist'#
static get_expiration(ttl: int | None) → datetime | None#

Gets the expiration date for a key with the given TTL.

classmethod from_dict(doc: dict[str, Any]) → KeyDocument#

Creates a KeyDocument from a dictionary.

to_mongo_dict(exclude_id: bool = True) → dict[str, Any]#

Serializes the document to a MongoDB dictionary.

class fiftyone.operators.store.KeyPolicy(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: str, Enum

Defines the eviction policy for a key in the execution store.

  • PERSIST: The key is stored persistently and will never be automatically removed. It must be explicitly deleted.

  • EVICT: The key is considered cacheable and may be removed automatically if a TTL is set, or manually via clear_cache().

Attributes:

PERSIST

EVICT

Methods:

encode([encoding, errors])

Encode the string using the codec registered for encoding.

replace(old, new[, count])

Return a copy with all occurrences of substring old replaced by new.

split([sep, maxsplit])

Return a list of the substrings in the string, using sep as the separator string.

rsplit([sep, maxsplit])

Return a list of the substrings in the string, using sep as the separator string.

join(iterable, /)

Concatenate any number of strings.

capitalize()

Return a capitalized version of the string.

casefold()

Return a version of the string suitable for caseless comparisons.

title()

Return a version of the string where each word is titlecased.

center(width[, fillchar])

Return a centered string of length width.

count(sub[, start[, end]])

Return the number of non-overlapping occurrences of substring sub in string S[start:end].

expandtabs([tabsize])

Return a copy where all tab characters are expanded using spaces.

find(sub[, start[, end]])

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].

partition(sep, /)

Partition the string into three parts using the given separator.

index(sub[, start[, end]])

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].

ljust(width[, fillchar])

Return a left-justified string of length width.

lower()

Return a copy of the string converted to lowercase.

lstrip([chars])

Return a copy of the string with leading whitespace removed.

rfind(sub[, start[, end]])

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].

rindex(sub[, start[, end]])

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].

rjust(width[, fillchar])

Return a right-justified string of length width.

rstrip([chars])

Return a copy of the string with trailing whitespace removed.

rpartition(sep, /)

Partition the string into three parts using the given separator.

splitlines([keepends])

Return a list of the lines in the string, breaking at line boundaries.

strip([chars])

Return a copy of the string with leading and trailing whitespace removed.

swapcase()

Convert uppercase characters to lowercase and lowercase characters to uppercase.

translate(table, /)

Replace each character in the string using the given translation table.

upper()

Return a copy of the string converted to uppercase.

startswith(prefix[, start[, end]])

Return True if S starts with the specified prefix, False otherwise.

endswith(suffix[, start[, end]])

Return True if S ends with the specified suffix, False otherwise.

removeprefix(prefix, /)

Return a str with the given prefix string removed if present.

removesuffix(suffix, /)

Return a str with the given suffix string removed if present.

isascii()

Return True if all characters in the string are ASCII, False otherwise.

islower()

Return True if the string is a lowercase string, False otherwise.

isupper()

Return True if the string is an uppercase string, False otherwise.

istitle()

Return True if the string is a title-cased string, False otherwise.

isspace()

Return True if the string is a whitespace string, False otherwise.

isdecimal()

Return True if the string is a decimal string, False otherwise.

isdigit()

Return True if the string is a digit string, False otherwise.

isnumeric()

Return True if the string is a numeric string, False otherwise.

isalpha()

Return True if the string is an alphabetic string, False otherwise.

isalnum()

Return True if the string is an alpha-numeric string, False otherwise.

isidentifier()

Return True if the string is a valid Python identifier, False otherwise.

isprintable()

Return True if the string is printable, False otherwise.

zfill(width, /)

Pad a numeric string with zeros on the left, to fill a field of the given width.

format(*args, **kwargs)

Return a formatted version of S, using substitutions from args and kwargs.

format_map(mapping)

Return a formatted version of S, using substitutions from mapping.

maketrans

Return a translation table usable for str.translate().
PERSIST = 'persist'#
EVICT = 'evict'#
encode(encoding='utf-8', errors='strict')#

Encode the string using the codec registered for encoding.

encoding

The encoding in which to encode the string.

errors

The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.

replace(old, new, count=-1, /)#

Return a copy with all occurrences of substring old replaced by new.

count

Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.

If the optional argument count is given, only the first count occurrences are replaced.

split(sep=None, maxsplit=-1)#

Return a list of the substrings in the string, using sep as the separator string.

sep

The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit

Maximum number of splits. -1 (the default value) means no limit.

Splitting starts at the front of the string and works to the end.

Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.

rsplit(sep=None, maxsplit=-1)#

Return a list of the substrings in the string, using sep as the separator string.

sep

The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit

Maximum number of splits. -1 (the default value) means no limit.

Splitting starts at the end of the string and works to the front.

join(iterable, /)#

Concatenate any number of strings.

The string whose method is called is inserted in between each given string. The result is returned as a new string.

Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’

capitalize()#

Return a capitalized version of the string.

More specifically, make the first character have upper case and the rest lower case.

casefold()#

Return a version of the string suitable for caseless comparisons.

title()#

Return a version of the string where each word is titlecased.

More specifically, words start with uppercased characters and all remaining cased characters have lower case.

center(width, fillchar=' ', /)#

Return a centered string of length width.

Padding is done using the specified fill character (default is a space).

count(sub[, start[, end]]) → int#

Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.

expandtabs(tabsize=8)#

Return a copy where all tab characters are expanded using spaces.

If tabsize is not given, a tab size of 8 characters is assumed.

find(sub[, start[, end]]) → int#

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

partition(sep, /)#

Partition the string into three parts using the given separator.

This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing the original string and two empty strings.

index(sub[, start[, end]]) → int#

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

ljust(width, fillchar=' ', /)#

Return a left-justified string of length width.

Padding is done using the specified fill character (default is a space).

lower()#

Return a copy of the string converted to lowercase.

lstrip(chars=None, /)#

Return a copy of the string with leading whitespace removed.

If chars is given and not None, remove characters in chars instead.

rfind(sub[, start[, end]]) → int#

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

rindex(sub[, start[, end]]) → int#

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

rjust(width, fillchar=' ', /)#

Return a right-justified string of length width.

Padding is done using the specified fill character (default is a space).

rstrip(chars=None, /)#

Return a copy of the string with trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

rpartition(sep, /)#

Partition the string into three parts using the given separator.

This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing two empty strings and the original string.

splitlines(keepends=False)#

Return a list of the lines in the string, breaking at line boundaries.

Line breaks are not included in the resulting list unless keepends is given and true.

strip(chars=None, /)#

Return a copy of the string with leading and trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

swapcase()#

Convert uppercase characters to lowercase and lowercase characters to uppercase.

translate(table, /)#

Replace each character in the string using the given translation table.

table

Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.

The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted.

upper()#

Return a copy of the string converted to uppercase.

startswith(prefix[, start[, end]]) → bool#

Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.

endswith(suffix[, start[, end]]) → bool#

Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.

removeprefix(prefix, /)#

Return a str with the given prefix string removed if present.

If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.

removesuffix(suffix, /)#

Return a str with the given suffix string removed if present.

If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.

isascii()#

Return True if all characters in the string are ASCII, False otherwise.

ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.

islower()#

Return True if the string is a lowercase string, False otherwise.

A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.

isupper()#

Return True if the string is an uppercase string, False otherwise.

A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.

istitle()#

Return True if the string is a title-cased string, False otherwise.

In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.

isspace()#

Return True if the string is a whitespace string, False otherwise.

A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.

isdecimal()#

Return True if the string is a decimal string, False otherwise.

A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.

isdigit()#

Return True if the string is a digit string, False otherwise.

A string is a digit string if all characters in the string are digits and there is at least one character in the string.

isnumeric()#

Return True if the string is a numeric string, False otherwise.

A string is numeric if all characters in the string are numeric and there is at least one character in the string.

isalpha()#

Return True if the string is an alphabetic string, False otherwise.

A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.

isalnum()#

Return True if the string is an alpha-numeric string, False otherwise.

A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.

isidentifier()#

Return True if the string is a valid Python identifier, False otherwise.

Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.

isprintable()#

Return True if the string is printable, False otherwise.

A string is printable if all of its characters are considered printable in repr() or if it is empty.

zfill(width, /)#

Pad a numeric string with zeros on the left, to fill a field of the given width.

The string is never truncated.

format(*args, **kwargs) → str#

Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{’ and ‘}’).

format_map(mapping) → str#

Return a formatted version of S, using substitutions from mapping. The substitutions are identified by braces (‘{’ and ‘}’).

static maketrans()#

Return a translation table usable for str.translate().

If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will be then converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in x will be mapped to the character at the same position in y. If there is a third argument, it must be a string, whose characters will be mapped to None in the result.

IN THIS ARTICLE