fiftyone.operators.executor#
FiftyOne operator execution.
Classes:
Enumeration of the available operator run states. |
|
|
Represents a request to invoke an operator. |
|
Represents the status of an operator execution. |
|
Handles the execution phase of the operator lifecycle. |
|
Represents the execution context of an operator. |
|
Represents the result of an operator execution. |
|
A validation error. |
|
Represents the validation context of an operator. |
|
Represents the execution options of an operation. |
Functions:
|
Executes the operator with the given name. |
|
Executes the operator with the given name. |
|
|
|
|
|
Resolves the inputs property type of the operator with the given name. |
|
Resolves the "inputs" or "outputs" schema of an operator with the given context. |
|
Resolves the execution options of the operator with the given name. |
|
Resolves the placement of the operator with the given name. |
Exceptions:
An error that occurs while executing an operator. |
- class fiftyone.operators.executor.ExecutionRunState#
Bases:
object
Enumeration of the available operator run states.
Attributes:
- SCHEDULED = 'scheduled'#
- QUEUED = 'queued'#
- RUNNING = 'running'#
- PROCESSING = 'processing'#
- COMPLETED = 'completed'#
- FAILED = 'failed'#
- class fiftyone.operators.executor.InvocationRequest(operator_uri, params=None)#
Bases:
object
Represents a request to invoke an operator.
- Parameters:
operator_uri – the URI of the operator to invoke
params (None) – an optional dictionary of parameters
Methods:
to_json
()- to_json()#
- class fiftyone.operators.executor.ExecutionProgress(progress=None, label=None)#
Bases:
object
Represents the status of an operator execution.
- Parameters:
progress (None) – an optional float between 0 and 1 (0% to 100%)
label (None) – an optional label to display
- class fiftyone.operators.executor.Executor(requests=None, logs=None)#
Bases:
object
Handles the execution phase of the operator lifecycle.
- Parameters:
requests (None) – an optional list of InvocationRequest objects
logs (None) – an optional list of log messages
Methods:
trigger
(operator_name[, params])Triggers an invocation of the operator with the given name.
log
(message)Logs a message.
to_json
()- trigger(operator_name, params=None)#
Triggers an invocation of the operator with the given name.
- Parameters:
operator_name – the name of the operator
params (None) – a dictionary of parameters for the operator
- Returns:
a
fiftyone.operators.message.GeneratedMessage
containing instructions for the FiftyOne App to invoke the operator
- log(message)#
Logs a message.
- to_json()#
- fiftyone.operators.executor.execute_operator(operator_uri, ctx=None, **kwargs)#
Executes the operator with the given name.
- Parameters:
operator_uri – the URI of the operator
ctx (None) –
a dictionary of parameters defining the execution context. The supported keys are:
dataset
: afiftyone.core.dataset.Dataset
or the name of a dataset to process. This is required unless aview
is providedview
(None): an optionalfiftyone.core.view.DatasetView
to processselected
([]): an optional list of selected sample IDsselected_labels
([]): an optional list of selected labels in the format returned byfiftyone.core.session.Session.selected_labels
current_sample
(None): an optional ID of the current sample being processedextended_selection
(None): an optional extended selection of the view.params
: a dictionary of parameters for the operator. Consult the operator’s documentation for detailsrequest_delegation
(False): whether to request delegated execution, if supported by the operatordelegation_target
(None): an optional orchestrator on which to schedule the operation, if it is delegatedactive_fields
([]): a list of active field namesworkspace_name
(None): an optional name of the workspace to use for the operationspaces
(None): an optional dictionary defining spaces to use for the operationgroup_slice
(None): an optional group slice to use for the operation’s view. This is only applicable to group datasetsquery_performance
(None): whether to enable query performance
num_distributed_tasks
(None): the number of tasks to split
the operation into, if it is delegated.
**kwargs – you can optionally provide any of the supported
ctx
keys as keyword arguments rather than including them inctx
- Returns:
an
ExecutionResult
, or anasyncio.Task
if you run this method in a notebook context- Raises:
ExecutionError – if an error occurred while immediately executing an operation or scheduling a delegated operation
- async fiftyone.operators.executor.execute_or_delegate_operator(operator_uri, request_params, exhaust=False)#
Executes the operator with the given name.
- Parameters:
operator_uri – the URI of the operator
request_params – a dictionary of parameters for the operator
exhaust (False) – whether to immediately exhaust generator operators
- Returns:
- async fiftyone.operators.executor.prepare_operator_executor(operator_uri, request_params, set_progress=None, delegated_operation_id=None)#
- async fiftyone.operators.executor.do_execute_operator(operator, ctx, exhaust=False)#
- async fiftyone.operators.executor.resolve_type(registry, operator_uri, request_params)#
Resolves the inputs property type of the operator with the given name.
- Parameters:
registry – an
fiftyone.operators.registry.OperatorRegistry
operator_uri – the URI of the operator
request_params – a dictionary of request parameters
- Returns:
the type of the inputs
fiftyone.operators.types.Property
of the operator, or None
- async fiftyone.operators.executor.resolve_type_with_context(operator, context)#
Resolves the “inputs” or “outputs” schema of an operator with the given context.
- Parameters:
operator – the
fiftyone.operators.Operator
context – the
ExecutionContext
of an operator
- Returns:
the “inputs” or “outputs” schema
fiftyone.operators.types.Property
of an operator, or None
- async fiftyone.operators.executor.resolve_execution_options(registry, operator_uri, request_params)#
Resolves the execution options of the operator with the given name.
- Parameters:
registry – an
fiftyone.operators.registry.OperatorRegistry
operator_uri – the URI of the operator
request_params – a dictionary of request parameters
- Returns:
- fiftyone.operators.executor.resolve_placement(operator, request_params)#
Resolves the placement of the operator with the given name.
- Parameters:
operator – the
fiftyone.operators.operator.Operator
request_params – a dictionary of request parameters
- Returns:
the placement of the operator or
None
- class fiftyone.operators.executor.ExecutionContext(request_params=None, executor=None, set_progress=None, delegated_operation_id=None, operator_uri=None, required_secrets=None)#
Bases:
object
Represents the execution context of an operator.
Operators can use the execution context to access the view, dataset, and selected samples, as well as to trigger other operators.
- Parameters:
request_params (None) – a optional dictionary of request parameters
executor (None) – an optional
Executor
instanceset_progress (None) – an optional function to set the progress of the current operation
delegated_operation_id (None) – an optional ID of the delegated operation
operator_uri (None) – the unique id of the operator
required_secrets (None) – the list of required secrets from the plugin’s definition
Attributes:
The
fiftyone.core.dataset.Dataset
being operated on.The name of the
fiftyone.core.dataset.Dataset
being operated on.The ID of the
fiftyone.core.dataset.Dataset
being operated on.The
fiftyone.core.view.DatasetView
being operated on.Whether the operator has a custom view.
The current spaces layout in the FiftyOne App.
The list of selected sample IDs (if any).
A list of selected labels (if any).
The extended selection of the view (if any).
The ID of the current sample being processed (if any).
The list of currently active fields in the FiftyOne App sidebar.
The ID of the user executing the operation, if known.
The request token authenticating the user executing the operation, if known.
The ID of the panel that invoked the operator, if any.
The current panel state.
A
fiftyone.operators.panel.PanelRef
instance that you can use to read and write the state and data of the current panel.Whether the operation was delegated.
Whether delegated execution was requested for the operation.
The orchestrator to which the operation was delegated (if any).
A
dict
of results for the current operation.A read-only mapping of keys to their resolved values.
A
fiftyone.operators.operations.Operations
instance that you can use to trigger builtin operations on the current context.The current group slice of the view (if any).
The number of tasks this job should be split into.
Whether query performance is enabled.
An identifier for the prompt, unique to each instance of a user opening a prompt in the FiftyOne App.
The URI of the target operator.
Methods:
target_view
([param_name])The target
fiftyone.core.view.DatasetView
for the operator being executed.prompt
(operator_uri[, params, on_success, ...])Prompts the user to execute the operator with the given URI.
secret
(key)Retrieves the secret with the given key.
resolve_secret_values
(keys, **kwargs)Resolves the values of the given secrets keys.
trigger
(operator_name[, params])Triggers an invocation of the operator with the given name.
log
(message)Logs a message to the browser console.
set_progress
([progress, label])Sets the progress of the current operation.
store
(store_name)Retrieves the execution store with the given name.
Serializes the execution context.
to_dict
()Returns the properties of the execution context as a dict.
- property dataset#
The
fiftyone.core.dataset.Dataset
being operated on.
- property dataset_name#
The name of the
fiftyone.core.dataset.Dataset
being operated on.
- property dataset_id#
The ID of the
fiftyone.core.dataset.Dataset
being operated on.
- property view#
The
fiftyone.core.view.DatasetView
being operated on.
- target_view(param_name='view_target')#
The target
fiftyone.core.view.DatasetView
for the operator being executed.- Parameters:
param_name ("view_target") – the name of the enum parameter defining the target view choice
- Returns:
- property has_custom_view#
Whether the operator has a custom view.
- property spaces#
The current spaces layout in the FiftyOne App.
- property selected#
The list of selected sample IDs (if any).
- property selected_labels#
A list of selected labels (if any).
Items are dictionaries with the following keys:
label_id
: the ID of the labelsample_id
: the ID of the sample containing the labelfield
: the field name containing the labelframe_number
: the frame number containing the label (only applicable to video samples)
- property extended_selection#
The extended selection of the view (if any).
- property current_sample#
The ID of the current sample being processed (if any).
When executed via the FiftyOne App, this is set when the user opens a sample in the modal.
- property active_fields#
The list of currently active fields in the FiftyOne App sidebar.
- property user_id#
The ID of the user executing the operation, if known.
- property user_request_token#
The request token authenticating the user executing the operation, if known.
- property panel_id#
The ID of the panel that invoked the operator, if any.
- property panel_state#
The current panel state.
Only available when the operator is invoked from a panel.
- property panel#
A
fiftyone.operators.panel.PanelRef
instance that you can use to read and write the state and data of the current panel.Only available when the operator is invoked from a panel.
- property delegated#
Whether the operation was delegated.
- property requesting_delegated_execution#
Whether delegated execution was requested for the operation.
- property delegation_target#
The orchestrator to which the operation was delegated (if any).
- property results#
A
dict
of results for the current operation.
- property secrets#
A read-only mapping of keys to their resolved values.
- property ops#
A
fiftyone.operators.operations.Operations
instance that you can use to trigger builtin operations on the current context.
- property group_slice#
The current group slice of the view (if any).
- property num_distributed_tasks#
The number of tasks this job should be split into.
- property query_performance#
Whether query performance is enabled.
- property prompt_id#
An identifier for the prompt, unique to each instance of a user opening a prompt in the FiftyOne App.
- property operator_uri#
The URI of the target operator.
- prompt(operator_uri, params=None, on_success=None, on_error=None, skip_prompt=False)#
Prompts the user to execute the operator with the given URI.
- Parameters:
operator_uri – the URI of the operator
params (None) – a dictionary of parameters for the operator
on_success (None) – a callback to invoke if the user successfully executes the operator
on_error (None) – a callback to invoke if the execution fails
skip_prompt (False) – whether to skip the prompt
- Returns:
a
fiftyone.operators.message.GeneratedMessage
containing instructions for the FiftyOne App to prompt the user
- secret(key)#
Retrieves the secret with the given key.
- Parameters:
key – a secret key
- Returns:
the secret value
- async resolve_secret_values(keys, **kwargs)#
Resolves the values of the given secrets keys.
- Parameters:
keys – a list of secret keys
**kwargs – additional keyword arguments to pass to the secrets client for authentication if required
- trigger(operator_name, params=None)#
Triggers an invocation of the operator with the given name.
This method is only available when the operator is invoked via the FiftyOne App. You can check this via
ctx.executor
.Example:
def execute(self, ctx): # Trigger the `reload_dataset` operator after this operator # finishes executing ctx.trigger("reload_dataset") # Immediately trigger the `reload_dataset` operator while a # generator operator is executing yield ctx.trigger("reload_dataset")
- Parameters:
operator_name – the name of the operator
params (None) – a dictionary of parameters for the operator
- Returns:
a
fiftyone.operators.message.GeneratedMessage
containing instructions for the FiftyOne App to invoke the operator
- log(message)#
Logs a message to the browser console.
Note
This method is only available to non-delegated operators. You can only use this method during the execution of an operator.
- Parameters:
message – a message to log
- Returns:
a
fiftyone.operators.message.GeneratedMessage
containing instructions for the FiftyOne App to invoke the operator
- set_progress(progress=None, label=None)#
Sets the progress of the current operation.
- Parameters:
progress (None) – an optional float between 0 and 1 (0% to 100%)
label (None) – an optional label to display
- store(store_name)#
Retrieves the execution store with the given name.
The store is automatically created if necessary.
- Parameters:
store_name – the name of the store
- Returns:
- serialize()#
Serializes the execution context.
- Returns:
a JSON dict
- to_dict()#
Returns the properties of the execution context as a dict.
- class fiftyone.operators.executor.ExecutionResult(result=None, executor=None, error=None, error_message=None, validation_ctx=None, delegated=False, outputs_schema=None)#
Bases:
object
Represents the result of an operator execution.
- Parameters:
result (None) – the execution result
executor (None) – an
Executor
error (None) – an error traceback, if an error occurred
error_message (None) – an error message, if an error occurred
validation_ctx (None) – a
ValidationContext
delegated (False) – whether execution was delegated
outputs_schema (None) – a JSON dict representing the output schema of the operator
Attributes:
Whether the result is a generator or an async generator.
Methods:
Raises an
ExecutionError
(only) if the operation failed.Returns an
ExecutionError
representing a failed execution result.to_json
()Returns a JSON dict representation of the result.
- property is_generator#
Whether the result is a generator or an async generator.
- raise_exceptions()#
Raises an
ExecutionError
(only) if the operation failed.
- to_exception()#
Returns an
ExecutionError
representing a failed execution result.- Returns:
a
ExecutionError
, or None if the execution did not fail
- to_json()#
Returns a JSON dict representation of the result.
- Returns:
a JSON dict
- exception fiftyone.operators.executor.ExecutionError#
Bases:
Exception
An error that occurs while executing an operator.
Methods:
Exception.add_note(note) -- add a note to the exception
Exception.with_traceback(tb) -- set self.__traceback__ to tb and return self.
Attributes:
- add_note()#
Exception.add_note(note) – add a note to the exception
- args#
- with_traceback()#
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
- class fiftyone.operators.executor.ValidationError(reason, property, path, custom=False)#
Bases:
object
A validation error.
- Parameters:
reason – the reason
property – the property
path – the path
Methods:
to_json
()Returns a JSON dict representation of the error.
- to_json()#
Returns a JSON dict representation of the error.
- Returns:
a JSON dict
- class fiftyone.operators.executor.ValidationContext(ctx, inputs_property, operator)#
Bases:
object
Represents the validation context of an operator.
- Parameters:
ctx – the
ExecutionContext
inputs_property – the
fiftyone.operators.types.Property
of the operator inputsoperator – the
fiftyone.operators.operator.Operator
Methods:
to_json
()Returns a JSON dict representation of the context.
add_error
(error)Adds a validation error.
validate_enum
(path, property, value)Validates an enum value.
validate_list
(path, property, value)Validates a list value.
validate_property
(path, property, value)Validates a property value.
validate_object
(path, property, value)Validates an object value.
validate_primitive
(path, property, value)Validates a primitive value.
exists_or_non_required
(property, value)- to_json()#
Returns a JSON dict representation of the context.
- Returns:
a JSON dict
- add_error(error)#
Adds a validation error.
- Parameters:
error – a
ValidationError
- validate_enum(path, property, value)#
Validates an enum value.
- Parameters:
path – the path to the property
property – the
fiftyone.operators.types.Property
value – the value to validate
- Returns:
a
ValidationError
, if the value is invalid
- validate_list(path, property, value)#
Validates a list value.
- Parameters:
path – the path to the property
property – the
fiftyone.operators.types.Property
value – the value to validate
- Returns:
a
ValidationError
, if the value is invalid
- validate_property(path, property, value)#
Validates a property value.
- Parameters:
path – the path to the property
property – the
fiftyone.operators.types.Property
value – the value to validate
- Returns:
a
ValidationError
, if the value is invalid
- validate_object(path, property, value)#
Validates an object value.
- Parameters:
path – the path to the property
property – the
fiftyone.operators.types.Property
value – the value to validate
- Returns:
a
ValidationError
, if the value is invalid
- validate_primitive(path, property, value)#
Validates a primitive value.
- Parameters:
path – the path to the property
property – the
fiftyone.operators.types.Property
value – the value to validate
- Returns:
a
ValidationError
, if the value is invalid
- exists_or_non_required(property, value)#
- class fiftyone.operators.executor.ExecutionOptions(allow_immediate_execution=True, allow_delegated_execution=False, default_choice_to_delegated=False, allow_distributed_execution=False, min_distributed_tasks=2, max_distributed_tasks=None, recommended_distributed_tasks=None, **_)#
Bases:
object
Represents the execution options of an operation.
- Parameters:
allow_immediate_execution (True) – whether the operation can be executed immediately
allow_delegated_execution (False) – whether the operation can be delegated to an orchestrator
default_choice_to_delegated (False) – whether to default to delegated execution, if allowed
allow_distributed_execution (False) – whether the operator supports distributing delegated execution across parallel workers. Only valid for delegated operations.
min_distributed_tasks (2) – the minimum number of tasks that a distributed delegated operation can be split into. None means no limit. Only valid for distributed and delegated operations.
max_distributed_tasks (None) – the maximum number of tasks that a distributed delegated operation can be split into. None means no limit. Only valid for distributed and delegated operations.
recommended_distributed_tasks (None) – the recommended number of tasks that a distributed delegated operation should be split into. None means no recommendation. Only valid for distributed and delegated operations.
Attributes:
Methods:
- property allow_immediate_execution#
- property allow_delegated_execution#
- property allow_distributed_execution#
- property default_choice_to_delegated#
- property min_distributed_tasks#
- property max_distributed_tasks#
- property recommended_distributed_tasks#
- property available_orchestrators#
- property orchestrator_registration_enabled#
- update(available_orchestrators=None)#
- to_dict()#