plugins.operators.model_evaluation#

plugins.operators.model_evaluation.utils

Module contents#

Scenario plugin.

Copyright 2017-2025, Voxel51, Inc.
voxel51.com

Functions:

dataset_serialize_deserialize(dataset)

Classes:

ConfigureScenario([_builtin])

plugins.operators.model_evaluation.dataset_serialize_deserialize(dataset)#

class plugins.operators.model_evaluation.ConfigureScenario(_builtin=False)#

Bases: Operator

Attributes:

`last_view_type_used`
`config`	The `OperatorConfig` for the operator.
`builtin`	Whether the operator is builtin.
`name`
`uri`	The unique identifier of the operator: `plugin_name/operator_name`.

Methods:

`get_dataset`(ctx)	Returns the dataset for the current context.
`get_samples_count`(ctx)	Returns the number of samples in the dataset for the current context.
`get_default_for_distribution_preview`(ctx)	Returns the default value for the sample distribution preview.
`render_name_input`(ctx, inputs)
`render_scenario_types`(inputs, selected_type)
`extract_evaluation_keys`(ctx)
`extract_evaluation_id`(ctx)
`get_subset_def_data_for_eval_key`(ctx, ...)	Builds and returns an execution cache key for each type of scenario.
`get_evaluations_results`(ctx)	Returns the evaluation results for the current context.
`get_subset_def_data_for_eval`(ctx, _, ...)
`get_sample_distribution`(ctx, subset_expressions)
`convert_to_plotly_data`(preview_data)
`is_sample_distribution_enabled_for_custom_code`(params)
`render_empty_sample_distribution`(ctx, ...[, ...])
`get_label_attribute_path`(params)	Returns the last part of the label attribute path to use in use_subset(type="attribute", ...)
`render_sample_distribution`(ctx, inputs, ...)
`render_plot_preview_toggle`(ctx, inputs)
`render_sample_distribution_graph`(ctx, ...)
`get_custom_code_key`(params)
`extract_custom_code`(ctx[, example_type])
`render_custom_code`(ctx, inputs[, example_type])
`render_no_values_warning`(inputs, field_name)
`render_use_custom_code_warning`(inputs[, reason])
`render_use_custom_code_instead`(ctx, inputs)
`get_saved_view_scenarios_picker_type`(ctx)	Returns the view mode for saved views based on the number of available saved views.
`render_saved_views`(ctx, inputs)	Renders sample distribution for subsets using saved views.
`get_scenario_values_key`(params)	returns a unique key that holds the latest selected values for a - certain scenario type.
`get_selected_values`(params)
`render_checkbox_view`(ctx, values, inputs[, ...])
`get_scenarios_picker_type`(ctx, field_name)	Determines the scenario picker type for a given field based on its type and distinct values.
`render_auto_complete_view`(ctx, values, inputs)	Renders an auto-complete view as a more compact/efficient input option when, - There are too many distinct values to display and The field is of type string.
`render_scenario_picker_view`(ctx, field_name, ...)	Renders the appropriate UI component based on the picker type.
`get_valid_label_attribute_path_options`(...)	Returns all the paths in the dataset schema that are valid label attributes types.
`render_label_attribute`(ctx, inputs, gt_field)
`get_valid_sample_field_path_options`(...)	Get valid sample field path options based on the schema. - Filters out - fields having roots with type as List of documents - non-primitives - non-list of primitives.
`render_sample_fields`(ctx, inputs[, field_name])
`get_scenario_type`(params)
`get_modal_title`(ctx)
`get_scenario_names`(ctx)
`resolve_input`(ctx)	Returns the resolved input property.
`render_custom_code_content`(inputs, ...)
`execute`(ctx)	Executes the operator.
`add_secrets`(secrets)	Adds secrets to the operator.
`method_to_uri`(method_name)	Converts a method name to a URI.
`resolve_delegation`(ctx)	Returns the resolved forced delegation flag.
`resolve_execution_options`(ctx)	Returns the resolved execution options.
`resolve_output`(ctx)	Returns the resolved output property.
`resolve_placement`(ctx)	Returns the resolved placement of the operator.
`resolve_run_name`(ctx)	Returns the resolved run name of the operator.
`resolve_type`(ctx, type)	Returns the resolved input or output property.
`to_json`()	Returns a JSON representation of the operator.

last_view_type_used = None#

property config#: The OperatorConfig for the operator.

get_dataset(ctx)#: Returns the dataset for the current context.

get_samples_count(ctx)#: Returns the number of samples in the dataset for the current context.

get_default_for_distribution_preview(ctx)#: Returns the default value for the sample distribution preview.

render_name_input(ctx, inputs)#

render_scenario_types(inputs, selected_type)#

extract_evaluation_keys(ctx)#

extract_evaluation_id(ctx)#

get_subset_def_data_for_eval_key(ctx, eval_key, _, name, subset_def)#: Builds and returns an execution cache key for each type of scenario. - eval key + name + type + subset definition

get_evaluations_results(ctx)#: Returns the evaluation results for the current context.

get_subset_def_data_for_eval(ctx, _, eval_result, name, subset_def)#

get_sample_distribution(ctx, subset_expressions)#

convert_to_plotly_data(preview_data)#

is_sample_distribution_enabled_for_custom_code(params)#

render_empty_sample_distribution(ctx, inputs, params, description=None)#

get_label_attribute_path(params)#: Returns the last part of the label attribute path to use in use_subset(type=”attribute”, …)

render_sample_distribution(ctx, inputs, scenario_type, values)#

render_plot_preview_toggle(ctx, inputs)#

render_sample_distribution_graph(ctx, inputs, subset_expressions)#

get_custom_code_key(params)#

extract_custom_code(ctx, example_type=ScenarioType.CUSTOM_CODE)#

render_custom_code(ctx, inputs, example_type=ScenarioType.CUSTOM_CODE)#

render_no_values_warning(inputs, field_name, link='')#

render_use_custom_code_warning(inputs, reason=CustomCodeViewReason.TOO_MANY_CATEGORIES)#

render_use_custom_code_instead(ctx, inputs, reason=CustomCodeViewReason.TOO_MANY_CATEGORIES)#

get_saved_view_scenarios_picker_type(ctx)#: Returns the view mode for saved views based on the number of available saved views.

render_saved_views(ctx, inputs)#: Renders sample distribution for subsets using saved views.

get_scenario_values_key(params)#: returns a unique key that holds the latest selected values for a - certain scenario type. has to be one of type ScenarioType - certain scenario field (ex: “tags”, “labels”)

get_selected_values(params)#

render_checkbox_view(ctx, values, inputs, with_description=None)#

get_scenarios_picker_type(ctx, field_name)#

Determines the scenario picker type for a given field based on its type and distinct values. - if the field is of type float, shows custom code mode - if the field is boolean, shows checkbox mode - if the field has no distinct values, shows empty mode - if the field has too many distinct values, shows auto-complete or custom code mode - if the field has a manageable number of distinct values, shows checkbox mode

Returns:: Picker type and corresponding values.
Return type:: Tuple[str, Any]

render_auto_complete_view(ctx, values, inputs)#

Renders an auto-complete view as a more compact/efficient input option when, - There are too many distinct values to display and The field is of type string. - There are too many distinct saved views to display as checkbox.

Eventually, if there are selected values, it attempts to render the sample distribution preview graph

render_scenario_picker_view(ctx, field_name, inputs)#: Renders the appropriate UI component based on the picker type.

get_valid_label_attribute_path_options(schema, gt_field)#: Returns all the paths in the dataset schema that are valid label attributes types. - primitives - OR list of primitives - AND path starts with gt_field

render_label_attribute(ctx, inputs, gt_field, label_attr=None)#

get_valid_sample_field_path_options(flat_field_schema)#

Get valid sample field path options based on the schema. - Filters out

fields having roots with type as List of documents

non-primitives

non-list of primitives

render_sample_fields(ctx, inputs, field_name=None)#

get_scenario_type(params)#

get_modal_title(ctx)#

get_scenario_names(ctx)#

resolve_input(ctx)#

Returns the resolved input property.

Subclasses can implement this method to define the inputs to the operator. This method should never be called directly. Instead use resolve_type().

By default, this method is called once when the operator is created. If the operator is dynamic, this method is called each time the input changes.

Parameters:: ctx – the fiftyone.operators.executor.ExecutionContext
Returns:: a fiftyone.operators.types.Property, or None

render_custom_code_content(inputs, custom_code, code_key)#

execute(ctx)#

Executes the operator.

Subclasses must implement this method.

Parameters:: ctx – the fiftyone.operators.executor.ExecutionContext
Returns:: JSON serializable data, or None

add_secrets(secrets)#

Adds secrets to the operator.

Parameters:: secrets – a list of secrets

property builtin#: Whether the operator is builtin.

method_to_uri(method_name)#

Converts a method name to a URI.

Parameters:: method_name – the method name
Returns:: a URI

property name#

resolve_delegation(ctx)#

Returns the resolved forced delegation flag.

Subclasses can implement this method to decide if delegated execution should be forced for the given operation.

Parameters:: ctx – the fiftyone.operators.executor.ExecutionContext
Returns:: whether the operation should be delegated (True), run immediately (False), or None to defer to resolve_execution_options() to specify the available options

resolve_execution_options(ctx)#

Returns the resolved execution options.

Subclasses can implement this method to define the execution options available for the operation.

Parameters:: ctx – the fiftyone.operators.executor.ExecutionContext
Returns:: a fiftyone.operators.executor.ExecutionOptions instance

resolve_output(ctx)#

Returns the resolved output property.

Subclasses can implement this method to define the outputs of the operator.

By default, this method is called once when the operator is created. If the operator is dynamic, this method is called after the operator is executed.

Parameters:: ctx – the fiftyone.operators.executor.ExecutionContext
Returns:: a fiftyone.operators.types.Property, or None

resolve_placement(ctx)#

Returns the resolved placement of the operator.

Subclasses can implement this method to define the placement of the operator.

Parameters:: ctx – the fiftyone.operators.executor.ExecutionContext
Returns:: a fiftyone.operators.types.Placement, or None

resolve_run_name(ctx)#

Returns the resolved run name of the operator.

Subclasses can implement this method to define the run name of the operator.

Parameters:: ctx – the fiftyone.operators.executor.ExecutionContext
Returns:: a string, or None

resolve_type(ctx, type)#

Returns the resolved input or output property.

Parameters:

ctx – the fiftyone.operators.executor.ExecutionContext
type – the type of property to resolve, either "inputs" or "outputs"

Returns:

a fiftyone.operators.types.Property, or None

to_json()#

Returns a JSON representation of the operator.

Returns:: a JSON dict

property uri#: The unique identifier of the operator: plugin_name/operator_name.