plugins.panels.model_evaluation¶

Module contents¶

Model evaluation panel.

Copyright 2017-2025, Voxel51, Inc.
voxel51.com

Classes:

EvaluationPanel([_builtin])

class plugins.panels.model_evaluation.EvaluationPanel(_builtin=False)¶

Bases: fiftyone.operators.panel.Panel

Attributes:

`config`	The `OperatorConfig` for the operator.
`builtin`	Whether the operator is builtin.
`name`
`uri`	The unique identifier of the operator: `plugin_name/operator_name`.

Methods:

`get_dataset_id`(ctx)
`get_store`(ctx)
`get_evaluation_id`(dataset, eval_key)
`get_permissions`(ctx)
`can_evaluate`(ctx)
`can_edit_note`(ctx)
`can_edit_status`(ctx)
`can_delete_evaluation`(ctx)
`can_rename`(ctx)
`can_delete_scenario`(ctx)
`can_edit_scenario`(ctx)
`on_load`(ctx)
`get_confidences`(per_class_metrics)
`get_avg_confidence`(confidences)
`get_confidence_distribution`(confidences)
`get_avg_iou`(per_class_metrics)
`get_tp_fp_fn`(info, results)
`get_map`(results)
`get_mar`(results)
`get_custom_metrics`(results)
`set_status`(ctx)
`set_note`(ctx)
`rename_evaluation`(ctx)
`delete_evaluation`(ctx)
`load_evaluation_view`(ctx)
`compute_avg_confs`(results)
`compute_avg_ious`(results)
`get_per_class_metrics`(info, results)
`get_confusion_matrix_colorscale`(matrix[, …])
`get_confusion_matrices`(results[, …])
`get_correct_incorrect`(results)
`get_scenarios`(ctx, eval_id)
`get_scenario`(ctx, eval_id, scenario_id)
`load_scenarios`(ctx)
`get_evaluation_data`(ctx)
`get_evaluation_data_cache_key_fn`(ctx)
`get_evaluation_data_cacheable`(ctx)
`load_evaluation`(ctx)
`on_change_view`(ctx)
`has_evaluation_results`(dataset, eval_key)
`load_pending_evaluations`(ctx[, skip_update])
`on_evaluate_model_success`(ctx)
`on_evaluate_model`(ctx)
`load_view`(ctx)
`load_compare_evaluation_results`(ctx)
`get_subset_def_data`(info, results, …)
`get_subset_def_data_for_eval_key`(ctx, scenario)	Builds and returns an execution cache key for each type of scenario.
`get_scenario_data`(ctx, scenario)
`validate_scenario_subsets`(ctx, scenario)	Validates the subsets in a scenario and compiles basic changes. 1. For view scenarios, check if the views exist in the dataset. 2. For sample_field and label_attribute scenarios, check if the field exists in the dataset. - If the field is tags, check if the tags exist.
`load_scenario`(ctx)	Tries to load the scenario given its id and evaluation key If it fails, it shows an error page with options to edit/delete.
`delete_scenario`(ctx)
`render`(ctx)	Defines the panel’s layout and events.
`add_secrets`(secrets)	Adds secrets to the operator.
`execute`(ctx)	Executes the operator.
`method_to_uri`(method_name)	Converts a method name to a URI.
`on_startup`(ctx)
`resolve_delegation`(ctx)	Returns the resolved forced delegation flag.
`resolve_execution_options`(ctx)	Returns the resolved execution options.
`resolve_input`(ctx)	Returns the resolved input property.
`resolve_output`(ctx)	Returns the resolved output property.
`resolve_placement`(ctx)	Returns the resolved placement of the operator.
`resolve_type`(ctx, type)	Returns the resolved input or output property.
`to_json`()	Returns a JSON representation of the operator.

property config¶: The OperatorConfig for the operator.

get_dataset_id(ctx)¶

get_store(ctx)¶

get_evaluation_id(dataset, eval_key)¶

get_permissions(ctx)¶

can_evaluate(ctx)¶

can_edit_note(ctx)¶

can_edit_status(ctx)¶

can_delete_evaluation(ctx)¶

can_rename(ctx)¶

can_delete_scenario(ctx)¶

can_edit_scenario(ctx)¶

on_load(ctx)¶

get_confidences(per_class_metrics)¶

get_avg_confidence(confidences)¶

get_confidence_distribution(confidences)¶

get_avg_iou(per_class_metrics)¶

get_tp_fp_fn(info, results)¶

get_map(results)¶

get_mar(results)¶

get_custom_metrics(results)¶

set_status(ctx)¶

set_note(ctx)¶

rename_evaluation(ctx)¶

delete_evaluation(ctx)¶

load_evaluation_view(ctx)¶

compute_avg_confs(results)¶

compute_avg_ious(results)¶

get_per_class_metrics(info, results)¶

get_confusion_matrix_colorscale(matrix, colorscale_name=None)¶

get_confusion_matrices(results, colorscale_name=None)¶

get_correct_incorrect(results)¶

get_scenarios(ctx, eval_id)¶

get_scenario(ctx, eval_id, scenario_id)¶

load_scenarios(ctx)¶

get_evaluation_data(ctx)¶

get_evaluation_data_cache_key_fn(ctx)¶

get_evaluation_data_cacheable(ctx)¶

load_evaluation(ctx)¶

on_change_view(ctx)¶

has_evaluation_results(dataset, eval_key)¶

load_pending_evaluations(ctx, skip_update=False)¶

on_evaluate_model_success(ctx)¶

on_evaluate_model(ctx)¶

load_view(ctx)¶

load_compare_evaluation_results(ctx)¶

get_subset_def_data(info, results, subset_def, is_compare)¶

get_subset_def_data_for_eval_key(ctx, scenario)¶: Builds and returns an execution cache key for each type of scenario.

get_scenario_data(ctx, scenario)¶

validate_scenario_subsets(ctx, scenario)¶

Validates the subsets in a scenario and compiles basic changes. 1. For view scenarios, check if the views exist in the dataset. 2. For sample_field and label_attribute scenarios, check if the field exists in the dataset.

If the field is tags, check if the tags exist.

returns the validated scenario and a list of changes. If the scenario is not valid, it will return an empty list of subsets. If the scenario is valid, it will return the original scenario with the validated subsets.

load_scenario(ctx)¶: Tries to load the scenario given its id and evaluation key If it fails, it shows an error page with options to edit/delete.

delete_scenario(ctx)¶

render(ctx)¶

Defines the panel’s layout and events.

This method is called after every panel event is called (on load, button callback, context change event, etc).

Parameters: ctx – the fiftyone.operators.executor.ExecutionContext
Returns: a fiftyone.operators.types.Property

add_secrets(secrets)¶

Adds secrets to the operator.

Parameters: secrets – a list of secrets

property builtin¶: Whether the operator is builtin.

execute(ctx)¶

Executes the operator.

Subclasses must implement this method.

Parameters: ctx – the fiftyone.operators.executor.ExecutionContext
Returns: JSON serializable data, or None

method_to_uri(method_name)¶

Converts a method name to a URI.

Parameters: method_name – the method name
Returns: a URI

property name¶

on_startup(ctx)¶

resolve_delegation(ctx)¶

Returns the resolved forced delegation flag.

Subclasses can implement this method to decide if delegated execution should be forced for the given operation.

Parameters: ctx – the fiftyone.operators.executor.ExecutionContext
Returns: whether the operation should be delegated (True), run immediately (False), or None to defer to resolve_execution_options() to specify the available options

resolve_execution_options(ctx)¶

Returns the resolved execution options.

Subclasses can implement this method to define the execution options available for the operation.

Parameters: ctx – the fiftyone.operators.executor.ExecutionContext
Returns: a fiftyone.operators.executor.ExecutionOptions instance

resolve_input(ctx)¶

Returns the resolved input property.

Subclasses can implement this method to define the inputs to the operator. This method should never be called directly. Instead use resolve_type().

By default, this method is called once when the operator is created. If the operator is dynamic, this method is called each time the input changes.

Parameters: ctx – the fiftyone.operators.executor.ExecutionContext
Returns: a fiftyone.operators.types.Property, or None

resolve_output(ctx)¶

Returns the resolved output property.

Subclasses can implement this method to define the outputs of the operator.

By default, this method is called once when the operator is created. If the operator is dynamic, this method is called after the operator is executed.