# plugins.panels.model_evaluation * [plugins.panels.model_evaluation.utils](plugins.panels.model_evaluation.utils.md) * [`compress_confusion_matrix()`](plugins.panels.model_evaluation.utils.md#plugins.panels.model_evaluation.utils.compress_confusion_matrix) * [`decompress_confusion_matrix()`](plugins.panels.model_evaluation.utils.md#plugins.panels.model_evaluation.utils.decompress_confusion_matrix) * [`compress_and_serialize()`](plugins.panels.model_evaluation.utils.md#plugins.panels.model_evaluation.utils.compress_and_serialize) * [`decompress_and_deserialize()`](plugins.panels.model_evaluation.utils.md#plugins.panels.model_evaluation.utils.decompress_and_deserialize) * [`compress_and_serialize_scenario()`](plugins.panels.model_evaluation.utils.md#plugins.panels.model_evaluation.utils.compress_and_serialize_scenario) * [`decompress_and_deserialize_scenario()`](plugins.panels.model_evaluation.utils.md#plugins.panels.model_evaluation.utils.decompress_and_deserialize_scenario) ## Module contents Model evaluation panel. Copyright 2017-2026, Voxel51, Inc.
[voxel51.com](https://voxel51.com/)

**Classes:** | [`EvaluationPanel`](#plugins.panels.model_evaluation.EvaluationPanel)([_builtin]) | | |-------------------------------------------------------------------------------------|----| ### *class* plugins.panels.model_evaluation.EvaluationPanel(\_builtin=False) Bases: [`Panel`](../../api/fiftyone.operators.panel.md#fiftyone.operators.panel.Panel) **Attributes:** | [`config`](#plugins.panels.model_evaluation.EvaluationPanel.config) | The `OperatorConfig` for the operator. | |-----------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------| | [`builtin`](#plugins.panels.model_evaluation.EvaluationPanel.builtin) | Whether the operator is builtin. | | [`name`](#plugins.panels.model_evaluation.EvaluationPanel.name) | | | [`risk_level`](#plugins.panels.model_evaluation.EvaluationPanel.risk_level) | The effective risk level of the operator, which is used by guardrail systems of an agent to classify tool calls. | | [`uri`](#plugins.panels.model_evaluation.EvaluationPanel.uri) | The unique identifier of the operator: `plugin_name/operator_name`. | **Methods:** | [`get_dataset`](#plugins.panels.model_evaluation.EvaluationPanel.get_dataset)(ctx) | | |----------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [`get_evaluation_id`](#plugins.panels.model_evaluation.EvaluationPanel.get_evaluation_id)(dataset, eval_key) | | | [`get_evaluation_type`](#plugins.panels.model_evaluation.EvaluationPanel.get_evaluation_type)(config) | | | [`get_permissions`](#plugins.panels.model_evaluation.EvaluationPanel.get_permissions)(ctx) | | | [`can_evaluate`](#plugins.panels.model_evaluation.EvaluationPanel.can_evaluate)(ctx) | | | [`can_edit_note`](#plugins.panels.model_evaluation.EvaluationPanel.can_edit_note)(ctx) | | | [`can_edit_status`](#plugins.panels.model_evaluation.EvaluationPanel.can_edit_status)(ctx) | | | [`can_delete_evaluation`](#plugins.panels.model_evaluation.EvaluationPanel.can_delete_evaluation)(ctx) | | | [`can_rename`](#plugins.panels.model_evaluation.EvaluationPanel.can_rename)(ctx) | | | [`can_delete_scenario`](#plugins.panels.model_evaluation.EvaluationPanel.can_delete_scenario)(ctx) | | | [`can_edit_scenario`](#plugins.panels.model_evaluation.EvaluationPanel.can_edit_scenario)(ctx) | | | [`on_load`](#plugins.panels.model_evaluation.EvaluationPanel.on_load)(ctx) | | | [`get_confidences`](#plugins.panels.model_evaluation.EvaluationPanel.get_confidences)(per_class_metrics) | | | [`get_avg_confidence`](#plugins.panels.model_evaluation.EvaluationPanel.get_avg_confidence)(confidences) | | | [`get_confidence_distribution`](#plugins.panels.model_evaluation.EvaluationPanel.get_confidence_distribution)(confidences) | | | [`get_avg_iou`](#plugins.panels.model_evaluation.EvaluationPanel.get_avg_iou)(per_class_metrics) | | | [`get_tp_fp_fn`](#plugins.panels.model_evaluation.EvaluationPanel.get_tp_fp_fn)(info, results) | | | [`get_map`](#plugins.panels.model_evaluation.EvaluationPanel.get_map)(results) | | | [`get_mar`](#plugins.panels.model_evaluation.EvaluationPanel.get_mar)(results) | | | [`get_custom_metrics`](#plugins.panels.model_evaluation.EvaluationPanel.get_custom_metrics)(results) | | | [`set_status`](#plugins.panels.model_evaluation.EvaluationPanel.set_status)(ctx) | | | [`set_note`](#plugins.panels.model_evaluation.EvaluationPanel.set_note)(ctx) | | | [`rename_evaluation`](#plugins.panels.model_evaluation.EvaluationPanel.rename_evaluation)(ctx) | | | [`delete_evaluation`](#plugins.panels.model_evaluation.EvaluationPanel.delete_evaluation)(ctx) | | | [`load_evaluation_view`](#plugins.panels.model_evaluation.EvaluationPanel.load_evaluation_view)(ctx) | | | [`compute_avg_confs`](#plugins.panels.model_evaluation.EvaluationPanel.compute_avg_confs)(results) | | | [`compute_avg_ious`](#plugins.panels.model_evaluation.EvaluationPanel.compute_avg_ious)(results) | | | [`get_per_class_metrics`](#plugins.panels.model_evaluation.EvaluationPanel.get_per_class_metrics)(info, results) | | | [`get_confusion_matrix_colorscale`](#plugins.panels.model_evaluation.EvaluationPanel.get_confusion_matrix_colorscale)(matrix, ...) | | | [`get_confusion_matrix`](#plugins.panels.model_evaluation.EvaluationPanel.get_confusion_matrix)(results) | | | [`get_correct_incorrect`](#plugins.panels.model_evaluation.EvaluationPanel.get_correct_incorrect)(results) | | | [`get_scenario`](#plugins.panels.model_evaluation.EvaluationPanel.get_scenario)(ctx, scenario_id) | | | [`load_scenarios`](#plugins.panels.model_evaluation.EvaluationPanel.load_scenarios)(ctx) | | | [`get_evaluation_data`](#plugins.panels.model_evaluation.EvaluationPanel.get_evaluation_data)(ctx) | | | [`get_evaluation_data_cache_key_fn`](#plugins.panels.model_evaluation.EvaluationPanel.get_evaluation_data_cache_key_fn)(ctx) | | | [`get_evaluation_data_cacheable`](#plugins.panels.model_evaluation.EvaluationPanel.get_evaluation_data_cacheable)(ctx) | | | [`load_evaluation`](#plugins.panels.model_evaluation.EvaluationPanel.load_evaluation)(ctx) | | | [`on_change_view`](#plugins.panels.model_evaluation.EvaluationPanel.on_change_view)(ctx) | | | [`has_evaluation_results`](#plugins.panels.model_evaluation.EvaluationPanel.has_evaluation_results)(dataset, eval_key) | | | [`load_pending_evaluations`](#plugins.panels.model_evaluation.EvaluationPanel.load_pending_evaluations)(ctx[, skip_update]) | | | [`on_evaluate_model_success`](#plugins.panels.model_evaluation.EvaluationPanel.on_evaluate_model_success)(ctx) | | | [`on_evaluate_model`](#plugins.panels.model_evaluation.EvaluationPanel.on_evaluate_model)(ctx) | | | [`load_view`](#plugins.panels.model_evaluation.EvaluationPanel.load_view)(ctx) | | | [`load_compare_evaluation_results`](#plugins.panels.model_evaluation.EvaluationPanel.load_compare_evaluation_results)(ctx) | | | [`get_subset_def_data`](#plugins.panels.model_evaluation.EvaluationPanel.get_subset_def_data)(info, results, ...) | | | [`get_scenario_data`](#plugins.panels.model_evaluation.EvaluationPanel.get_scenario_data)(ctx, scenario) | | | [`get_subset_def_data_for_eval_key`](#plugins.panels.model_evaluation.EvaluationPanel.get_subset_def_data_for_eval_key)(ctx, scenario) | Builds and returns an execution cache key for each type of scenario. | | [`get_scenario_data_cacheable`](#plugins.panels.model_evaluation.EvaluationPanel.get_scenario_data_cacheable)(ctx, scenario) | | | [`validate_scenario_subsets`](#plugins.panels.model_evaluation.EvaluationPanel.validate_scenario_subsets)(ctx, scenario) | Validates the subsets in a scenario and compiles basic changes. 1. For view scenarios, check if the views exist in the dataset. 2. For sample_field and label_attribute scenarios, check if the field exists in the dataset. - If the field is tags, check if the tags exist. | | [`load_scenario`](#plugins.panels.model_evaluation.EvaluationPanel.load_scenario)(ctx) | Tries to load the scenario given its id and evaluation key If it fails, it shows an error page with options to edit/delete. | | [`delete_scenario`](#plugins.panels.model_evaluation.EvaluationPanel.delete_scenario)(ctx) | | | [`render`](#plugins.panels.model_evaluation.EvaluationPanel.render)(ctx) | Defines the panel's layout and events. | | [`add_secrets`](#plugins.panels.model_evaluation.EvaluationPanel.add_secrets)(secrets) | Adds secrets to the operator. | | [`execute`](#plugins.panels.model_evaluation.EvaluationPanel.execute)(ctx) | Executes the operator. | | [`method_to_uri`](#plugins.panels.model_evaluation.EvaluationPanel.method_to_uri)(method_name) | Converts a method name to a URI. | | [`on_startup`](#plugins.panels.model_evaluation.EvaluationPanel.on_startup)(ctx) | | | [`resolve_delegation`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_delegation)(ctx) | Returns the resolved *forced* delegation flag. | | [`resolve_execution_options`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_execution_options)(ctx) | Returns the resolved execution options. | | [`resolve_input`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_input)(ctx) | Returns the resolved input property. | | [`resolve_output`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_output)(ctx) | Returns the resolved output property. | | [`resolve_panel_config`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_panel_config)() | Returns the registration config for this panel. | | [`resolve_placement`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_placement)(ctx) | Returns the resolved placement of the operator. | | [`resolve_run_name`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_run_name)(ctx) | Returns the resolved run name of the operator. | | [`resolve_type`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_type)(ctx, type) | Returns the resolved input or output property. | | [`to_json`](#plugins.panels.model_evaluation.EvaluationPanel.to_json)() | Returns a JSON representation of the operator. | #### *property* config The `OperatorConfig` for the operator. #### get_dataset(ctx) #### get_evaluation_id(dataset, eval_key) #### get_evaluation_type(config) #### get_permissions(ctx) #### can_evaluate(ctx) #### can_edit_note(ctx) #### can_edit_status(ctx) #### can_delete_evaluation(ctx) #### can_rename(ctx) #### can_delete_scenario(ctx) #### can_edit_scenario(ctx) #### on_load(ctx) #### get_confidences(per_class_metrics) #### get_avg_confidence(confidences) #### get_confidence_distribution(confidences) #### get_avg_iou(per_class_metrics) #### get_tp_fp_fn(info, results) #### get_map(results) #### get_mar(results) #### get_custom_metrics(results) #### set_status(ctx) #### set_note(ctx) #### rename_evaluation(ctx) #### delete_evaluation(ctx) #### load_evaluation_view(ctx) #### compute_avg_confs(results) #### compute_avg_ious(results) #### get_per_class_metrics(info, results) #### get_confusion_matrix_colorscale(matrix, colorscale_name, , logarithmic=False) #### get_confusion_matrix(results) #### get_correct_incorrect(results) #### get_scenario(ctx, scenario_id) #### load_scenarios(ctx) #### get_evaluation_data(ctx) #### get_evaluation_data_cache_key_fn(ctx) #### get_evaluation_data_cacheable(ctx) #### load_evaluation(ctx) #### on_change_view(ctx) #### has_evaluation_results(dataset, eval_key) #### load_pending_evaluations(ctx, skip_update=False) #### on_evaluate_model_success(ctx) #### on_evaluate_model(ctx) #### load_view(ctx) #### load_compare_evaluation_results(ctx) #### get_subset_def_data(info, results, subset_def, is_compare) #### get_scenario_data(ctx, scenario) #### get_subset_def_data_for_eval_key(ctx, scenario) Builds and returns an execution cache key for each type of scenario. #### get_scenario_data_cacheable(ctx, scenario) #### validate_scenario_subsets(ctx, scenario) Validates the subsets in a scenario and compiles basic changes. 1. For view scenarios, check if the views exist in the dataset. 2. For sample_field and label_attribute scenarios, check if the field exists in the dataset. > - If the field is tags, check if the tags exist. returns the validated scenario and a list of changes. If the scenario is not valid, it will return an empty list of subsets. If the scenario is valid, it will return the original scenario with the validated subsets. #### load_scenario(ctx) Tries to load the scenario given its id and evaluation key If it fails, it shows an error page with options to edit/delete. #### delete_scenario(ctx) #### render(ctx) Defines the panel’s layout and events. This method is called after every panel event is called (on load, button callback, context change event, etc). * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property) #### add_secrets(secrets) Adds secrets to the operator. * **Parameters:** **secrets** – a list of secrets #### *property* builtin Whether the operator is builtin. #### execute(ctx) Executes the operator. Subclasses must implement this method. * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** JSON serializable data, or None #### method_to_uri(method_name) Converts a method name to a URI. * **Parameters:** **method_name** – the method name * **Returns:** a URI #### *property* name #### on_startup(ctx) #### resolve_delegation(ctx) → [bool](../../api/fiftyone.core.stages.md#fiftyone.core.stages.Exists.bool) | None Returns the resolved *forced* delegation flag. Subclasses can implement this method to decide if delegated execution should be *forced* for the given operation. * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** whether the operation should be delegated (True), run immediately (False), or None to defer to [`resolve_execution_options()`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_execution_options) to specify the available options #### resolve_execution_options(ctx) Returns the resolved execution options. Subclasses can implement this method to define the execution options available for the operation. * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** a [`fiftyone.operators.executor.ExecutionOptions`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionOptions) instance #### resolve_input(ctx) Returns the resolved input property. Subclasses can implement this method to define the inputs to the operator. This method should never be called directly. Instead use [`resolve_type()`](#plugins.panels.model_evaluation.EvaluationPanel.resolve_type). By default, this method is called once when the operator is created. If the operator is dynamic, this method is called each time the input changes. * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None #### resolve_output(ctx) Returns the resolved output property. Subclasses can implement this method to define the outputs of the operator. By default, this method is called once when the operator is created. If the operator is dynamic, this method is called after the operator is executed. * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None #### resolve_panel_config() Returns the registration config for this panel. #### resolve_placement(ctx) Returns the resolved placement of the operator. Subclasses can implement this method to define the placement of the operator. * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** a [`fiftyone.operators.types.Placement`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Placement), or None #### resolve_run_name(ctx) Returns the resolved run name of the operator. Subclasses can implement this method to define the run name of the operator. * **Parameters:** **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **Returns:** a string, or None #### resolve_type(ctx, type) Returns the resolved input or output property. * **Parameters:** * **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext) * **type** – the type of property to resolve, either `"inputs"` or `"outputs"` * **Returns:** a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None #### *property* risk_level The effective risk level of the operator, which is used by guardrail systems of an agent to classify tool calls. #### to_json() Returns a JSON representation of the operator. * **Returns:** a JSON dict #### *property* uri The unique identifier of the operator: `plugin_name/operator_name`.