<!-- # hard line break macro for HTML -->

# plugins.operators.model_evaluation

* [plugins.operators.model_evaluation.utils](plugins.operators.model_evaluation.utils.md)
  * [`CustomCodeViewReason`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason)
    * [`CustomCodeViewReason.TOO_MANY_CATEGORIES`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.TOO_MANY_CATEGORIES)
    * [`CustomCodeViewReason.TOO_MANY_INT_CATEGORIES`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.TOO_MANY_INT_CATEGORIES)
    * [`CustomCodeViewReason.FLOAT_TYPE`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.FLOAT_TYPE)
    * [`CustomCodeViewReason.SLOW`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.SLOW)
    * [`CustomCodeViewReason.encode()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.encode)
    * [`CustomCodeViewReason.replace()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.replace)
    * [`CustomCodeViewReason.split()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.split)
    * [`CustomCodeViewReason.rsplit()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.rsplit)
    * [`CustomCodeViewReason.join()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.join)
    * [`CustomCodeViewReason.capitalize()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.capitalize)
    * [`CustomCodeViewReason.casefold()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.casefold)
    * [`CustomCodeViewReason.title()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.title)
    * [`CustomCodeViewReason.center()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.center)
    * [`CustomCodeViewReason.count()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.count)
    * [`CustomCodeViewReason.expandtabs()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.expandtabs)
    * [`CustomCodeViewReason.find()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.find)
    * [`CustomCodeViewReason.partition()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.partition)
    * [`CustomCodeViewReason.index()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.index)
    * [`CustomCodeViewReason.ljust()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.ljust)
    * [`CustomCodeViewReason.lower()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.lower)
    * [`CustomCodeViewReason.lstrip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.lstrip)
    * [`CustomCodeViewReason.rfind()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.rfind)
    * [`CustomCodeViewReason.rindex()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.rindex)
    * [`CustomCodeViewReason.rjust()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.rjust)
    * [`CustomCodeViewReason.rstrip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.rstrip)
    * [`CustomCodeViewReason.rpartition()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.rpartition)
    * [`CustomCodeViewReason.splitlines()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.splitlines)
    * [`CustomCodeViewReason.strip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.strip)
    * [`CustomCodeViewReason.swapcase()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.swapcase)
    * [`CustomCodeViewReason.translate()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.translate)
    * [`CustomCodeViewReason.upper()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.upper)
    * [`CustomCodeViewReason.startswith()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.startswith)
    * [`CustomCodeViewReason.endswith()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.endswith)
    * [`CustomCodeViewReason.removeprefix()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.removeprefix)
    * [`CustomCodeViewReason.removesuffix()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.removesuffix)
    * [`CustomCodeViewReason.isascii()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isascii)
    * [`CustomCodeViewReason.islower()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.islower)
    * [`CustomCodeViewReason.isupper()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isupper)
    * [`CustomCodeViewReason.istitle()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.istitle)
    * [`CustomCodeViewReason.isspace()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isspace)
    * [`CustomCodeViewReason.isdecimal()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isdecimal)
    * [`CustomCodeViewReason.isdigit()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isdigit)
    * [`CustomCodeViewReason.isnumeric()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isnumeric)
    * [`CustomCodeViewReason.isalpha()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isalpha)
    * [`CustomCodeViewReason.isalnum()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isalnum)
    * [`CustomCodeViewReason.isidentifier()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isidentifier)
    * [`CustomCodeViewReason.isprintable()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.isprintable)
    * [`CustomCodeViewReason.zfill()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.zfill)
    * [`CustomCodeViewReason.format()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.format)
    * [`CustomCodeViewReason.format_map()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.format_map)
    * [`CustomCodeViewReason.maketrans()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.CustomCodeViewReason.maketrans)
  * [`ShowOptionsMethod`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod)
    * [`ShowOptionsMethod.CODE`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.CODE)
    * [`ShowOptionsMethod.CHECKBOX`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.CHECKBOX)
    * [`ShowOptionsMethod.EMPTY`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.EMPTY)
    * [`ShowOptionsMethod.AUTOCOMPLETE`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.AUTOCOMPLETE)
    * [`ShowOptionsMethod.encode()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.encode)
    * [`ShowOptionsMethod.replace()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.replace)
    * [`ShowOptionsMethod.split()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.split)
    * [`ShowOptionsMethod.rsplit()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.rsplit)
    * [`ShowOptionsMethod.join()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.join)
    * [`ShowOptionsMethod.capitalize()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.capitalize)
    * [`ShowOptionsMethod.casefold()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.casefold)
    * [`ShowOptionsMethod.title()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.title)
    * [`ShowOptionsMethod.center()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.center)
    * [`ShowOptionsMethod.count()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.count)
    * [`ShowOptionsMethod.expandtabs()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.expandtabs)
    * [`ShowOptionsMethod.find()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.find)
    * [`ShowOptionsMethod.partition()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.partition)
    * [`ShowOptionsMethod.index()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.index)
    * [`ShowOptionsMethod.ljust()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.ljust)
    * [`ShowOptionsMethod.lower()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.lower)
    * [`ShowOptionsMethod.lstrip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.lstrip)
    * [`ShowOptionsMethod.rfind()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.rfind)
    * [`ShowOptionsMethod.rindex()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.rindex)
    * [`ShowOptionsMethod.rjust()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.rjust)
    * [`ShowOptionsMethod.rstrip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.rstrip)
    * [`ShowOptionsMethod.rpartition()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.rpartition)
    * [`ShowOptionsMethod.splitlines()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.splitlines)
    * [`ShowOptionsMethod.strip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.strip)
    * [`ShowOptionsMethod.swapcase()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.swapcase)
    * [`ShowOptionsMethod.translate()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.translate)
    * [`ShowOptionsMethod.upper()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.upper)
    * [`ShowOptionsMethod.startswith()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.startswith)
    * [`ShowOptionsMethod.endswith()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.endswith)
    * [`ShowOptionsMethod.removeprefix()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.removeprefix)
    * [`ShowOptionsMethod.removesuffix()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.removesuffix)
    * [`ShowOptionsMethod.isascii()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isascii)
    * [`ShowOptionsMethod.islower()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.islower)
    * [`ShowOptionsMethod.isupper()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isupper)
    * [`ShowOptionsMethod.istitle()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.istitle)
    * [`ShowOptionsMethod.isspace()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isspace)
    * [`ShowOptionsMethod.isdecimal()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isdecimal)
    * [`ShowOptionsMethod.isdigit()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isdigit)
    * [`ShowOptionsMethod.isnumeric()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isnumeric)
    * [`ShowOptionsMethod.isalpha()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isalpha)
    * [`ShowOptionsMethod.isalnum()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isalnum)
    * [`ShowOptionsMethod.isidentifier()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isidentifier)
    * [`ShowOptionsMethod.isprintable()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.isprintable)
    * [`ShowOptionsMethod.zfill()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.zfill)
    * [`ShowOptionsMethod.format()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.format)
    * [`ShowOptionsMethod.format_map()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.format_map)
    * [`ShowOptionsMethod.maketrans()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ShowOptionsMethod.maketrans)
  * [`ScenarioType`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType)
    * [`ScenarioType.VIEW`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.VIEW)
    * [`ScenarioType.CUSTOM_CODE`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.CUSTOM_CODE)
    * [`ScenarioType.LABEL_ATTRIBUTE`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.LABEL_ATTRIBUTE)
    * [`ScenarioType.SAMPLE_FIELD`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.SAMPLE_FIELD)
    * [`ScenarioType.encode()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.encode)
    * [`ScenarioType.replace()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.replace)
    * [`ScenarioType.split()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.split)
    * [`ScenarioType.rsplit()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.rsplit)
    * [`ScenarioType.join()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.join)
    * [`ScenarioType.capitalize()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.capitalize)
    * [`ScenarioType.casefold()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.casefold)
    * [`ScenarioType.title()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.title)
    * [`ScenarioType.center()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.center)
    * [`ScenarioType.count()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.count)
    * [`ScenarioType.expandtabs()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.expandtabs)
    * [`ScenarioType.find()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.find)
    * [`ScenarioType.partition()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.partition)
    * [`ScenarioType.index()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.index)
    * [`ScenarioType.ljust()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.ljust)
    * [`ScenarioType.lower()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.lower)
    * [`ScenarioType.lstrip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.lstrip)
    * [`ScenarioType.rfind()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.rfind)
    * [`ScenarioType.rindex()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.rindex)
    * [`ScenarioType.rjust()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.rjust)
    * [`ScenarioType.rstrip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.rstrip)
    * [`ScenarioType.rpartition()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.rpartition)
    * [`ScenarioType.splitlines()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.splitlines)
    * [`ScenarioType.strip()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.strip)
    * [`ScenarioType.swapcase()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.swapcase)
    * [`ScenarioType.translate()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.translate)
    * [`ScenarioType.upper()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.upper)
    * [`ScenarioType.startswith()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.startswith)
    * [`ScenarioType.endswith()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.endswith)
    * [`ScenarioType.removeprefix()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.removeprefix)
    * [`ScenarioType.removesuffix()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.removesuffix)
    * [`ScenarioType.isascii()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isascii)
    * [`ScenarioType.islower()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.islower)
    * [`ScenarioType.isupper()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isupper)
    * [`ScenarioType.istitle()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.istitle)
    * [`ScenarioType.isspace()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isspace)
    * [`ScenarioType.isdecimal()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isdecimal)
    * [`ScenarioType.isdigit()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isdigit)
    * [`ScenarioType.isnumeric()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isnumeric)
    * [`ScenarioType.isalpha()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isalpha)
    * [`ScenarioType.isalnum()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isalnum)
    * [`ScenarioType.isidentifier()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isidentifier)
    * [`ScenarioType.isprintable()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.isprintable)
    * [`ScenarioType.zfill()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.zfill)
    * [`ScenarioType.format()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.format)
    * [`ScenarioType.format_map()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.format_map)
    * [`ScenarioType.maketrans()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.ScenarioType.maketrans)
  * [`get_scenario_example()`](plugins.operators.model_evaluation.utils.md#plugins.operators.model_evaluation.utils.get_scenario_example)

## Module contents

Scenario plugin.

Copyright 2017-2026, Voxel51, Inc.
<br/>
[voxel51.com](https://voxel51.com/)
<br/>
<br/>

**Classes:**

| [`ConfigureScenario`](#plugins.operators.model_evaluation.ConfigureScenario)([_builtin])                         |    |
|------------------------------------------------------------------------------------------------------------------|----|
| [`ConfigureScenarioPlotResolver`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver)([_builtin]) |    |
| [`ListScenarios`](#plugins.operators.model_evaluation.ListScenarios)([_builtin])                                 |    |

**Functions:**

| [`get_dataset`](#plugins.operators.model_evaluation.get_dataset)(ctx)                                   | Returns the dataset for the current context.                                                  |
|---------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|
| [`extract_evaluation_keys`](#plugins.operators.model_evaluation.extract_evaluation_keys)(ctx)           |                                                                                               |
| [`get_evaluations_results`](#plugins.operators.model_evaluation.get_evaluations_results)(ctx)           | Returns the evaluation results for the current context.                                       |
| [`get_label_attribute_path`](#plugins.operators.model_evaluation.get_label_attribute_path)(params)      | Returns the last part of the label attribute path to use in use_subset(type="attribute", ...) |
| [`get_scenario_type`](#plugins.operators.model_evaluation.get_scenario_type)(params)                    |                                                                                               |
| [`resolve_subsets`](#plugins.operators.model_evaluation.resolve_subsets)(scenario_type, params, values) |                                                                                               |

### *class* plugins.operators.model_evaluation.ConfigureScenario(\_builtin=False)

Bases: [`Operator`](../../api/fiftyone.operators.operator.md#fiftyone.operators.operator.Operator)

**Attributes:**

| [`last_view_type_used`](#plugins.operators.model_evaluation.ConfigureScenario.last_view_type_used)   |                                                                                                                  |
|------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------|
| [`config`](#plugins.operators.model_evaluation.ConfigureScenario.config)                             | The `OperatorConfig` for the operator.                                                                           |
| [`builtin`](#plugins.operators.model_evaluation.ConfigureScenario.builtin)                           | Whether the operator is builtin.                                                                                 |
| [`name`](#plugins.operators.model_evaluation.ConfigureScenario.name)                                 |                                                                                                                  |
| [`risk_level`](#plugins.operators.model_evaluation.ConfigureScenario.risk_level)                     | The effective risk level of the operator, which is used by guardrail systems of an agent to classify tool calls. |
| [`uri`](#plugins.operators.model_evaluation.ConfigureScenario.uri)                                   | The unique identifier of the operator: `plugin_name/operator_name`.                                              |

**Methods:**

| [`get_samples_count`](#plugins.operators.model_evaluation.ConfigureScenario.get_samples_count)(ctx)                                           | Returns the number of samples in the dataset for the current context.                                                                                                              |
|-----------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`get_default_for_distribution_preview`](#plugins.operators.model_evaluation.ConfigureScenario.get_default_for_distribution_preview)(ctx)     | Returns the default value for the sample distribution preview.                                                                                                                     |
| [`render_name_input`](#plugins.operators.model_evaluation.ConfigureScenario.render_name_input)(ctx, inputs)                                   |                                                                                                                                                                                    |
| [`render_scenario_types`](#plugins.operators.model_evaluation.ConfigureScenario.render_scenario_types)(inputs, selected_type)                 |                                                                                                                                                                                    |
| [`extract_evaluation_id`](#plugins.operators.model_evaluation.ConfigureScenario.extract_evaluation_id)(ctx)                                   |                                                                                                                                                                                    |
| [`convert_to_plotly_data`](#plugins.operators.model_evaluation.ConfigureScenario.convert_to_plotly_data)(preview_data)                        |                                                                                                                                                                                    |
| [`render_empty_sample_distribution`](#plugins.operators.model_evaluation.ConfigureScenario.render_empty_sample_distribution)(ctx, ...[, ...]) |                                                                                                                                                                                    |
| [`get_label_attribute_path`](#plugins.operators.model_evaluation.ConfigureScenario.get_label_attribute_path)(params)                          | Returns the last part of the label attribute path to use in use_subset(type="attribute", ...)                                                                                      |
| [`render_sample_distribution`](#plugins.operators.model_evaluation.ConfigureScenario.render_sample_distribution)(ctx, inputs, ...)            |                                                                                                                                                                                    |
| [`render_plot_preview_toggle`](#plugins.operators.model_evaluation.ConfigureScenario.render_plot_preview_toggle)(ctx, inputs)                 |                                                                                                                                                                                    |
| [`render_sample_distribution_graph`](#plugins.operators.model_evaluation.ConfigureScenario.render_sample_distribution_graph)(ctx, ...)        |                                                                                                                                                                                    |
| [`get_custom_code_key`](#plugins.operators.model_evaluation.ConfigureScenario.get_custom_code_key)(params)                                    |                                                                                                                                                                                    |
| [`extract_custom_code`](#plugins.operators.model_evaluation.ConfigureScenario.extract_custom_code)(ctx[, example_type])                       |                                                                                                                                                                                    |
| [`render_custom_code`](#plugins.operators.model_evaluation.ConfigureScenario.render_custom_code)(ctx, inputs[, example_type])                 |                                                                                                                                                                                    |
| [`render_no_values_warning`](#plugins.operators.model_evaluation.ConfigureScenario.render_no_values_warning)(inputs, field_name)              |                                                                                                                                                                                    |
| [`render_use_custom_code_warning`](#plugins.operators.model_evaluation.ConfigureScenario.render_use_custom_code_warning)(inputs[, reason])    |                                                                                                                                                                                    |
| [`render_use_custom_code_instead`](#plugins.operators.model_evaluation.ConfigureScenario.render_use_custom_code_instead)(ctx, inputs)         |                                                                                                                                                                                    |
| [`get_saved_view_scenarios_picker_type`](#plugins.operators.model_evaluation.ConfigureScenario.get_saved_view_scenarios_picker_type)(ctx)     | Returns the view mode for saved views based on the number of available saved views.                                                                                                |
| [`render_saved_views`](#plugins.operators.model_evaluation.ConfigureScenario.render_saved_views)(ctx, inputs)                                 | Renders sample distribution for subsets using saved views.                                                                                                                         |
| [`get_scenario_values_key`](#plugins.operators.model_evaluation.ConfigureScenario.get_scenario_values_key)(params)                            | returns a unique key that holds the latest selected values for a - certain scenario type.                                                                                          |
| [`get_selected_values`](#plugins.operators.model_evaluation.ConfigureScenario.get_selected_values)(params)                                    |                                                                                                                                                                                    |
| [`render_checkbox_view`](#plugins.operators.model_evaluation.ConfigureScenario.render_checkbox_view)(ctx, values, inputs[, ...])              |                                                                                                                                                                                    |
| [`get_scenarios_picker_type`](#plugins.operators.model_evaluation.ConfigureScenario.get_scenarios_picker_type)(ctx, field_name)               | Determines the scenario picker type for a given field based on its type and distinct values.                                                                                       |
| [`render_auto_complete_view`](#plugins.operators.model_evaluation.ConfigureScenario.render_auto_complete_view)(ctx, values, inputs)           | Renders an auto-complete view as a more compact/efficient input option when, - There are too many distinct values to display and The field is of type string.                      |
| [`render_scenario_picker_view`](#plugins.operators.model_evaluation.ConfigureScenario.render_scenario_picker_view)(ctx, field_name, ...)      | Renders the appropriate UI component based on the picker type.                                                                                                                     |
| [`get_valid_label_attribute_path_options`](#plugins.operators.model_evaluation.ConfigureScenario.get_valid_label_attribute_path_options)(...) | Returns all the paths in the dataset schema that are valid label attributes types.                                                                                                 |
| [`render_label_attribute`](#plugins.operators.model_evaluation.ConfigureScenario.render_label_attribute)(ctx, inputs, gt_field)               |                                                                                                                                                                                    |
| [`get_valid_sample_field_path_options`](#plugins.operators.model_evaluation.ConfigureScenario.get_valid_sample_field_path_options)(...)       | Get valid sample field path options based on the schema. - Filters out     - fields having roots with type as List of documents     - non-primitives     - non-list of primitives. |
| [`render_sample_fields`](#plugins.operators.model_evaluation.ConfigureScenario.render_sample_fields)(ctx, inputs[, field_name])               |                                                                                                                                                                                    |
| [`get_modal_title`](#plugins.operators.model_evaluation.ConfigureScenario.get_modal_title)(ctx)                                               |                                                                                                                                                                                    |
| [`get_scenario_names`](#plugins.operators.model_evaluation.ConfigureScenario.get_scenario_names)(ctx)                                         |                                                                                                                                                                                    |
| [`resolve_input`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_input)(ctx)                                                   | Returns the resolved input property.                                                                                                                                               |
| [`render_custom_code_content`](#plugins.operators.model_evaluation.ConfigureScenario.render_custom_code_content)(inputs, ...)                 |                                                                                                                                                                                    |
| [`execute`](#plugins.operators.model_evaluation.ConfigureScenario.execute)(ctx)                                                               | Executes the operator.                                                                                                                                                             |
| [`add_secrets`](#plugins.operators.model_evaluation.ConfigureScenario.add_secrets)(secrets)                                                   | Adds secrets to the operator.                                                                                                                                                      |
| [`method_to_uri`](#plugins.operators.model_evaluation.ConfigureScenario.method_to_uri)(method_name)                                           | Converts a method name to a URI.                                                                                                                                                   |
| [`resolve_delegation`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_delegation)(ctx)                                         | Returns the resolved *forced* delegation flag.                                                                                                                                     |
| [`resolve_execution_options`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_execution_options)(ctx)                           | Returns the resolved execution options.                                                                                                                                            |
| [`resolve_output`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_output)(ctx)                                                 | Returns the resolved output property.                                                                                                                                              |
| [`resolve_placement`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_placement)(ctx)                                           | Returns the resolved placement of the operator.                                                                                                                                    |
| [`resolve_run_name`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_run_name)(ctx)                                             | Returns the resolved run name of the operator.                                                                                                                                     |
| [`resolve_type`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_type)(ctx, type)                                               | Returns the resolved input or output property.                                                                                                                                     |
| [`to_json`](#plugins.operators.model_evaluation.ConfigureScenario.to_json)()                                                                  | Returns a JSON representation of the operator.                                                                                                                                     |

#### last_view_type_used *= None*

#### *property* config

The `OperatorConfig` for the operator.

#### get_samples_count(ctx)

Returns the number of samples in the dataset for the current context.

#### get_default_for_distribution_preview(ctx)

Returns the default value for the sample distribution preview.

#### render_name_input(ctx, inputs)

#### render_scenario_types(inputs, selected_type)

#### extract_evaluation_id(ctx)

#### convert_to_plotly_data(preview_data)

#### render_empty_sample_distribution(ctx, inputs, params, description=None, is_invalid=True)

#### get_label_attribute_path(params)

Returns the last part of the label attribute path to use in use_subset(type=”attribute”, …)

#### render_sample_distribution(ctx, inputs, scenario_type, values)

#### render_plot_preview_toggle(ctx, inputs)

#### render_sample_distribution_graph(ctx, inputs, subset_expressions, scenario_type)

#### get_custom_code_key(params)

#### extract_custom_code(ctx, example_type=ScenarioType.CUSTOM_CODE)

#### render_custom_code(ctx, inputs, example_type=ScenarioType.CUSTOM_CODE)

#### render_no_values_warning(inputs, field_name, link='')

#### render_use_custom_code_warning(inputs, reason=CustomCodeViewReason.TOO_MANY_CATEGORIES)

#### render_use_custom_code_instead(ctx, inputs, reason=CustomCodeViewReason.TOO_MANY_CATEGORIES)

#### get_saved_view_scenarios_picker_type(ctx)

Returns the view mode for saved views based on the number of available saved views.

#### render_saved_views(ctx, inputs)

Renders sample distribution for subsets using saved views.

#### get_scenario_values_key(params)

returns a unique key that holds the latest selected values for a
- certain scenario type. has to be one of type ScenarioType
- certain scenario field (ex: “tags”, “labels”)

#### get_selected_values(params)

#### render_checkbox_view(ctx, values, inputs, with_description=None)

#### get_scenarios_picker_type(ctx, field_name)

Determines the scenario picker type for a given field based on its type and distinct values.
- if the field is of type float, shows custom code mode
- if the field is boolean, shows checkbox mode
- if the field has no distinct values, shows empty mode
- if the field has too many distinct values, shows auto-complete or custom code mode
- if the field has a manageable number of distinct values, shows checkbox mode

* **Returns:**
  Picker type and corresponding values.
* **Return type:**
  [Tuple](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Tuple)[str, Any]

#### render_auto_complete_view(ctx, values, inputs)

Renders an auto-complete view as a more compact/efficient input option when,
- There are too many distinct values to display and The field is of type string.
- There are too many distinct saved views to display as checkbox.

Eventually, if there are selected values, it attempts to render the sample distribution preview graph

#### render_scenario_picker_view(ctx, field_name, inputs)

Renders the appropriate UI component based on the picker type.

#### get_valid_label_attribute_path_options(schema, gt_field)

Returns all the paths in the dataset schema that are valid label attributes types.
- primitives
- OR list of primitives
- AND path starts with gt_field

#### render_label_attribute(ctx, inputs, gt_field, label_attr=None)

#### get_valid_sample_field_path_options(flat_field_schema)

Get valid sample field path options based on the schema.
- Filters out

> - fields having roots with type as List of documents
> - non-primitives
> - non-list of primitives

#### render_sample_fields(ctx, inputs, field_name=None)

#### get_modal_title(ctx)

#### get_scenario_names(ctx)

#### resolve_input(ctx)

Returns the resolved input property.

Subclasses can implement this method to define the inputs to the
operator. This method should never be called directly. Instead
use [`resolve_type()`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_type).

By default, this method is called once when the operator is created.
If the operator is dynamic, this method is called each time the input
changes.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### render_custom_code_content(inputs, custom_code, code_key)

#### execute(ctx)

Executes the operator.

Subclasses must implement this method.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  JSON serializable data, or None

#### add_secrets(secrets)

Adds secrets to the operator.

* **Parameters:**
  **secrets** – a list of secrets

#### *property* builtin

Whether the operator is builtin.

#### method_to_uri(method_name)

Converts a method name to a URI.

* **Parameters:**
  **method_name** – the method name
* **Returns:**
  a URI

#### *property* name

#### resolve_delegation(ctx) → [bool](../../api/fiftyone.core.stages.md#fiftyone.core.stages.Exists.bool) | None

Returns the resolved *forced* delegation flag.

Subclasses can implement this method to decide if delegated execution
should be *forced* for the given operation.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  whether the operation should be delegated (True), run immediately
  (False), or None to defer to [`resolve_execution_options()`](#plugins.operators.model_evaluation.ConfigureScenario.resolve_execution_options) to
  specify the available options

#### resolve_execution_options(ctx)

Returns the resolved execution options.

Subclasses can implement this method to define the execution options
available for the operation.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.executor.ExecutionOptions`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionOptions) instance

#### resolve_output(ctx)

Returns the resolved output property.

Subclasses can implement this method to define the outputs of the
operator.

By default, this method is called once when the operator is created.
If the operator is dynamic, this method is called after the operator is
executed.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### resolve_placement(ctx)

Returns the resolved placement of the operator.

Subclasses can implement this method to define the placement of the
operator.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Placement`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Placement), or None

#### resolve_run_name(ctx)

Returns the resolved run name of the operator.

Subclasses can implement this method to define the run name of the
operator.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a string, or None

#### resolve_type(ctx, type)

Returns the resolved input or output property.

* **Parameters:**
  * **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
  * **type** – the type of property to resolve, either `"inputs"` or
    `"outputs"`
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### *property* risk_level

The effective risk level of the operator, which is used by guardrail
systems of an agent to classify tool calls.

#### to_json()

Returns a JSON representation of the operator.

* **Returns:**
  a JSON dict

#### *property* uri

The unique identifier of the operator:
`plugin_name/operator_name`.

### *class* plugins.operators.model_evaluation.ConfigureScenarioPlotResolver(\_builtin=False)

Bases: [`Operator`](../../api/fiftyone.operators.operator.md#fiftyone.operators.operator.Operator)

**Attributes:**

| [`config`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.config)         | The `OperatorConfig` for the operator.                                                                           |
|----------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------|
| [`builtin`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.builtin)       | Whether the operator is builtin.                                                                                 |
| [`name`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.name)             |                                                                                                                  |
| [`risk_level`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.risk_level) | The effective risk level of the operator, which is used by guardrail systems of an agent to classify tool calls. |
| [`uri`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.uri)               | The unique identifier of the operator: `plugin_name/operator_name`.                                              |

**Methods:**

| [`execute`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.execute)(ctx)                                                        | Executes the operator.                                               |
|----------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------|
| [`get_subset_def_data_for_eval_key`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.get_subset_def_data_for_eval_key)(ctx, ...) | Builds and returns an execution cache key for each type of scenario. |
| [`get_subset_def_data_for_eval`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.get_subset_def_data_for_eval)(ctx, \_, ...)     |                                                                      |
| [`get_sample_distribution`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.get_sample_distribution)(ctx, subset_expressions)    |                                                                      |
| [`add_secrets`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.add_secrets)(secrets)                                            | Adds secrets to the operator.                                        |
| [`method_to_uri`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.method_to_uri)(method_name)                                    | Converts a method name to a URI.                                     |
| [`resolve_delegation`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_delegation)(ctx)                                  | Returns the resolved *forced* delegation flag.                       |
| [`resolve_execution_options`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_execution_options)(ctx)                    | Returns the resolved execution options.                              |
| [`resolve_input`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_input)(ctx)                                            | Returns the resolved input property.                                 |
| [`resolve_output`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_output)(ctx)                                          | Returns the resolved output property.                                |
| [`resolve_placement`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_placement)(ctx)                                    | Returns the resolved placement of the operator.                      |
| [`resolve_run_name`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_run_name)(ctx)                                      | Returns the resolved run name of the operator.                       |
| [`resolve_type`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_type)(ctx, type)                                        | Returns the resolved input or output property.                       |
| [`to_json`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.to_json)()                                                           | Returns a JSON representation of the operator.                       |

#### *property* config

The `OperatorConfig` for the operator.

#### execute(ctx)

Executes the operator.

Subclasses must implement this method.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  JSON serializable data, or None

#### get_subset_def_data_for_eval_key(ctx, eval_key, \_, name, subset_def)

Builds and returns an execution cache key for each type of scenario.
- eval key + name + type + subset definition

#### get_subset_def_data_for_eval(ctx, \_, eval_result, name, subset_def)

#### get_sample_distribution(ctx, subset_expressions)

#### add_secrets(secrets)

Adds secrets to the operator.

* **Parameters:**
  **secrets** – a list of secrets

#### *property* builtin

Whether the operator is builtin.

#### method_to_uri(method_name)

Converts a method name to a URI.

* **Parameters:**
  **method_name** – the method name
* **Returns:**
  a URI

#### *property* name

#### resolve_delegation(ctx) → [bool](../../api/fiftyone.core.stages.md#fiftyone.core.stages.Exists.bool) | None

Returns the resolved *forced* delegation flag.

Subclasses can implement this method to decide if delegated execution
should be *forced* for the given operation.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  whether the operation should be delegated (True), run immediately
  (False), or None to defer to [`resolve_execution_options()`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_execution_options) to
  specify the available options

#### resolve_execution_options(ctx)

Returns the resolved execution options.

Subclasses can implement this method to define the execution options
available for the operation.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.executor.ExecutionOptions`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionOptions) instance

#### resolve_input(ctx) → [Property](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property) | None

Returns the resolved input property.

Subclasses can implement this method to define the inputs to the
operator. This method should never be called directly. Instead
use [`resolve_type()`](#plugins.operators.model_evaluation.ConfigureScenarioPlotResolver.resolve_type).

By default, this method is called once when the operator is created.
If the operator is dynamic, this method is called each time the input
changes.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### resolve_output(ctx)

Returns the resolved output property.

Subclasses can implement this method to define the outputs of the
operator.

By default, this method is called once when the operator is created.
If the operator is dynamic, this method is called after the operator is
executed.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### resolve_placement(ctx)

Returns the resolved placement of the operator.

Subclasses can implement this method to define the placement of the
operator.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Placement`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Placement), or None

#### resolve_run_name(ctx)

Returns the resolved run name of the operator.

Subclasses can implement this method to define the run name of the
operator.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a string, or None

#### resolve_type(ctx, type)

Returns the resolved input or output property.

* **Parameters:**
  * **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
  * **type** – the type of property to resolve, either `"inputs"` or
    `"outputs"`
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### *property* risk_level

The effective risk level of the operator, which is used by guardrail
systems of an agent to classify tool calls.

#### to_json()

Returns a JSON representation of the operator.

* **Returns:**
  a JSON dict

#### *property* uri

The unique identifier of the operator:
`plugin_name/operator_name`.

### *class* plugins.operators.model_evaluation.ListScenarios(\_builtin=False)

Bases: [`Operator`](../../api/fiftyone.operators.operator.md#fiftyone.operators.operator.Operator)

**Attributes:**

| [`config`](#plugins.operators.model_evaluation.ListScenarios.config)         | The `OperatorConfig` for the operator.                                                                           |
|------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------|
| [`builtin`](#plugins.operators.model_evaluation.ListScenarios.builtin)       | Whether the operator is builtin.                                                                                 |
| [`name`](#plugins.operators.model_evaluation.ListScenarios.name)             |                                                                                                                  |
| [`risk_level`](#plugins.operators.model_evaluation.ListScenarios.risk_level) | The effective risk level of the operator, which is used by guardrail systems of an agent to classify tool calls. |
| [`uri`](#plugins.operators.model_evaluation.ListScenarios.uri)               | The unique identifier of the operator: `plugin_name/operator_name`.                                              |

**Methods:**

| [`execute`](#plugins.operators.model_evaluation.ListScenarios.execute)(ctx)                                     | Executes the operator.                          |
|-----------------------------------------------------------------------------------------------------------------|-------------------------------------------------|
| [`add_secrets`](#plugins.operators.model_evaluation.ListScenarios.add_secrets)(secrets)                         | Adds secrets to the operator.                   |
| [`method_to_uri`](#plugins.operators.model_evaluation.ListScenarios.method_to_uri)(method_name)                 | Converts a method name to a URI.                |
| [`resolve_delegation`](#plugins.operators.model_evaluation.ListScenarios.resolve_delegation)(ctx)               | Returns the resolved *forced* delegation flag.  |
| [`resolve_execution_options`](#plugins.operators.model_evaluation.ListScenarios.resolve_execution_options)(ctx) | Returns the resolved execution options.         |
| [`resolve_input`](#plugins.operators.model_evaluation.ListScenarios.resolve_input)(ctx)                         | Returns the resolved input property.            |
| [`resolve_output`](#plugins.operators.model_evaluation.ListScenarios.resolve_output)(ctx)                       | Returns the resolved output property.           |
| [`resolve_placement`](#plugins.operators.model_evaluation.ListScenarios.resolve_placement)(ctx)                 | Returns the resolved placement of the operator. |
| [`resolve_run_name`](#plugins.operators.model_evaluation.ListScenarios.resolve_run_name)(ctx)                   | Returns the resolved run name of the operator.  |
| [`resolve_type`](#plugins.operators.model_evaluation.ListScenarios.resolve_type)(ctx, type)                     | Returns the resolved input or output property.  |
| [`to_json`](#plugins.operators.model_evaluation.ListScenarios.to_json)()                                        | Returns a JSON representation of the operator.  |

#### *property* config

The `OperatorConfig` for the operator.

#### execute(ctx)

Executes the operator.

Subclasses must implement this method.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  JSON serializable data, or None

#### add_secrets(secrets)

Adds secrets to the operator.

* **Parameters:**
  **secrets** – a list of secrets

#### *property* builtin

Whether the operator is builtin.

#### method_to_uri(method_name)

Converts a method name to a URI.

* **Parameters:**
  **method_name** – the method name
* **Returns:**
  a URI

#### *property* name

#### resolve_delegation(ctx) → [bool](../../api/fiftyone.core.stages.md#fiftyone.core.stages.Exists.bool) | None

Returns the resolved *forced* delegation flag.

Subclasses can implement this method to decide if delegated execution
should be *forced* for the given operation.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  whether the operation should be delegated (True), run immediately
  (False), or None to defer to [`resolve_execution_options()`](#plugins.operators.model_evaluation.ListScenarios.resolve_execution_options) to
  specify the available options

#### resolve_execution_options(ctx)

Returns the resolved execution options.

Subclasses can implement this method to define the execution options
available for the operation.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.executor.ExecutionOptions`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionOptions) instance

#### resolve_input(ctx) → [Property](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property) | None

Returns the resolved input property.

Subclasses can implement this method to define the inputs to the
operator. This method should never be called directly. Instead
use [`resolve_type()`](#plugins.operators.model_evaluation.ListScenarios.resolve_type).

By default, this method is called once when the operator is created.
If the operator is dynamic, this method is called each time the input
changes.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### resolve_output(ctx)

Returns the resolved output property.

Subclasses can implement this method to define the outputs of the
operator.

By default, this method is called once when the operator is created.
If the operator is dynamic, this method is called after the operator is
executed.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### resolve_placement(ctx)

Returns the resolved placement of the operator.

Subclasses can implement this method to define the placement of the
operator.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a [`fiftyone.operators.types.Placement`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Placement), or None

#### resolve_run_name(ctx)

Returns the resolved run name of the operator.

Subclasses can implement this method to define the run name of the
operator.

* **Parameters:**
  **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
* **Returns:**
  a string, or None

#### resolve_type(ctx, type)

Returns the resolved input or output property.

* **Parameters:**
  * **ctx** – the [`fiftyone.operators.executor.ExecutionContext`](../../api/fiftyone.operators.executor.md#fiftyone.operators.executor.ExecutionContext)
  * **type** – the type of property to resolve, either `"inputs"` or
    `"outputs"`
* **Returns:**
  a [`fiftyone.operators.types.Property`](../../api/fiftyone.operators.types.md#fiftyone.operators.types.Property), or None

#### *property* risk_level

The effective risk level of the operator, which is used by guardrail
systems of an agent to classify tool calls.

#### to_json()

Returns a JSON representation of the operator.

* **Returns:**
  a JSON dict

#### *property* uri

The unique identifier of the operator:
`plugin_name/operator_name`.

### plugins.operators.model_evaluation.get_dataset(ctx)

Returns the dataset for the current context.

### plugins.operators.model_evaluation.extract_evaluation_keys(ctx)

### plugins.operators.model_evaluation.get_evaluations_results(ctx)

Returns the evaluation results for the current context.

### plugins.operators.model_evaluation.get_label_attribute_path(params)

Returns the last part of the label attribute path to use in use_subset(type=”attribute”, …)

### plugins.operators.model_evaluation.get_scenario_type(params)

### plugins.operators.model_evaluation.resolve_subsets(scenario_type, params, values)