# fiftyone.utils.sam3
[Segment Anything 3](https://github.com/facebookresearch/sam3)
wrapper for the FiftyOne Model Zoo.
Copyright 2017-2026, Voxel51, Inc.
[voxel51.com](https://voxel51.com/)
**Classes:**
| [`SegmentAnything3ImageModelConfig`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig)(cfg_dict) | Configuration for running a [`SegmentAnything3ImageModel`](#fiftyone.utils.sam3.SegmentAnything3ImageModel). |
|------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem)([...]) | A `GetItem` that loads images, bounding boxes and/or keypoints to feed to [`SegmentAnything3ImageModel`](#fiftyone.utils.sam3.SegmentAnything3ImageModel) instances. |
| [`SegmentAnything3ImageGetItemForVideo`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItemForVideo)([...]) | Workaround for applying image model to video reader frames. |
| [`SAM3ConceptSegmenterOutputProcessor`](#fiftyone.utils.sam3.SAM3ConceptSegmenterOutputProcessor)([...]) | Converts SAM3 concept model outputs to intermediate format for processing. |
| [`SegmentAnything3ImageModel`](#fiftyone.utils.sam3.SegmentAnything3ImageModel)(config) | Wrapper for running [Segment Anything 3](https://ai.meta.com/research/sam3) inference. |
| [`SegmentAnything3VideoModelConfig`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig)(cfg_dict) | Configuration for running a [`SegmentAnything3VideoModel`](#fiftyone.utils.sam3.SegmentAnything3VideoModel). |
| [`SegmentAnything3VideoModel`](#fiftyone.utils.sam3.SegmentAnything3VideoModel)(config) | Wrapper for running [Segment Anything 3](https://ai.meta.com/sam3) inference on videos. |
**Functions:**
| [`build_sam_datapoint_transform`](#fiftyone.utils.sam3.build_sam_datapoint_transform)() | Builds transforms for SAM3 datapoints. |
|------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------|
| [`load_fiftyone_video_frames_sam3`](#fiftyone.utils.sam3.load_fiftyone_video_frames_sam3)(video_path, ...) | Load video frames from a FiftyOne video reader for SAM 3. |
### *class* fiftyone.utils.sam3.SegmentAnything3ImageModelConfig(cfg_dict)
Bases: [`TorchImageModelConfig`](fiftyone.utils.torch.md#fiftyone.utils.torch.TorchImageModelConfig), [`HasZooModel`](fiftyone.zoo.models.md#fiftyone.zoo.models.HasZooModel)
Configuration for running a [`SegmentAnything3ImageModel`](#fiftyone.utils.sam3.SegmentAnything3ImageModel).
See [`fiftyone.utils.torch.TorchImageModelConfig`](fiftyone.utils.torch.md#fiftyone.utils.torch.TorchImageModelConfig) for additional
arguments.
* **Parameters:**
* **points_mask_index** (*None*) – an optional mask index to use for each
keypoint output
* **get_item_cls** (*None*) – a string like
`"fiftyone.utils.sam.SegmentAnything3ImageGetItem"` specifying the
`GetItem` to use for SAM3
* **get_item_args** (*None*) – a dictionary of arguments for
`get_item_cls(field_mapping=field_mapping, **kwargs)`
* **tokenizer_source** (*None*) – the URL to download the SAM3 tokenizer BPE vocab
* **operation_mode** ( *"concept"*) – concept or visual mode of operation for inference
**Methods:**
| [`attributes`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.attributes)() | Returns a list of class attributes to be serialized. |
|------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------|
| [`builder`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.builder)() | Returns a ConfigBuilder instance for this class. |
| [`copy`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.copy)() | Returns a deep copy of the object. |
| [`custom_attributes`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.custom_attributes)([dynamic, private]) | Returns a customizable list of class attributes. |
| [`default`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.default)() | Returns the default config instance. |
| [`download_model_if_necessary`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.download_model_if_necessary)() | Downloads the published model specified by the config, if necessary. |
| [`from_dict`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.from_dict)(d) | Constructs a Config object from a JSON dictionary. |
| [`from_json`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.from_json)(path, \*args, \*\*kwargs) | Constructs a Serializable object from a JSON file. |
| [`from_kwargs`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.from_kwargs)(\*\*kwargs) | Constructs a Config object from keyword arguments. |
| [`from_str`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.from_str)(s, \*args, \*\*kwargs) | Constructs a Serializable object from a JSON string. |
| [`get_class_name`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.get_class_name)() | Returns the fully-qualified class name string of this object. |
| [`init`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.init)(d) | Initializes the published model config. |
| [`load_default`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.load_default)() | Loads the default config instance from file. |
| [`parse_array`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_array)(d, key[, default]) | Parses a raw array attribute. |
| [`parse_bool`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_bool)(d, key[, default]) | Parses a boolean value. |
| [`parse_categorical`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_categorical)(d, key, choices[, default]) | Parses a categorical JSON field, which must take a value from among the given choices. |
| [`parse_dict`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_dict)(d, key[, default]) | Parses a dictionary attribute. |
| [`parse_int`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_int)(d, key[, default]) | Parses an integer attribute. |
| [`parse_mutually_exclusive_fields`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_mutually_exclusive_fields)(fields) | Parses a mutually exclusive dictionary of pre-parsed fields, which must contain exactly one field with a truthy value. |
| [`parse_number`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_number)(d, key[, default]) | Parses a number attribute. |
| [`parse_object`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_object)(d, key, cls[, default]) | Parses an object attribute. |
| [`parse_object_array`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_object_array)(d, key, cls[, default]) | Parses an array of objects. |
| [`parse_object_dict`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_object_dict)(d, key, cls[, default]) | Parses a dictionary whose values are objects. |
| [`parse_path`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_path)(d, key[, default]) | Parses a path attribute. |
| [`parse_raw`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_raw)(d, key[, default]) | Parses a raw (arbitrary) JSON field. |
| [`parse_string`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.parse_string)(d, key[, default]) | Parses a string attribute. |
| [`serialize`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.serialize)([reflective]) | Serializes the object into a dictionary. |
| [`to_str`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.to_str)([pretty_print]) | Returns a string representation of this object. |
| [`validate_all_or_nothing_fields`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.validate_all_or_nothing_fields)(fields) | Validates a dictionary of pre-parsed fields checking that either all or none of the fields have a truthy value. |
| [`write_json`](#fiftyone.utils.sam3.SegmentAnything3ImageModelConfig.write_json)(path[, pretty_print]) | Serializes the object and writes it to disk. |
#### attributes()
Returns a list of class attributes to be serialized.
This method is called internally by `serialize()` to determine the
class attributes to serialize.
Subclasses can override this method, but, by default, all attributes in
vars(self) are returned, minus private attributes, i.e., those starting
with “_”. The order of the attributes in this list is preserved when
serializing objects, so a common pattern is for subclasses to override
this method if they want their JSON files to be organized in a
particular way.
* **Returns:**
a list of class attributes to be serialized
#### *classmethod* builder()
Returns a ConfigBuilder instance for this class.
#### copy()
Returns a deep copy of the object.
* **Returns:**
a Serializable instance
#### custom_attributes(dynamic=False, private=False)
Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private
attributes (those starting with “_”).
* **Parameters:**
* **dynamic** – whether to include dynamic properties, e.g., those defined
by getter/setter methods or the `@property` decorator. By
default, this is False
* **private** – whether to include private properties, i.e., those
starting with “_”. By default, this is False
* **Returns:**
a list of class attributes
#### *classmethod* default()
Returns the default config instance.
By default, this method instantiates the class from an empty
dictionary, which will only succeed if all attributes are optional.
Otherwise, subclasses should override this method to provide the
desired default configuration.
#### download_model_if_necessary()
Downloads the published model specified by the config, if necessary.
After this method is called, the `model_path` attribute will always
contain the path to the model on disk.
#### *classmethod* from_dict(d)
Constructs a Config object from a JSON dictionary.
Config subclass constructors accept JSON dictionaries, so this method
simply passes the dictionary to cls().
* **Parameters:**
**d** – a dict of fields expected by cls
* **Returns:**
an instance of cls
#### *classmethod* from_json(path, \*args, \*\*kwargs)
Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method
simply reads the JSON and calls from_dict(), which subclasses must
implement.
* **Parameters:**
* **path** – the path to the JSON file on disk
* **\*args** – optional positional arguments for `self.from_dict()`
* **\*\*kwargs** – optional keyword arguments for `self.from_dict()`
* **Returns:**
an instance of the Serializable class
#### *classmethod* from_kwargs(\*\*kwargs)
Constructs a Config object from keyword arguments.
* **Parameters:**
**\*\*kwargs** – keyword arguments that define the fields expected by cls
* **Returns:**
an instance of cls
#### *classmethod* from_str(s, \*args, \*\*kwargs)
Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method
simply parses the string and calls from_dict(), which subclasses must
implement.
* **Parameters:**
* **s** – a JSON string representation of a Serializable object
* **\*args** – optional positional arguments for `self.from_dict()`
* **\*\*kwargs** – optional keyword arguments for `self.from_dict()`
* **Returns:**
an instance of the Serializable class
#### *classmethod* get_class_name()
Returns the fully-qualified class name string of this object.
#### init(d)
Initializes the published model config.
This method should be called by `ModelConfig.__init__()`, and it
performs the following tasks:
- Parses the `model_name` and `model_path` parameters
- Populates any default parameters in the provided ModelConfig dict
* **Parameters:**
**d** – a ModelConfig dict
* **Returns:**
a ModelConfig dict with any default parameters populated
#### *classmethod* load_default()
Loads the default config instance from file.
Subclasses must implement this method if they intend to support
default instances.
#### *static* parse_array(d, key, default=)
Parses a raw array attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default list to return if key is not present
* **Returns:**
a list of raw (untouched) values
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_bool(d, key, default=)
Parses a boolean value.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default bool to return if key is not present
* **Returns:**
True/False
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_categorical(d, key, choices, default=)
Parses a categorical JSON field, which must take a value from among
the given choices.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **choices** – either an iterable of possible values or an enum-like
class whose attributes define the possible values
* **default** – a default value to return if key is not present
* **Returns:**
the raw (untouched) value of the given field, which is equal to a
value from `choices`
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the key was present in the dictionary but its value
was not an allowed choice, or if no default value was provided
and the key was not found in the dictionary
#### *static* parse_dict(d, key, default=)
Parses a dictionary attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default dict to return if key is not present
* **Returns:**
a dictionary
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_int(d, key, default=)
Parses an integer attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default integer value to return if key is not present
* **Returns:**
an int
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_mutually_exclusive_fields(fields)
Parses a mutually exclusive dictionary of pre-parsed fields, which
must contain exactly one field with a truthy value.
* **Parameters:**
**fields** – a dictionary of pre-parsed fields
* **Returns:**
the (field, value) that was set
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if zero or more than one truthy value was found
#### *static* parse_number(d, key, default=)
Parses a number attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default numeric value to return if key is not present
* **Returns:**
a number (e.g. int, float)
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_object(d, key, cls, default=)
Parses an object attribute.
The value of d[key] can be either an instance of cls or a serialized
dict from an instance of cls.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **cls** – the class of d[key]
* **default** – a default cls instance to return if key is not present
* **Returns:**
an instance of cls
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_object_array(d, key, cls, default=)
Parses an array of objects.
The values in d[key] can be either instances of cls or serialized
dicts from instances of cls.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **cls** – the class of the elements of list d[key]
* **default** – the default list to return if key is not present
* **Returns:**
a list of cls instances
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_object_dict(d, key, cls, default=)
Parses a dictionary whose values are objects.
The values in d[key] can be either instances of cls or serialized
dicts from instances of cls.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **cls** – the class of the values of dictionary d[key]
* **default** – the default dict of cls instances to return if key is not
present
* **Returns:**
a dictionary whose values are cls instances
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_path(d, key, default=)
Parses a path attribute.
The path is converted to an absolute path if necessary via
`os.path.abspath(os.path.expanduser(value))`.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default string to return if key is not present
* **Returns:**
a path string
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_raw(d, key, default=)
Parses a raw (arbitrary) JSON field.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default value to return if key is not present
* **Returns:**
the raw (untouched) value of the given field
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if no default value was provided and the key was not
found in the dictionary
#### *static* parse_string(d, key, default=)
Parses a string attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default string to return if key is not present
* **Returns:**
a string
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### serialize(reflective=False)
Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object,
including element-wise serialization of lists and dictionary values.
* **Parameters:**
**reflective** – whether to include reflective attributes when
serializing the object. By default, this is False
* **Returns:**
a JSON dictionary representation of the object
#### to_str(pretty_print=True, \*\*kwargs)
Returns a string representation of this object.
* **Parameters:**
* **pretty_print** – whether to render the JSON in human readable format
with newlines and indentations. By default, this is True
* **\*\*kwargs** – optional keyword arguments for `self.serialize()`
* **Returns:**
a string representation of the object
#### *static* validate_all_or_nothing_fields(fields)
Validates a dictionary of pre-parsed fields checking that either
all or none of the fields have a truthy value.
* **Parameters:**
**fields** – a dictionary of pre-parsed fields
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if some values are truth and some are not
#### write_json(path, pretty_print=False, \*\*kwargs)
Serializes the object and writes it to disk.
* **Parameters:**
* **path** – the output path
* **pretty_print** – whether to render the JSON in human readable format
with newlines and indentations. By default, this is False
* **\*\*kwargs** – optional keyword arguments for `self.serialize()`
### *class* fiftyone.utils.sam3.SegmentAnything3ImageGetItem(field_mapping=None, transform=None, use_numpy=False, box_transform=None, point_transform=None, text_prompts=None, operation_mode='concept', \*\*kwargs)
Bases: [`SegmentAnythingImageGetItem`](fiftyone.utils.sam.md#fiftyone.utils.sam.SegmentAnythingImageGetItem)
A `GetItem` that loads images, bounding boxes and/or keypoints to feed to
[`SegmentAnything3ImageModel`](#fiftyone.utils.sam3.SegmentAnything3ImageModel) instances.
* **Parameters:**
* **field_mapping** (*None*) – the user-supplied dict mapping keys in
[`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem.required_keys) to field names of their dataset that contain
the required values
* **transform** (*None*) – SAM specific image transform function to apply
* **use_numpy** (*False*) – whether to use numpy arrays rather than PIL images
and Torch tensors when loading data
* **box_transform** (*None*) – SAM specific box transform function to apply
* **point_transform** (*None*) – SAM specific point transform function to apply
* **text_prompts** (*None*) – Text prompts for concept prompting the model
* **operation_mode** ( *"concept"*) – Operation mode of the model (required for collate_fn)
**Attributes:**
| [`field_mapping`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem.field_mapping) | A user-supplied dictionary mappings keys in [`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem.required_keys) to field names of their dataset that contain the required values. |
|--------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem.required_keys) | The list of keys that must exist on the dicts provided to the `__call__()` method at runtime. |
#### *property* field_mapping
A user-supplied dictionary mappings keys in [`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem.required_keys)
to field names of their dataset that contain the required values.
#### *property* required_keys
The list of keys that must exist on the dicts provided to the
`__call__()` method at runtime.
### *class* fiftyone.utils.sam3.SegmentAnything3ImageGetItemForVideo(field_mapping=None, transform=None, use_numpy=False, box_transform=None, point_transform=None, text_prompts=None, operation_mode='concept', \*\*kwargs)
Bases: [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem)
Workaround for applying image model to video reader frames.
Frames are not stored on disk and therefore cannot be loaded.
**Attributes:**
| [`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItemForVideo.required_keys) | The list of keys that must exist on the dicts provided to the `__call__()` method at runtime. |
|----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`field_mapping`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItemForVideo.field_mapping) | A user-supplied dictionary mappings keys in [`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItemForVideo.required_keys) to field names of their dataset that contain the required values. |
#### *property* required_keys
The list of keys that must exist on the dicts provided to the
`__call__()` method at runtime.
#### *property* field_mapping
A user-supplied dictionary mappings keys in [`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItemForVideo.required_keys)
to field names of their dataset that contain the required values.
### fiftyone.utils.sam3.build_sam_datapoint_transform()
Builds transforms for SAM3 datapoints.
* **Returns:**
composed transforms
### *class* fiftyone.utils.sam3.SAM3ConceptSegmenterOutputProcessor(classes=None, mask_thresh=0.5)
Bases: [`OutputProcessor`](fiftyone.utils.torch.md#fiftyone.utils.torch.OutputProcessor)
Converts SAM3 concept model outputs to intermediate format for processing.
Each detected mask is associated with the label of the text query that
produced it, passed forward as `detection_labels` for the output
processor.
* **Parameters:**
* **classes** (*None*) – ordered list of text prompts used as queries
* **mask_thresh** (*0.5*) – threshold for pre-filtering detections by confidence
### *class* fiftyone.utils.sam3.SegmentAnything3ImageModel(config)
Bases: [`SegmentAnything2ImageModel`](fiftyone.utils.sam2.md#fiftyone.utils.sam2.SegmentAnything2ImageModel)
Wrapper for running [Segment Anything 3](https://ai.meta.com/research/sam3)
inference.
* **Parameters:**
**config** – a `SegmentAnything3ModelConfig`
**Attributes:**
| [`operation_mode`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.operation_mode) | Whether to use the model in visual or concept segmentation mode |
|------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------|
| [`can_embed_prompts`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.can_embed_prompts) | Whether this model can generate prompt embeddings. |
| [`classes`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.classes) | The list of class labels for the model, if known. |
| [`device`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.device) | The `torch:torch.torch.device` that the model is using. |
| [`has_collate_fn`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.has_collate_fn) | Whether this model has a custom collate function. |
| [`has_embeddings`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.has_embeddings) | Whether this model has embeddings. |
| [`has_logits`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.has_logits) | Whether this instance can generate logits. |
| [`mask_targets`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.mask_targets) | The mask targets for the model, if any. |
| [`media_type`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.media_type) | The media type processed by the model. |
| [`num_classes`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.num_classes) | The number of classes for the model, if known. |
| [`preprocess`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.preprocess) | Whether to apply preprocessing transforms for inference, if any. |
| [`ragged_batches`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.ragged_batches) | Whether [`transforms()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.transforms) may return tensors of different sizes. |
| [`required_keys`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.required_keys) | The required keys that must be provided as parameters to methods like `apply_model()` and `compute_embeddings()` at runtime. |
| [`skeleton`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.skeleton) | The keypoint skeleton for the model, if any. |
| [`store_logits`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.store_logits) | Whether the model should store logits in its predictions. |
| [`transforms`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.transforms) | A `torchvision.transforms` function that will be applied to each input before prediction, if any. |
| [`using_gpu`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.using_gpu) | Whether the model is using GPU. |
| [`using_half_precision`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.using_half_precision) | Whether the model is using half precision. |
**Methods:**
| [`collate_fn`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.collate_fn)(batch) | Collates a batch of inputs where each input is generated from [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem). |
|----------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`build_get_item`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.build_get_item)([field_mapping]) | Builds a [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem) for loading model input from samples. |
| [`embed`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.embed)(arg) | Generates an embedding for the given data. |
| [`embed_all`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.embed_all)(args) | Generates embeddings for the given iterable of data. |
| [`from_config`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.from_config)(config) | Instantiates a Configurable class from a Config instance. |
| [`from_dict`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.from_dict)(d) | Instantiates a Configurable class from a Config dict. |
| [`from_json`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.from_json)(json_path) | Instantiates a Configurable class from a Config JSON file. |
| [`from_kwargs`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.from_kwargs)(\*\*kwargs) | Instantiates a Configurable class from keyword arguments defining the attributes of a Config. |
| [`get_embeddings`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.get_embeddings)() | Returns the embeddings generated by the last forward pass of the model. |
| [`parse`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.parse)(class_name[, module_name]) | Parses a Configurable subclass name string. |
| [`predict`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict)(img[, sample]) | Performs prediction a single image. |
| [`predict_all`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict_all)(imgs[, samples]) | Performs prediction on multiple images. |
| [`predict_interactive`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict_interactive)([sample, boxes, points, ...]) | Generates predictions in interactive mode. |
| [`validate`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.validate)(config) | Validates that the given config is an instance of Config. |
#### *property* operation_mode
Whether to use the model in visual or concept segmentation mode
#### *static* collate_fn(batch)
Collates a batch of inputs where each input is generated from [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem).
* **Parameters:**
**batch** – a list of dict containing model input from [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem)
* **Returns:**
a collated dictionary of model input for the batch.
#### build_get_item(field_mapping=None)
Builds a [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem) for loading model input from samples.
* **Parameters:**
**field_mapping** (*None*) – a dict mapping required keys to sample fields
* **Returns:**
a [`SegmentAnything3ImageGetItem`](#fiftyone.utils.sam3.SegmentAnything3ImageGetItem) instance
#### *property* can_embed_prompts
Whether this model can generate prompt embeddings.
This method returns `False` by default. Models that can generate
prompt embeddings should override this via implementing the
`PromptMixin` interface.
#### *property* classes
The list of class labels for the model, if known.
#### *property* device
The `torch:torch.torch.device` that the model is using.
#### embed(arg)
Generates an embedding for the given data.
Subclasses can override this method to increase efficiency, but, by
default, this method simply calls [`predict()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict) and then returns
[`get_embeddings()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.get_embeddings).
* **Parameters:**
**arg** – the data. See [`predict()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict) for details
* **Returns:**
a numpy array containing the embedding
#### embed_all(args)
Generates embeddings for the given iterable of data.
Subclasses can override this method to increase efficiency, but, by
default, this method simply iterates over the data and applies
[`embed()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.embed) to each.
* **Parameters:**
**args** – an iterable of data. See [`predict_all()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict_all) for details
* **Returns:**
a numpy array containing the embeddings stacked along axis 0
#### *classmethod* from_config(config)
Instantiates a Configurable class from a Config instance.
#### *classmethod* from_dict(d)
Instantiates a Configurable class from a Config dict.
* **Parameters:**
**d** – a dict to construct a Config
* **Returns:**
an instance of cls
#### *classmethod* from_json(json_path)
Instantiates a Configurable class from a Config JSON file.
* **Parameters:**
**json_path** – path to a JSON file for type Config
* **Returns:**
an instance of cls
#### *classmethod* from_kwargs(\*\*kwargs)
Instantiates a Configurable class from keyword arguments defining
the attributes of a Config.
* **Parameters:**
**\*\*kwargs** – keyword arguments that define the fields of a
Config dict
* **Returns:**
an instance of cls
#### get_embeddings()
Returns the embeddings generated by the last forward pass of the
model.
By convention, this method should always return an array whose first
axis represents batch size (which will always be 1 when [`predict()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict)
was last used).
* **Returns:**
a numpy array containing the embedding(s)
#### *property* has_collate_fn
Whether this model has a custom collate function.
Set this to `True` if you want [`collate_fn()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.collate_fn) to be used during
inference.
#### *property* has_embeddings
Whether this model has embeddings.
#### *property* has_logits
Whether this instance can generate logits.
#### *property* mask_targets
The mask targets for the model, if any.
#### *property* media_type
The media type processed by the model.
#### *property* num_classes
The number of classes for the model, if known.
#### *static* parse(class_name, module_name=None)
Parses a Configurable subclass name string.
Assumes both the Configurable class and the Config class are defined
in the same module. The module containing the classes will be loaded
if necessary.
* **Parameters:**
* **class_name** – a string containing the name of the Configurable class,
e.g. “ClassName”, or a fully-qualified class name, e.g.
“eta.core.config.ClassName”
* **module_name** – a string containing the fully-qualified module name,
e.g. “eta.core.config”, or None if class_name includes the
module name. Set module_name = \_\_name_\_ to load a class from
the calling module
* **Returns:**
the Configurable class
config_cls: the Config class associated with cls
* **Return type:**
[cls](fiftyone.brain.internal.core.elasticsearch.md#fiftyone.brain.internal.core.elasticsearch.ElasticsearchSimilarityConfig.cls)
#### predict(img, sample=None)
Performs prediction a single image.
* **Parameters:**
* **img** – a dictionary containing image, original size, and prompts. See `fiftyone.utils.sam.SegmentAnythingGetItem` for details.
* **sample** (*None*) – sample is no longer used. Available for backward compatibility.
* **Returns:**
a [`fiftyone.core.labels.Detections`](fiftyone.core.labels.md#fiftyone.core.labels.Detections) instance or a dict
containing the “masks”, “iou_predictions”, “low_res_logits” from SAM model output.
#### predict_all(imgs, samples=None)
Performs prediction on multiple images.
To generate imgs dictionary and run prediction:
> field_mapping = {“box_prompt_field”: “ground-truth”}
> get_item = model.build_get_item(field_mapping=field_mapping)
> model_inputs = fout.get_model_inputs_from_get_item(samples, get_item)
> outputs = model.predict_all(model_inputs)
* **Parameters:**
* **imgs** – a list of dictionary or a dictionary containing images, original sizes, and prompts. See `fiftyone.utils.sam.SegmentAnythingGetItem` for details.
* **samples** (*None*) – samples is no longer used. Available for backward compatibility.
* **Returns:**
a list of [`fiftyone.core.labels.Detections`](fiftyone.core.labels.md#fiftyone.core.labels.Detections) instances or a list of dict
containing the “masks”, “iou_predictions”, “low_res_logits” from SAM model output.
#### predict_interactive(sample=None, boxes=None, points=None, point_labels=None, prompt_classes=None, boxes_xyxy=None)
Generates predictions in interactive mode. Image embedding is cached.
* **Parameters:**
* **sample** (*None*) – a FiftyOne Sample with image media
* **boxes** (*None*) – a tensor of Bx4 pre-processed SAM transformed boxes in XYXY pixels
* **points** (*None*) – a tensor of BxNx2 or a list of B tensors with pre-processed points in XY pixels
* **point_labels** (*None*) – a BxN tensor or a list of B tensors of labels for the point prompts
* **prompt_classes** (*None*) – a list of B class labels
* **boxes_xyxy** – a list of Bx4 boxes in XYXY pixels in original image space
* **Returns:**
[`fiftyone.core.labels.Detections`](fiftyone.core.labels.md#fiftyone.core.labels.Detections) or dict
containing the “masks”, “iou_predictions”, “low_res_logits” from SAM model output.
#### *property* preprocess
Whether to apply preprocessing transforms for inference, if any.
#### *property* ragged_batches
Whether [`transforms()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.transforms) may return tensors of different sizes.
If True, then passing ragged lists of images to [`predict_all()`](#fiftyone.utils.sam3.SegmentAnything3ImageModel.predict_all) may
not be not allowed.
#### *property* required_keys
The required keys that must be provided as parameters to methods
like `apply_model()` and `compute_embeddings()` at runtime.
#### *property* skeleton
The keypoint skeleton for the model, if any.
#### *property* store_logits
Whether the model should store logits in its predictions.
#### *property* transforms
A `torchvision.transforms` function that will be applied to each
input before prediction, if any.
#### *property* using_gpu
Whether the model is using GPU.
#### *property* using_half_precision
Whether the model is using half precision.
#### *classmethod* validate(config)
Validates that the given config is an instance of Config.
* **Raises:**
**ConfigurableError** – if config is not an instance of Config
### *class* fiftyone.utils.sam3.SegmentAnything3VideoModelConfig(cfg_dict)
Bases: [`TorchImageModelConfig`](fiftyone.utils.torch.md#fiftyone.utils.torch.TorchImageModelConfig), [`HasZooModel`](fiftyone.zoo.models.md#fiftyone.zoo.models.HasZooModel)
Configuration for running a [`SegmentAnything3VideoModel`](#fiftyone.utils.sam3.SegmentAnything3VideoModel).
See [`fiftyone.utils.torch.TorchImageModelConfig`](fiftyone.utils.torch.md#fiftyone.utils.torch.TorchImageModelConfig) for additional
arguments.
* **Parameters:**
* **operation_mode** ( *"concept"*) – concept or visual mode of operation for inference
* **propagation_direction** ( *"both"*) – direction to propagate in video;
supported values are `"forward"`, `"backward"`, and `"both"`
* **propagation_precedence** – detection that takes precendence when merging results for bidirection propagation;
supported values are `"forward"` and `"backward"`
* **prompt_frame_indices** (*None*) – 1-based frame indices for visual / exemplar prompts
* **text_frame_idx** (*1*) – 1-based frame index for text concept prompts
* **tokenizer_source** (*None*) – the URL to download the SAM3 tokenizer BPE vocab
* **concept_predictor_fcn** (*None*) – a function or string like
`"sam3.model_builder.build_sam3_video_model"` specifying the entrypoint
function that loads the concept predictor
**Methods:**
| [`attributes`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.attributes)() | Returns a list of class attributes to be serialized. |
|------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------|
| [`builder`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.builder)() | Returns a ConfigBuilder instance for this class. |
| [`copy`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.copy)() | Returns a deep copy of the object. |
| [`custom_attributes`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.custom_attributes)([dynamic, private]) | Returns a customizable list of class attributes. |
| [`default`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.default)() | Returns the default config instance. |
| [`download_model_if_necessary`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.download_model_if_necessary)() | Downloads the published model specified by the config, if necessary. |
| [`from_dict`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.from_dict)(d) | Constructs a Config object from a JSON dictionary. |
| [`from_json`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.from_json)(path, \*args, \*\*kwargs) | Constructs a Serializable object from a JSON file. |
| [`from_kwargs`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.from_kwargs)(\*\*kwargs) | Constructs a Config object from keyword arguments. |
| [`from_str`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.from_str)(s, \*args, \*\*kwargs) | Constructs a Serializable object from a JSON string. |
| [`get_class_name`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.get_class_name)() | Returns the fully-qualified class name string of this object. |
| [`init`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.init)(d) | Initializes the published model config. |
| [`load_default`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.load_default)() | Loads the default config instance from file. |
| [`parse_array`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_array)(d, key[, default]) | Parses a raw array attribute. |
| [`parse_bool`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_bool)(d, key[, default]) | Parses a boolean value. |
| [`parse_categorical`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_categorical)(d, key, choices[, default]) | Parses a categorical JSON field, which must take a value from among the given choices. |
| [`parse_dict`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_dict)(d, key[, default]) | Parses a dictionary attribute. |
| [`parse_int`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_int)(d, key[, default]) | Parses an integer attribute. |
| [`parse_mutually_exclusive_fields`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_mutually_exclusive_fields)(fields) | Parses a mutually exclusive dictionary of pre-parsed fields, which must contain exactly one field with a truthy value. |
| [`parse_number`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_number)(d, key[, default]) | Parses a number attribute. |
| [`parse_object`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_object)(d, key, cls[, default]) | Parses an object attribute. |
| [`parse_object_array`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_object_array)(d, key, cls[, default]) | Parses an array of objects. |
| [`parse_object_dict`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_object_dict)(d, key, cls[, default]) | Parses a dictionary whose values are objects. |
| [`parse_path`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_path)(d, key[, default]) | Parses a path attribute. |
| [`parse_raw`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_raw)(d, key[, default]) | Parses a raw (arbitrary) JSON field. |
| [`parse_string`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.parse_string)(d, key[, default]) | Parses a string attribute. |
| [`serialize`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.serialize)([reflective]) | Serializes the object into a dictionary. |
| [`to_str`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.to_str)([pretty_print]) | Returns a string representation of this object. |
| [`validate_all_or_nothing_fields`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.validate_all_or_nothing_fields)(fields) | Validates a dictionary of pre-parsed fields checking that either all or none of the fields have a truthy value. |
| [`write_json`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig.write_json)(path[, pretty_print]) | Serializes the object and writes it to disk. |
#### attributes()
Returns a list of class attributes to be serialized.
This method is called internally by `serialize()` to determine the
class attributes to serialize.
Subclasses can override this method, but, by default, all attributes in
vars(self) are returned, minus private attributes, i.e., those starting
with “_”. The order of the attributes in this list is preserved when
serializing objects, so a common pattern is for subclasses to override
this method if they want their JSON files to be organized in a
particular way.
* **Returns:**
a list of class attributes to be serialized
#### *classmethod* builder()
Returns a ConfigBuilder instance for this class.
#### copy()
Returns a deep copy of the object.
* **Returns:**
a Serializable instance
#### custom_attributes(dynamic=False, private=False)
Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private
attributes (those starting with “_”).
* **Parameters:**
* **dynamic** – whether to include dynamic properties, e.g., those defined
by getter/setter methods or the `@property` decorator. By
default, this is False
* **private** – whether to include private properties, i.e., those
starting with “_”. By default, this is False
* **Returns:**
a list of class attributes
#### *classmethod* default()
Returns the default config instance.
By default, this method instantiates the class from an empty
dictionary, which will only succeed if all attributes are optional.
Otherwise, subclasses should override this method to provide the
desired default configuration.
#### download_model_if_necessary()
Downloads the published model specified by the config, if necessary.
After this method is called, the `model_path` attribute will always
contain the path to the model on disk.
#### *classmethod* from_dict(d)
Constructs a Config object from a JSON dictionary.
Config subclass constructors accept JSON dictionaries, so this method
simply passes the dictionary to cls().
* **Parameters:**
**d** – a dict of fields expected by cls
* **Returns:**
an instance of cls
#### *classmethod* from_json(path, \*args, \*\*kwargs)
Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method
simply reads the JSON and calls from_dict(), which subclasses must
implement.
* **Parameters:**
* **path** – the path to the JSON file on disk
* **\*args** – optional positional arguments for `self.from_dict()`
* **\*\*kwargs** – optional keyword arguments for `self.from_dict()`
* **Returns:**
an instance of the Serializable class
#### *classmethod* from_kwargs(\*\*kwargs)
Constructs a Config object from keyword arguments.
* **Parameters:**
**\*\*kwargs** – keyword arguments that define the fields expected by cls
* **Returns:**
an instance of cls
#### *classmethod* from_str(s, \*args, \*\*kwargs)
Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method
simply parses the string and calls from_dict(), which subclasses must
implement.
* **Parameters:**
* **s** – a JSON string representation of a Serializable object
* **\*args** – optional positional arguments for `self.from_dict()`
* **\*\*kwargs** – optional keyword arguments for `self.from_dict()`
* **Returns:**
an instance of the Serializable class
#### *classmethod* get_class_name()
Returns the fully-qualified class name string of this object.
#### init(d)
Initializes the published model config.
This method should be called by `ModelConfig.__init__()`, and it
performs the following tasks:
- Parses the `model_name` and `model_path` parameters
- Populates any default parameters in the provided ModelConfig dict
* **Parameters:**
**d** – a ModelConfig dict
* **Returns:**
a ModelConfig dict with any default parameters populated
#### *classmethod* load_default()
Loads the default config instance from file.
Subclasses must implement this method if they intend to support
default instances.
#### *static* parse_array(d, key, default=)
Parses a raw array attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default list to return if key is not present
* **Returns:**
a list of raw (untouched) values
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_bool(d, key, default=)
Parses a boolean value.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default bool to return if key is not present
* **Returns:**
True/False
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_categorical(d, key, choices, default=)
Parses a categorical JSON field, which must take a value from among
the given choices.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **choices** – either an iterable of possible values or an enum-like
class whose attributes define the possible values
* **default** – a default value to return if key is not present
* **Returns:**
the raw (untouched) value of the given field, which is equal to a
value from `choices`
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the key was present in the dictionary but its value
was not an allowed choice, or if no default value was provided
and the key was not found in the dictionary
#### *static* parse_dict(d, key, default=)
Parses a dictionary attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default dict to return if key is not present
* **Returns:**
a dictionary
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_int(d, key, default=)
Parses an integer attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default integer value to return if key is not present
* **Returns:**
an int
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_mutually_exclusive_fields(fields)
Parses a mutually exclusive dictionary of pre-parsed fields, which
must contain exactly one field with a truthy value.
* **Parameters:**
**fields** – a dictionary of pre-parsed fields
* **Returns:**
the (field, value) that was set
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if zero or more than one truthy value was found
#### *static* parse_number(d, key, default=)
Parses a number attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default numeric value to return if key is not present
* **Returns:**
a number (e.g. int, float)
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_object(d, key, cls, default=)
Parses an object attribute.
The value of d[key] can be either an instance of cls or a serialized
dict from an instance of cls.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **cls** – the class of d[key]
* **default** – a default cls instance to return if key is not present
* **Returns:**
an instance of cls
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_object_array(d, key, cls, default=)
Parses an array of objects.
The values in d[key] can be either instances of cls or serialized
dicts from instances of cls.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **cls** – the class of the elements of list d[key]
* **default** – the default list to return if key is not present
* **Returns:**
a list of cls instances
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_object_dict(d, key, cls, default=)
Parses a dictionary whose values are objects.
The values in d[key] can be either instances of cls or serialized
dicts from instances of cls.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **cls** – the class of the values of dictionary d[key]
* **default** – the default dict of cls instances to return if key is not
present
* **Returns:**
a dictionary whose values are cls instances
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_path(d, key, default=)
Parses a path attribute.
The path is converted to an absolute path if necessary via
`os.path.abspath(os.path.expanduser(value))`.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default string to return if key is not present
* **Returns:**
a path string
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### *static* parse_raw(d, key, default=)
Parses a raw (arbitrary) JSON field.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default value to return if key is not present
* **Returns:**
the raw (untouched) value of the given field
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if no default value was provided and the key was not
found in the dictionary
#### *static* parse_string(d, key, default=)
Parses a string attribute.
* **Parameters:**
* **d** – a JSON dictionary
* **key** – the key to parse
* **default** – a default string to return if key is not present
* **Returns:**
a string
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if the field value was the wrong type or no default
value was provided and the key was not found in the dictionary
#### serialize(reflective=False)
Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object,
including element-wise serialization of lists and dictionary values.
* **Parameters:**
**reflective** – whether to include reflective attributes when
serializing the object. By default, this is False
* **Returns:**
a JSON dictionary representation of the object
#### to_str(pretty_print=True, \*\*kwargs)
Returns a string representation of this object.
* **Parameters:**
* **pretty_print** – whether to render the JSON in human readable format
with newlines and indentations. By default, this is True
* **\*\*kwargs** – optional keyword arguments for `self.serialize()`
* **Returns:**
a string representation of the object
#### *static* validate_all_or_nothing_fields(fields)
Validates a dictionary of pre-parsed fields checking that either
all or none of the fields have a truthy value.
* **Parameters:**
**fields** – a dictionary of pre-parsed fields
* **Raises:**
[**ConfigError**](fiftyone.zoo.md#fiftyone.zoo.ConfigError) – if some values are truth and some are not
#### write_json(path, pretty_print=False, \*\*kwargs)
Serializes the object and writes it to disk.
* **Parameters:**
* **path** – the output path
* **pretty_print** – whether to render the JSON in human readable format
with newlines and indentations. By default, this is False
* **\*\*kwargs** – optional keyword arguments for `self.serialize()`
### *class* fiftyone.utils.sam3.SegmentAnything3VideoModel(config)
Bases: [`SamplesMixin`](fiftyone.core.models.md#fiftyone.core.models.SamplesMixin), [`Model`](fiftyone.core.models.md#fiftyone.core.models.Model)
Wrapper for running [Segment Anything 3](https://ai.meta.com/sam3)
inference on videos.
SAM3 Video performs Promptable Concept Segmentation (PCS) on videos -
given a text prompt, it finds, segments, and tracks ALL instances of the
concept across video frames.
Video prompt example:
```default
import fiftyone as fo
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset("quickstart-video", max_samples=2)
model = foz.load_zoo_model("segment-anything-3-video-torch")
# Segment and track "person" instances across frames
dataset.apply_model(
model,
label_field="sam3_tracking",
classes=["person"],
)
session = fo.launch_app(dataset)
```
* **Parameters:**
**config** – a [`SegmentAnything3VideoModelConfig`](#fiftyone.utils.sam3.SegmentAnything3VideoModelConfig)
**Attributes:**
| [`needs_fields`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.needs_fields) | A dict mapping model-specific keys to sample field names. |
|------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`media_type`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.media_type) | The media type processed by the model. |
| [`operation_mode`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.operation_mode) | Whether to use the model in visual or concept segmentation mode |
| [`can_embed_prompts`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.can_embed_prompts) | Whether this model can generate prompt embeddings. |
| [`has_embeddings`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.has_embeddings) | Whether this model can generate embeddings. |
| [`has_logits`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.has_logits) | Whether this model can generate logits for its predictions. |
| [`preprocess`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.preprocess) | Whether to apply [`transforms()`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.transforms) during inference (True) or to assume that they have already been applied (False). |
| [`ragged_batches`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.ragged_batches) | True/False whether [`transforms()`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.transforms) may return tensors of different sizes. |
| [`transforms`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.transforms) | The preprocessing function that will/must be applied to each input before prediction, or `None` if no preprocessing is performed. |
**Methods:**
| [`predict`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.predict)(video_reader, sample) | Performs prediction on the given data. |
|-----------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------|
| [`from_config`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.from_config)(config) | Instantiates a Configurable class from a Config instance. |
| [`from_dict`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.from_dict)(d) | Instantiates a Configurable class from a Config dict. |
| [`from_json`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.from_json)(json_path) | Instantiates a Configurable class from a Config JSON file. |
| [`from_kwargs`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.from_kwargs)(\*\*kwargs) | Instantiates a Configurable class from keyword arguments defining the attributes of a Config. |
| [`parse`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.parse)(class_name[, module_name]) | Parses a Configurable subclass name string. |
| [`predict_all`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.predict_all)(args[, samples]) | Performs prediction on the given iterable of data. |
| [`validate`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.validate)(config) | Validates that the given config is an instance of Config. |
#### *property* needs_fields
A dict mapping model-specific keys to sample field names.
#### *property* media_type
The media type processed by the model.
Supported values are “image” and “video”.
#### *property* operation_mode
Whether to use the model in visual or concept segmentation mode
#### predict(video_reader, sample)
Performs prediction on the given data.
Image models should support, at minimum, processing `arg` values that
are uint8 numpy arrays (HWC).
Video models should support, at minimum, processing `arg` values that
are `eta.core.video.VideoReader` instances.
* **Parameters:**
* **arg** – the data
* **sample** (*None*) – the [`fiftyone.core.sample.Sample`](fiftyone.core.sample.md#fiftyone.core.sample.Sample) associated
with the data
* **Returns:**
a [`fiftyone.core.labels.Label`](fiftyone.core.labels.md#fiftyone.core.labels.Label) instance or dict of
[`fiftyone.core.labels.Label`](fiftyone.core.labels.md#fiftyone.core.labels.Label) instances containing the
predictions
#### *property* can_embed_prompts
Whether this model can generate prompt embeddings.
This method returns `False` by default. Models that can generate
prompt embeddings should override this via implementing the
`PromptMixin` interface.
#### *classmethod* from_config(config)
Instantiates a Configurable class from a Config instance.
#### *classmethod* from_dict(d)
Instantiates a Configurable class from a Config dict.
* **Parameters:**
**d** – a dict to construct a Config
* **Returns:**
an instance of cls
#### *classmethod* from_json(json_path)
Instantiates a Configurable class from a Config JSON file.
* **Parameters:**
**json_path** – path to a JSON file for type Config
* **Returns:**
an instance of cls
#### *classmethod* from_kwargs(\*\*kwargs)
Instantiates a Configurable class from keyword arguments defining
the attributes of a Config.
* **Parameters:**
**\*\*kwargs** – keyword arguments that define the fields of a
Config dict
* **Returns:**
an instance of cls
#### *property* has_embeddings
Whether this model can generate embeddings.
This method returns `False` by default. Models that can generate
embeddings should override this via implementing the
`EmbeddingsMixin` interface.
#### *property* has_logits
Whether this model can generate logits for its predictions.
This method returns `False` by default. Models that can generate
logits should override this via implementing the
`LogitsMixin` interface.
#### *static* parse(class_name, module_name=None)
Parses a Configurable subclass name string.
Assumes both the Configurable class and the Config class are defined
in the same module. The module containing the classes will be loaded
if necessary.
* **Parameters:**
* **class_name** – a string containing the name of the Configurable class,
e.g. “ClassName”, or a fully-qualified class name, e.g.
“eta.core.config.ClassName”
* **module_name** – a string containing the fully-qualified module name,
e.g. “eta.core.config”, or None if class_name includes the
module name. Set module_name = \_\_name_\_ to load a class from
the calling module
* **Returns:**
the Configurable class
config_cls: the Config class associated with cls
* **Return type:**
[cls](fiftyone.brain.internal.core.elasticsearch.md#fiftyone.brain.internal.core.elasticsearch.ElasticsearchSimilarityConfig.cls)
#### predict_all(args, samples=None)
Performs prediction on the given iterable of data.
Image models should support, at minimum, processing `args` values
that are either lists of uint8 numpy arrays (HWC) or numpy array
tensors (NHWC).
Video models should support, at minimum, processing `args` values
that are lists of `eta.core.video.VideoReader` instances.
Subclasses can override this method to increase efficiency, but, by
default, this method simply iterates over the data and applies
[`predict()`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.predict) to each.
* **Parameters:**
* **args** – an iterable of data
* **samples** (*None*) – an iterable of [`fiftyone.core.sample.Sample`](fiftyone.core.sample.md#fiftyone.core.sample.Sample)
instances associated with the data
* **Returns:**
a list of [`fiftyone.core.labels.Label`](fiftyone.core.labels.md#fiftyone.core.labels.Label) instances or a list
of dicts of [`fiftyone.core.labels.Label`](fiftyone.core.labels.md#fiftyone.core.labels.Label) instances
containing the predictions
#### *property* preprocess
Whether to apply [`transforms()`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.transforms) during inference (True) or to
assume that they have already been applied (False).
#### *property* ragged_batches
True/False whether [`transforms()`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.transforms) may return tensors of
different sizes. If True, then passing ragged lists of data to
[`predict_all()`](#fiftyone.utils.sam3.SegmentAnything3VideoModel.predict_all) is not allowed.
#### *property* transforms
The preprocessing function that will/must be applied to each input
before prediction, or `None` if no preprocessing is performed.
#### *classmethod* validate(config)
Validates that the given config is an instance of Config.
* **Raises:**
**ConfigurableError** – if config is not an instance of Config
### fiftyone.utils.sam3.load_fiftyone_video_frames_sam3(video_path, image_size, offload_video_to_cpu, img_mean=(0.5, 0.5, 0.5), img_std=(0.5, 0.5, 0.5), async_loading_frames=False, compute_device=device(type='cuda'))
Load video frames from a FiftyOne video reader for SAM 3.
SAM 3 uses different normalisation constants from SAM 2:
mean/std = 0.5 instead of ImageNet values.