segment-anything-3-image-torch#

Open-vocabulary instance segmentation that finds and segments all objects matching a text concept like ‘person’ or ‘yellow school bus’.

Details

  • Model name: segment-anything-3-image-torch

  • Model source: facebookresearch/sam3

  • Model author: Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, et al.

  • Model license: SAM License

  • Model size: 3.45 GB

  • Exposes embeddings? no

  • Tags: segment-anything, torch, zero-shot, transformer, official

Requirements

  • Packages: torch, torchvision, sam3

  • CPU support

    • yes

  • GPU support

    • yes

Example usage

 1import fiftyone as fo
 2import fiftyone.zoo as foz
 3
 4dataset = foz.load_zoo_dataset(
 5    "coco-2017",
 6    split="validation",
 7    dataset_name=fo.get_default_dataset_name(),
 8    max_samples=50,
 9    shuffle=True,
10)
11
12# Concept mode: find and segment all objects matching a text prompt
13model = foz.load_zoo_model(
14    "segment-anything-3-image-torch",
15    operation_mode="concept",
16    classes=["person", "car", "dog"],
17)
18dataset.apply_model(model, label_field="segmentations_concept")
19
20# Exemplar concept mode: find and segment object with text prompts and exemplar Detections
21model = foz.load_zoo_model(
22    "segment-anything-3-image-torch",
23    operation_mode="concept",
24    classes=["person"],
25)
26dataset.apply_model(
27        model,
28        label_field="segmentations_concept_with_exemplar",
29        prompt_field="person_detections", # contains exemplar Detections with positive / negative labels
30    )
31
32# Visual mode: segment inside boxes or using keypoints
33model = foz.load_zoo_model(
34    "segment-anything-3-image-torch",
35    operation_mode="visual",
36)
37dataset.apply_model(
38    model,
39    label_field="segmentations",
40    prompt_field="ground_truth",  # can contain Detections or Keypoints
41)
42
43session = fo.launch_app(dataset)