fc-clip-coco-instance-torch#

Open-vocabulary instance segmentation with FC-CLIP (COCO); returns only “thing” segments as instance masks. Uses frozen ConvNeXt-Large CLIP backbone + Mask2Former decoder..

Details

  • Model name: fc-clip-coco-instance-torch

  • Model source: https://huggingface.co/Voxel51/fc-clip

  • Model author: Jiarui Xu, et al. (ByteDance)

  • Model license: Apache 2.0

  • Model size: 79.16 MB

  • Exposes embeddings? no

  • Tags: instances, zero-shot, torch, transformers, official

Requirements

  • Packages: torch, torchvision, transformers, open_clip_torch, safetensors, timm>=1.0.17

  • CPU support

    • yes

  • GPU support

    • yes

Example usage

 1import fiftyone as fo
 2import fiftyone.zoo as foz
 3
 4dataset = foz.load_zoo_dataset(
 5    "coco-2017",
 6    split="validation",
 7    dataset_name=fo.get_default_dataset_name(),
 8    max_samples=50,
 9    shuffle=True,
10)
11
12
13classes = ["person", "dog", "cat", "bird", "car", "tree", "chair"]
14
15model = foz.load_zoo_model(
16    "fc-clip-coco-instance-torch",
17    classes=classes,
18)
19
20dataset.apply_model(model, label_field="predictions")
21
22session = fo.launch_app(dataset)