twelvelabs-marengo3.0

An enhanced multimodal embedding model that extends the capabilities of Marengo 2.7 with support for text and image interleaved input modality. This model generates high-quality vector representations of video, text, audio, image, and interleaved text-image content for similarity search, clustering, and other machine learning tasks.

Details

• Model name: twelvelabs-marengo3.0

• Model source: https://www.twelvelabs.io

• Model author: Twelve Labs, Inc.

• Model license: https://www.twelvelabs.io/legal/tos

• Exposes embeddings? yes

• Tags: video, embeddings, official

Requirements

• Packages: twelvelabs>=1.2.8

• CPU support

  • N/A

• GPU support

  • N/A

Example usage

import fiftyone as fo
import fiftyone.zoo as foz
import fiftyone.brain as fob

dataset = foz.load_zoo_dataset("quickstart-video")

model = foz.load_zoo_model("twelvelabs-marengo3.0")

dataset.apply_model(model, label_field="predictions")

dataset.compute_embeddings(model, embeddings_field="twelvelabs")

# Text-to-video search
fob.compute_similarity(
    dataset,
    model=model,
    embeddings="twelvelabs",
    brain_key="tl_sim",
)

view = dataset.sort_by_similarity(
    "a person riding a bike",
    brain_key="tl_sim",
    k=10,
)

session = fo.launch_app(view)