llamaindex/vdr-2b-multi-v1

Note

This is a remotely-sourced model from the visual_document_retrieval plugin, maintained by the community. It is not part of FiftyOne core and may have special installation requirements. Please review the plugin documentation and license before use.

vdr-2b-multi-v1 is a multilingual embedding model designed for visual document retrieval across multiple languages and domains. It encodes document page screenshots into dense single-vector representations, this will effectively allow to search and query visually rich multilingual documents without the need for any OCR, data extraction pipelines, and chunking. It’s trained on 🇮🇹 Italian, 🇪🇸 Spanish, 🇬🇧 English, 🇫🇷 French and 🇩🇪 German.

Details

• Model name: llamaindex/vdr-2b-multi-v1

• Model source: https://huggingface.co/llamaindex/vdr-2b-multi-v1

• Model author: LlamaIndex

• Model license: Apache-2.0 license

• Exposes embeddings? yes

• Tags: embeddings, ocr, VLM, document-retrieval

Requirements

• Packages: huggingface-hub, transformers, torch, torchvision, qwen-vl-utils, accelerate, autoawq==0.2.7.post3

• CPU support

  • yes

• GPU support

  • yes

Example usage

import fiftyone as fo
import fiftyone.zoo as foz

foz.register_zoo_model_source("https://github.com/harpreetsahota204/visual_document_retrieval")

dataset = foz.load_zoo_dataset(
    "coco-2017",
    split="validation",
    dataset_name=fo.get_default_dataset_name(),
    max_samples=50,
    shuffle=True,
)

model = foz.load_zoo_model("llamaindex/vdr-2b-multi-v1")

dataset.apply_model(model, label_field="predictions")

session = fo.launch_app(dataset)