fiftyone.core.camera#

Camera calibration data model for multi-sensor geometry workflows.

This module provides first-class data models for camera intrinsics and static transforms (poses), enabling 3D-to-2D projection, 2D-to-3D unprojection, and multi-sensor fusion workflows.

Key classes:

For low-level transformation utilities (quaternion math, coordinate system conversions, matrix operations), see fiftyone.utils.transforms.

Copyright 2017-2026, Voxel51, Inc.
voxel51.com

Data:

CAMERA_CONVENTION_OPENCV

Supported 3D camera axis conventions

DEFAULT_TRANSFORM_TARGET_FRAME

Default target frame for static transforms

Classes:

ProjectionModel()

Abstract base class for camera projection models.

OpenCVProjectionModel()

Standard OpenCV camera model with radial and tangential distortion.

FisheyeProjectionModel()

OpenCV fisheye camera model with equidistant projection.

CameraIntrinsics(*args, **kwargs)

Base class for camera intrinsics.

PinholeCameraIntrinsics(*args, **kwargs)

Pinhole camera model with no distortion.

OpenCVCameraIntrinsics(*args, **kwargs)

OpenCV Brown-Conrady camera model with radial and tangential distortion.

OpenCVFisheyeCameraIntrinsics(*args, **kwargs)

OpenCV fisheye camera model with equidistant projection.

StaticTransform(*args, **kwargs)

Represents a rigid 3D transformation (6-DOF pose).

CameraIntrinsicsRef(*args, **kwargs)

Reference to dataset-level camera intrinsics.

StaticTransformRef(*args, **kwargs)

Reference to dataset-level static transform.

CameraProjector(intrinsics[, ...])

Utility class for projecting points between 3D and 2D.
fiftyone.core.camera.CAMERA_CONVENTION_OPENCV = 'opencv'#

Supported 3D camera axis conventions

fiftyone.core.camera.DEFAULT_TRANSFORM_TARGET_FRAME = 'world'#

Default target frame for static transforms

class fiftyone.core.camera.ProjectionModel#

Bases: ABC

Abstract base class for camera projection models.

Encapsulates projection and undistortion operations for different camera models.

Methods:

project(points, K, distortion)

Project 3D points to 2D image coordinates.

undistort(points, K, distortion)

Undistort 2D image points, returning normalized coordinates.

undistort_image(image, K, distortion[, ...])

Undistort an image using this projection model.
abstractmethod project(points: ndarray, K: ndarray, distortion: ndarray | None) → ndarray#

Project 3D points to 2D image coordinates.

Parameters:

  • points – (N, 3) array of 3D points in camera frame

  • K – (3, 3) intrinsic matrix

  • distortion – distortion coefficients, or None

Returns:

(N, 2) array of 2D pixel coordinates

abstractmethod undistort(points: ndarray, K: ndarray, distortion: ndarray | None) → ndarray#

Undistort 2D image points, returning normalized coordinates.

Parameters:

  • points – (N, 2) array of 2D pixel coordinates

  • K – (3, 3) intrinsic matrix

  • distortion – distortion coefficients, or None

Returns:

(N, 2) array of normalized coordinates (z=1 plane in camera frame)

abstractmethod undistort_image(image: ndarray, K: ndarray, distortion: ndarray | None, alpha: float = 0.0, new_size: Tuple[int, int] | None = None) → ndarray#

Undistort an image using this projection model.

class fiftyone.core.camera.OpenCVProjectionModel#

Bases: ProjectionModel

Standard OpenCV camera model with radial and tangential distortion.

Methods:

project(points, K, distortion)

Project points using OpenCV's standard camera model.

undistort(points, K, distortion)

Undistort points using OpenCV's standard camera model.

undistort_image(image, K, distortion[, ...])

Undistort an image using this projection model.
project(points: ndarray, K: ndarray, distortion: ndarray | None) → ndarray#

Project points using OpenCV’s standard camera model.

undistort(points: ndarray, K: ndarray, distortion: ndarray | None) → ndarray#

Undistort points using OpenCV’s standard camera model.

Returns normalized (x, y) coordinates (z=1 plane in camera frame).

undistort_image(image: ndarray, K: ndarray, distortion: ndarray | None, alpha: float = 0.0, new_size: Tuple[int, int] | None = None) → ndarray#

Undistort an image using this projection model.

class fiftyone.core.camera.FisheyeProjectionModel#

Bases: ProjectionModel

OpenCV fisheye camera model with equidistant projection.

Methods:

project(points, K, distortion)

Project points using OpenCV's fisheye camera model.

undistort(points, K, distortion)

Undistort points using OpenCV's fisheye camera model.

undistort_image(image, K, distortion[, ...])

Undistort an image using this projection model.
project(points: ndarray, K: ndarray, distortion: ndarray | None) → ndarray#

Project points using OpenCV’s fisheye camera model.

undistort(points: ndarray, K: ndarray, distortion: ndarray | None) → ndarray#

Undistort points using OpenCV’s fisheye camera model.

Returns normalized (x, y) coordinates (z=1 plane in camera frame).

undistort_image(image: ndarray, K: ndarray, distortion: ndarray | None, alpha: float = 0.0, new_size: Tuple[int, int] | None = None) → ndarray#

Undistort an image using this projection model.

class fiftyone.core.camera.CameraIntrinsics(*args, **kwargs)#

Bases: DynamicEmbeddedDocument

Base class for camera intrinsics.

All camera intrinsics models share the following parameters:

Parameters:

  • fx – focal length in pixels (x-axis)

  • fy – focal length in pixels (y-axis)

  • cx – principal point x-coordinate in pixels

  • cy – principal point y-coordinate in pixels

  • skew (0.0) – skew coefficient (typically 0 for modern cameras)

intrinsic_matrix#

the 3x3 intrinsic matrix K

Example:

import fiftyone as fo

intrinsics = fo.PinholeCameraIntrinsics(
    fx=1000.0,
    fy=1000.0,
    cx=960.0,
    cy=540.0,
)

# Access the 3x3 intrinsic matrix
K = intrinsics.intrinsic_matrix

Attributes:

fx

A floating point number field.

fy

A floating point number field.

cx

A floating point number field.

cy

A floating point number field.

skew

A floating point number field.

intrinsic_matrix

Returns the 3x3 intrinsic matrix K.

STRICT

field_names

An ordered tuple of the public fields of this document.

Methods:

from_matrix(matrix, **kwargs)

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

get_distortion_coeffs()

Returns the distortion coefficients for this camera model.

get_projection_model()

Returns the ProjectionModel instance for this camera model.

camera_matrix_3x4([transform])

Returns the 3x4 camera projection matrix P = K @ [R|t].

undistort_image(image[, alpha, new_size])

Undistort an image using this camera's intrinsics and distortion.

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name, select_fields, ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name, value)

from_dict(d[, extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

get_field(field_name)

Gets the field of the document.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[, merge_lists, merge_dicts, overwrite])

Merges the contents of the given document into this document.

set_field(field_name, value[, create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

validate([clean])

Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass
fx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

fy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

cx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

cy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

skew#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

property intrinsic_matrix: ndarray#

Returns the 3x3 intrinsic matrix K.

The matrix has the form:

[[fx,  s, cx],
 [ 0, fy, cy],
 [ 0,  0,  1]]
Returns:

a (3, 3) numpy array

classmethod from_matrix(matrix: ndarray, **kwargs) → CameraIntrinsics#

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

Parameters:

  • matrix – a (3, 3) intrinsic matrix K

  • **kwargs – additional fields to set on the instance

Returns:

a CameraIntrinsics instance

get_distortion_coeffs() → ndarray | None#

Returns the distortion coefficients for this camera model.

Returns:

a numpy array of distortion coefficients, or None if no distortion

get_projection_model() → ProjectionModel#

Returns the ProjectionModel instance for this camera model.

Returns:

a ProjectionModel instance

camera_matrix_3x4(transform: StaticTransform | None = None) → ndarray#

Returns the 3x4 camera projection matrix P = K @ [R|t].

Note: This matrix is only valid when distortion is zero or has been pre-corrected in the image.

Parameters:

transform – optional transform defining the world-to-camera transformation (i.e., source_frame=world, target_frame=camera). If your transform is camera-to-world, call transform.inverse() first.

Returns:

a (3, 4) numpy array

undistort_image(image: ndarray, alpha: float = 0.0, new_size: Tuple[int, int] | None = None) → ndarray#

Undistort an image using this camera’s intrinsics and distortion.

Removes lens distortion from an image, producing a rectified image that follows the pinhole camera model.

Parameters:

  • image – input distorted image as a numpy array with shape (H, W) for grayscale or (H, W, C) for color images

  • alpha –

    free scaling parameter between 0 and 1:

    • 0: returns undistorted image with all pixels valid (cropped to remove black borders)

    • 1: retains all source image pixels (may have black borders where no source data exists)

    Intermediate values blend between the two extremes

  • new_size – optional output image size as (width, height). If None, uses the input image size

Returns:

undistorted image as a numpy array with the same dtype as input

Example:

import fiftyone as fo
import cv2

intrinsics = fo.OpenCVCameraIntrinsics(
    fx=1000.0, fy=1000.0, cx=960.0, cy=540.0,
    k1=-0.1, k2=0.05,
)

distorted = cv2.imread("distorted.jpg")
rectified = intrinsics.undistort_image(distorted)

# Keep all pixels (with black borders)
rectified_full = intrinsics.undistort_image(distorted, alpha=1.0)
STRICT = False#
clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name – the field name

Raises:

ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

  • class_name (None) – optional class name to use

  • select_fields (None) – iterable of field names to restrict to

  • exclude_fields (None) – iterable of field names to exclude

  • **kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

  • d – a dictionary

  • extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name – the field name

Returns:

the field value

Raises:

AttributeError – if the field does not exist

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name – the field name

Returns:

True/False

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

  • doc – a SerializableDocument of same type as this document

  • merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

  • field_name – the field name

  • value – the field value

  • create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

validate(clean=True)#

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.camera.PinholeCameraIntrinsics(*args, **kwargs)#

Bases: CameraIntrinsics

Pinhole camera model with no distortion.

Parameters:

  • fx – focal length in pixels (x-axis)

  • fy – focal length in pixels (y-axis)

  • cx – principal point x-coordinate in pixels

  • cy – principal point y-coordinate in pixels

  • skew (0.0) – skew coefficient (typically 0 for modern cameras)

Example:

import fiftyone as fo

intrinsics = fo.PinholeCameraIntrinsics(
    fx=1000.0,
    fy=1000.0,
    cx=960.0,
    cy=540.0,
)

Attributes:

STRICT

cx

A floating point number field.

cy

A floating point number field.

field_names

An ordered tuple of the public fields of this document.

fx

A floating point number field.

fy

A floating point number field.

intrinsic_matrix

Returns the 3x3 intrinsic matrix K.

skew

A floating point number field.

Methods:

camera_matrix_3x4([transform])

Returns the 3x4 camera projection matrix P = K @ [R|t].

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name, select_fields, ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name, value)

from_dict(d[, extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

from_matrix(matrix, **kwargs)

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

get_distortion_coeffs()

Returns the distortion coefficients for this camera model.

get_field(field_name)

Gets the field of the document.

get_projection_model()

Returns the ProjectionModel instance for this camera model.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[, merge_lists, merge_dicts, overwrite])

Merges the contents of the given document into this document.

set_field(field_name, value[, create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

undistort_image(image[, alpha, new_size])

Undistort an image using this camera's intrinsics and distortion.

validate([clean])

Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass
STRICT = False#
camera_matrix_3x4(transform: StaticTransform | None = None) → ndarray#

Returns the 3x4 camera projection matrix P = K @ [R|t].

Note: This matrix is only valid when distortion is zero or has been pre-corrected in the image.

Parameters:

transform – optional transform defining the world-to-camera transformation (i.e., source_frame=world, target_frame=camera). If your transform is camera-to-world, call transform.inverse() first.

Returns:

a (3, 4) numpy array

clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name – the field name

Raises:

ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

cx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

cy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

  • class_name (None) – optional class name to use

  • select_fields (None) – iterable of field names to restrict to

  • exclude_fields (None) – iterable of field names to exclude

  • **kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

  • d – a dictionary

  • extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

classmethod from_matrix(matrix: ndarray, **kwargs) → CameraIntrinsics#

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

Parameters:

  • matrix – a (3, 3) intrinsic matrix K

  • **kwargs – additional fields to set on the instance

Returns:

a CameraIntrinsics instance

fx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

fy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

get_distortion_coeffs() → ndarray | None#

Returns the distortion coefficients for this camera model.

Returns:

a numpy array of distortion coefficients, or None if no distortion

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name – the field name

Returns:

the field value

Raises:

AttributeError – if the field does not exist

get_projection_model() → ProjectionModel#

Returns the ProjectionModel instance for this camera model.

Returns:

a ProjectionModel instance

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name – the field name

Returns:

True/False

property intrinsic_matrix: ndarray#

Returns the 3x3 intrinsic matrix K.

The matrix has the form:

[[fx,  s, cx],
 [ 0, fy, cy],
 [ 0,  0,  1]]
Returns:

a (3, 3) numpy array

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

  • doc – a SerializableDocument of same type as this document

  • merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

  • field_name – the field name

  • value – the field value

  • create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

skew#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

undistort_image(image: ndarray, alpha: float = 0.0, new_size: Tuple[int, int] | None = None) → ndarray#

Undistort an image using this camera’s intrinsics and distortion.

Removes lens distortion from an image, producing a rectified image that follows the pinhole camera model.

Parameters:

  • image – input distorted image as a numpy array with shape (H, W) for grayscale or (H, W, C) for color images

  • alpha –

    free scaling parameter between 0 and 1:

    • 0: returns undistorted image with all pixels valid (cropped to remove black borders)

    • 1: retains all source image pixels (may have black borders where no source data exists)

    Intermediate values blend between the two extremes

  • new_size – optional output image size as (width, height). If None, uses the input image size

Returns:

undistorted image as a numpy array with the same dtype as input

Example:

import fiftyone as fo
import cv2

intrinsics = fo.OpenCVCameraIntrinsics(
    fx=1000.0, fy=1000.0, cx=960.0, cy=540.0,
    k1=-0.1, k2=0.05,
)

distorted = cv2.imread("distorted.jpg")
rectified = intrinsics.undistort_image(distorted)

# Keep all pixels (with black borders)
rectified_full = intrinsics.undistort_image(distorted, alpha=1.0)
validate(clean=True)#

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.camera.OpenCVCameraIntrinsics(*args, **kwargs)#

Bases: CameraIntrinsics

OpenCV Brown-Conrady camera model with radial and tangential distortion.

Distortion coefficients follow the OpenCV ordering: (k1, k2, p1, p2, k3, k4, k5, k6).

The radial distortion model is:

x_distorted = x * (1 + k1*r^2 + k2*r^4 + k3*r^6) /
                  (1 + k4*r^2 + k5*r^4 + k6*r^6)

The tangential distortion model is:

x_distorted += 2*p1*x*y + p2*(r^2 + 2*x^2)
y_distorted += p1*(r^2 + 2*y^2) + 2*p2*x*y
Parameters:

  • fx – focal length in pixels (x-axis)

  • fy – focal length in pixels (y-axis)

  • cx – principal point x-coordinate in pixels

  • cy – principal point y-coordinate in pixels

  • skew (0.0) – skew coefficient

  • k1 (0.0) – radial distortion coefficient

  • k2 (0.0) – radial distortion coefficient

  • p1 (0.0) – tangential distortion coefficient

  • p2 (0.0) – tangential distortion coefficient

  • k3 (0.0) – radial distortion coefficient

  • k4 (0.0) – radial distortion coefficient (rational model)

  • k5 (0.0) – radial distortion coefficient (rational model)

  • k6 (0.0) – radial distortion coefficient (rational model)

Example:

import fiftyone as fo

intrinsics = fo.OpenCVCameraIntrinsics(
    fx=1000.0,
    fy=1000.0,
    cx=960.0,
    cy=540.0,
    k1=-0.1,
    k2=0.05,
    p1=0.001,
    p2=-0.001,
)

Attributes:

k1

A floating point number field.

k2

A floating point number field.

p1

A floating point number field.

p2

A floating point number field.

k3

A floating point number field.

k4

A floating point number field.

k5

A floating point number field.

k6

A floating point number field.

STRICT

cx

A floating point number field.

cy

A floating point number field.

field_names

An ordered tuple of the public fields of this document.

fx

A floating point number field.

fy

A floating point number field.

intrinsic_matrix

Returns the 3x3 intrinsic matrix K.

skew

A floating point number field.

Methods:

get_distortion_coeffs()

Returns the OpenCV distortion coefficients.

camera_matrix_3x4([transform])

Returns the 3x4 camera projection matrix P = K @ [R|t].

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name, select_fields, ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name, value)

from_dict(d[, extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

from_matrix(matrix, **kwargs)

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

get_field(field_name)

Gets the field of the document.

get_projection_model()

Returns the ProjectionModel instance for this camera model.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[, merge_lists, merge_dicts, overwrite])

Merges the contents of the given document into this document.

set_field(field_name, value[, create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

undistort_image(image[, alpha, new_size])

Undistort an image using this camera's intrinsics and distortion.

validate([clean])

Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass
k1#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k2#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

p1#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

p2#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k3#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k4#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k5#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k6#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

get_distortion_coeffs() → ndarray#

Returns the OpenCV distortion coefficients.

Returns:

a (8,) numpy array with coefficients (k1, k2, p1, p2, k3, k4, k5, k6)

STRICT = False#
camera_matrix_3x4(transform: StaticTransform | None = None) → ndarray#

Returns the 3x4 camera projection matrix P = K @ [R|t].

Note: This matrix is only valid when distortion is zero or has been pre-corrected in the image.

Parameters:

transform – optional transform defining the world-to-camera transformation (i.e., source_frame=world, target_frame=camera). If your transform is camera-to-world, call transform.inverse() first.

Returns:

a (3, 4) numpy array

clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name – the field name

Raises:

ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

cx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

cy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

  • class_name (None) – optional class name to use

  • select_fields (None) – iterable of field names to restrict to

  • exclude_fields (None) – iterable of field names to exclude

  • **kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

  • d – a dictionary

  • extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

classmethod from_matrix(matrix: ndarray, **kwargs) → CameraIntrinsics#

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

Parameters:

  • matrix – a (3, 3) intrinsic matrix K

  • **kwargs – additional fields to set on the instance

Returns:

a CameraIntrinsics instance

fx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

fy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name – the field name

Returns:

the field value

Raises:

AttributeError – if the field does not exist

get_projection_model() → ProjectionModel#

Returns the ProjectionModel instance for this camera model.

Returns:

a ProjectionModel instance

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name – the field name

Returns:

True/False

property intrinsic_matrix: ndarray#

Returns the 3x3 intrinsic matrix K.

The matrix has the form:

[[fx,  s, cx],
 [ 0, fy, cy],
 [ 0,  0,  1]]
Returns:

a (3, 3) numpy array

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

  • doc – a SerializableDocument of same type as this document

  • merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

  • field_name – the field name

  • value – the field value

  • create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

skew#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

undistort_image(image: ndarray, alpha: float = 0.0, new_size: Tuple[int, int] | None = None) → ndarray#

Undistort an image using this camera’s intrinsics and distortion.

Removes lens distortion from an image, producing a rectified image that follows the pinhole camera model.

Parameters:

  • image – input distorted image as a numpy array with shape (H, W) for grayscale or (H, W, C) for color images

  • alpha –

    free scaling parameter between 0 and 1:

    • 0: returns undistorted image with all pixels valid (cropped to remove black borders)

    • 1: retains all source image pixels (may have black borders where no source data exists)

    Intermediate values blend between the two extremes

  • new_size – optional output image size as (width, height). If None, uses the input image size

Returns:

undistorted image as a numpy array with the same dtype as input

Example:

import fiftyone as fo
import cv2

intrinsics = fo.OpenCVCameraIntrinsics(
    fx=1000.0, fy=1000.0, cx=960.0, cy=540.0,
    k1=-0.1, k2=0.05,
)

distorted = cv2.imread("distorted.jpg")
rectified = intrinsics.undistort_image(distorted)

# Keep all pixels (with black borders)
rectified_full = intrinsics.undistort_image(distorted, alpha=1.0)
validate(clean=True)#

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.camera.OpenCVFisheyeCameraIntrinsics(*args, **kwargs)#

Bases: CameraIntrinsics

OpenCV fisheye camera model with equidistant projection.

Uses 4 distortion coefficients (k1, k2, k3, k4) for the fisheye model.

Parameters:

  • fx – focal length in pixels (x-axis)

  • fy – focal length in pixels (y-axis)

  • cx – principal point x-coordinate in pixels

  • cy – principal point y-coordinate in pixels

  • skew (0.0) – skew coefficient

  • k1 (0.0) – fisheye distortion coefficient

  • k2 (0.0) – fisheye distortion coefficient

  • k3 (0.0) – fisheye distortion coefficient

  • k4 (0.0) – fisheye distortion coefficient

Example:

import fiftyone as fo

intrinsics = fo.OpenCVFisheyeCameraIntrinsics(
    fx=500.0,
    fy=500.0,
    cx=640.0,
    cy=480.0,
    k1=0.1,
    k2=-0.05,
)

Attributes:

k1

A floating point number field.

k2

A floating point number field.

k3

A floating point number field.

k4

A floating point number field.

STRICT

cx

A floating point number field.

cy

A floating point number field.

field_names

An ordered tuple of the public fields of this document.

fx

A floating point number field.

fy

A floating point number field.

intrinsic_matrix

Returns the 3x3 intrinsic matrix K.

skew

A floating point number field.

Methods:

get_distortion_coeffs()

Returns the fisheye distortion coefficients.

get_projection_model()

Returns the ProjectionModel instance for fisheye distortion.

camera_matrix_3x4([transform])

Returns the 3x4 camera projection matrix P = K @ [R|t].

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name, select_fields, ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name, value)

from_dict(d[, extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

from_matrix(matrix, **kwargs)

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

get_field(field_name)

Gets the field of the document.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[, merge_lists, merge_dicts, overwrite])

Merges the contents of the given document into this document.

set_field(field_name, value[, create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

undistort_image(image[, alpha, new_size])

Undistort an image using this camera's intrinsics and distortion.

validate([clean])

Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass
k1#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k2#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k3#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

k4#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

get_distortion_coeffs() → ndarray#

Returns the fisheye distortion coefficients.

Returns:

a (4,) numpy array with coefficients (k1, k2, k3, k4)

get_projection_model() → ProjectionModel#

Returns the ProjectionModel instance for fisheye distortion.

Returns:

a FisheyeProjectionModel instance

STRICT = False#
camera_matrix_3x4(transform: StaticTransform | None = None) → ndarray#

Returns the 3x4 camera projection matrix P = K @ [R|t].

Note: This matrix is only valid when distortion is zero or has been pre-corrected in the image.

Parameters:

transform – optional transform defining the world-to-camera transformation (i.e., source_frame=world, target_frame=camera). If your transform is camera-to-world, call transform.inverse() first.

Returns:

a (3, 4) numpy array

clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name – the field name

Raises:

ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

cx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

cy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

  • class_name (None) – optional class name to use

  • select_fields (None) – iterable of field names to restrict to

  • exclude_fields (None) – iterable of field names to exclude

  • **kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

  • d – a dictionary

  • extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

classmethod from_matrix(matrix: ndarray, **kwargs) → CameraIntrinsics#

Creates a CameraIntrinsics instance from a 3x3 intrinsic matrix.

Parameters:

  • matrix – a (3, 3) intrinsic matrix K

  • **kwargs – additional fields to set on the instance

Returns:

a CameraIntrinsics instance

fx#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

fy#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name – the field name

Returns:

the field value

Raises:

AttributeError – if the field does not exist

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name – the field name

Returns:

True/False

property intrinsic_matrix: ndarray#

Returns the 3x3 intrinsic matrix K.

The matrix has the form:

[[fx,  s, cx],
 [ 0, fy, cy],
 [ 0,  0,  1]]
Returns:

a (3, 3) numpy array

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

  • doc – a SerializableDocument of same type as this document

  • merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

  • field_name – the field name

  • value – the field value

  • create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

skew#

A floating point number field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

undistort_image(image: ndarray, alpha: float = 0.0, new_size: Tuple[int, int] | None = None) → ndarray#

Undistort an image using this camera’s intrinsics and distortion.

Removes lens distortion from an image, producing a rectified image that follows the pinhole camera model.

Parameters:

  • image – input distorted image as a numpy array with shape (H, W) for grayscale or (H, W, C) for color images

  • alpha –

    free scaling parameter between 0 and 1:

    • 0: returns undistorted image with all pixels valid (cropped to remove black borders)

    • 1: retains all source image pixels (may have black borders where no source data exists)

    Intermediate values blend between the two extremes

  • new_size – optional output image size as (width, height). If None, uses the input image size

Returns:

undistorted image as a numpy array with the same dtype as input

Example:

import fiftyone as fo
import cv2

intrinsics = fo.OpenCVCameraIntrinsics(
    fx=1000.0, fy=1000.0, cx=960.0, cy=540.0,
    k1=-0.1, k2=0.05,
)

distorted = cv2.imread("distorted.jpg")
rectified = intrinsics.undistort_image(distorted)

# Keep all pixels (with black borders)
rectified_full = intrinsics.undistort_image(distorted, alpha=1.0)
validate(clean=True)#

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.camera.StaticTransform(*args, **kwargs)#

Bases: DynamicEmbeddedDocument

Represents a rigid 3D transformation (6-DOF pose).

Stored as translation + quaternion for efficiency. Defines transformation from source_frame to target_frame:

X_target = R @ X_source + t

The quaternion uses scalar-last convention [qx, qy, qz, qw], matching scipy and ROS conventions.

Parameters:

  • source_frame – name of source coordinate frame (e.g., “camera_front”). This is a required argument.

  • translation ([0, 0, 0]) – 3-element list [tx, ty, tz] (position in target frame)

  • quaternion ([0, 0, 0, 1]) – unit quaternion [qx, qy, qz, qw] (scalar-last convention, defaults to identity rotation)

  • target_frame (None) – name of target coordinate frame (e.g., “ego”, “world”)

  • timestamp (None) – optional timestamp in nanoseconds for interpolation

  • covariance (None) – optional 6-element diagonal pose uncertainty [σx, σy, σz, σroll, σpitch, σyaw] where translations are in metric and rotations are in radians

rotation_matrix#

the 3x3 rotation matrix R

transform_matrix#

the 4x4 homogeneous transformation matrix

Example:

import fiftyone as fo

# Camera to ego transformation
transform = fo.StaticTransform(
    translation=[1.5, 0.0, 1.2],
    # identity rotation
    quaternion=[0.0, 0.0, 0.0, 1.0],
    source_frame="camera_front",
    target_frame="ego",
)

# Access the 4x4 transformation matrix
T = transform.transform_matrix

Attributes:

translation

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

quaternion

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

source_frame

A unicode string field.

target_frame

A unicode string field.

timestamp

A 32 bit integer field.

covariance

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

rotation_matrix

Returns the 3x3 rotation matrix R.

transform_matrix

Returns the 4x4 homogeneous transformation matrix.

STRICT

field_names

An ordered tuple of the public fields of this document.

Methods:

validate([clean])

Validates the transform data.

from_matrix(matrix[, source_frame, target_frame])

Creates a StaticTransform instance from a 3x4 or 4x4 matrix.

inverse()

Returns the inverse transformation.

compose(other)

Composes this transform with another.

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name, select_fields, ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name, value)

from_dict(d[, extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

get_field(field_name)

Gets the field of the document.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[, merge_lists, merge_dicts, overwrite])

Merges the contents of the given document into this document.

set_field(field_name, value[, create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

Classes:

my_metaclass

alias of DocumentMetaclass
translation#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:

  • field (None) – an optional Field instance describing the type of the list elements

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

quaternion#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:

  • field (None) – an optional Field instance describing the type of the list elements

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

source_frame#

A unicode string field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

target_frame#

A unicode string field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

timestamp#

A 32 bit integer field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

covariance#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:

  • field (None) – an optional Field instance describing the type of the list elements

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

validate(clean=True)#

Validates the transform data.

This method is called by mongoengine during save/validation.

property rotation_matrix: ndarray#

Returns the 3x3 rotation matrix R.

Returns:

a (3, 3) numpy array

property transform_matrix: ndarray#

Returns the 4x4 homogeneous transformation matrix.

The matrix has the form:

[[R, t],
 [0, 1]]

where R is the 3x3 rotation and t is the 3x1 translation.

Returns:

a (4, 4) numpy array

classmethod from_matrix(matrix: ndarray, source_frame: str | None = None, target_frame: str | None = None, **kwargs) → StaticTransform#

Creates a StaticTransform instance from a 3x4 or 4x4 matrix.

Parameters:

  • matrix – a (3, 4) or (4, 4) transformation matrix [R|t]

  • source_frame – name of source coordinate frame

  • target_frame – name of target coordinate frame

  • **kwargs – additional fields to set on the instance

Returns:

a StaticTransform instance

inverse() → StaticTransform#

Returns the inverse transformation.

If this transform is source_frame -> target_frame, the inverse is target_frame -> source_frame.

Returns:

a StaticTransform representing the inverse transform

compose(other: StaticTransform) → StaticTransform#

Composes this transform with another.

If self is A->B and other is B->C, the result is A->C.

Mathematically:

X_C = T_BC @ X_B = T_BC @ (T_AB @ X_A) So T_AC = T_BC @ T_AB (other @ self)

Parameters:

other – another StaticTransform to compose with. The source_frame of other should match target_frame of self for the frames to chain correctly.

Returns:

a StaticTransform representing the composed transform

Raises:

ValueError – if the frames don’t chain (self.target_frame != other.source_frame when both are specified)

STRICT = False#
clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name – the field name

Raises:

ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

  • class_name (None) – optional class name to use

  • select_fields (None) – iterable of field names to restrict to

  • exclude_fields (None) – iterable of field names to exclude

  • **kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

  • d – a dictionary

  • extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name – the field name

Returns:

the field value

Raises:

AttributeError – if the field does not exist

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name – the field name

Returns:

True/False

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

  • doc – a SerializableDocument of same type as this document

  • merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

  • field_name – the field name

  • value – the field value

  • create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

class fiftyone.core.camera.CameraIntrinsicsRef(*args, **kwargs)#

Bases: EmbeddedDocument

Reference to dataset-level camera intrinsics.

Use this to reference intrinsics stored at the dataset level rather than embedding the full intrinsics data in each sample.

Parameters:

ref – the sensor/camera name key in dataset.camera_intrinsics

Example:

import fiftyone as fo

# Reference dataset-level intrinsics (field name can be anything)
sample["intrinsics"] = fo.CameraIntrinsicsRef(ref="camera_front")

Attributes:

ref

A unicode string field.

STRICT

field_names

An ordered tuple of the public fields of this document.

Methods:

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name, select_fields, ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name, value)

from_dict(d[, extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

get_field(field_name)

Gets the field of the document.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[, merge_lists, merge_dicts, overwrite])

Merges the contents of the given document into this document.

set_field(field_name, value[, create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

validate([clean])

Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass
ref#

A unicode string field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

STRICT = False#
clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name – the field name

Raises:

ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

  • class_name (None) – optional class name to use

  • select_fields (None) – iterable of field names to restrict to

  • exclude_fields (None) – iterable of field names to exclude

  • **kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

  • d – a dictionary

  • extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name – the field name

Returns:

the field value

Raises:

AttributeError – if the field does not exist

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name – the field name

Returns:

True/False

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

  • doc – a SerializableDocument of same type as this document

  • merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

  • field_name – the field name

  • value – the field value

  • create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

validate(clean=True)#

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.camera.StaticTransformRef(*args, **kwargs)#

Bases: EmbeddedDocument

Reference to dataset-level static transform.

Use this to reference transforms stored at the dataset level rather than embedding the full transform data in each sample.

Parameters:

ref – the key in dataset.static_transforms, either “source_frame::target_frame” or just “source_frame” (implies target is “world”)

Example:

import fiftyone as fo

# Reference dataset-level transform (field name can be anything)
sample["transform"] = [
    fo.StaticTransformRef(ref="camera_front::ego"),
]

Attributes:

ref

A unicode string field.

STRICT

field_names

An ordered tuple of the public fields of this document.

Methods:

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name, select_fields, ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name, value)

from_dict(d[, extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

get_field(field_name)

Gets the field of the document.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[, merge_lists, merge_dicts, overwrite])

Merges the contents of the given document into this document.

set_field(field_name, value[, create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

validate([clean])

Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass
ref#

A unicode string field.

Parameters:

  • description (None) – an optional description

  • info (None) – an optional info dict

  • read_only (False) – whether the field is read-only

  • created_at (None) – the datetime the field was created

STRICT = False#
clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name – the field name

Raises:

ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

  • class_name (None) – optional class name to use

  • select_fields (None) – iterable of field names to restrict to

  • exclude_fields (None) – iterable of field names to exclude

  • **kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

  • d – a dictionary

  • extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name – the field name

Returns:

the field value

Raises:

AttributeError – if the field does not exist

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name – the field name

Returns:

True/False

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

  • doc – a SerializableDocument of same type as this document

  • merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

  • field_name – the field name

  • value – the field value

  • create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

validate(clean=True)#

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.camera.CameraProjector(intrinsics: CameraIntrinsics, camera_to_reference: StaticTransform | None = None, camera_convention: str = 'opencv')#

Bases: object

Utility class for projecting points between 3D and 2D.

Combines camera intrinsics and optional transforms to perform projection and unprojection operations.

Parameters:

  • intrinsics – a CameraIntrinsics instance

  • camera_to_reference (None) –

    optional StaticTransform defining the camera-to-reference transformation (i.e., the camera’s pose in the reference frame). If provided, 3D points passed to project() are assumed to be in the reference frame (camera_to_reference.target_frame) and will be transformed to camera frame before projection.

    The transform should have:

    • source_frame: the camera/sensor name (e.g., “cam_front”)

    • target_frame: the reference frame (e.g., “world”, “ego”)

  • camera_convention ("opencv") – 3D camera axis convention, either “opencv” (z-forward, x-right, y-down) or “opengl” (z-backward, x-right, y-up). Note: This only affects the 3D coordinate axes. Pixel coordinates always follow image-space convention with +x right, +y down, origin at top-left

Important

Transform direction: This class expects camera-to-reference transforms, NOT reference-to-camera. If you have a reference-to-camera transform (e.g., world-to-camera), invert it first:

projector = fo.CameraProjector(intrinsics, world_to_cam.inverse())

Or use the from_reference_to_camera() constructor.

Example:

import fiftyone as fo
import numpy as np

intrinsics = fo.PinholeCameraIntrinsics(
    fx=1000.0, fy=1000.0, cx=960.0, cy=540.0
)

# Project points already in camera frame
projector = fo.CameraProjector(intrinsics)
points_3d = np.array([[0, 0, 10], [1, 2, 10]])
points_2d = projector.project(points_3d, in_camera_frame=True)

# Project world points using camera-to-world transform
cam_to_world = fo.StaticTransform(
    translation=[0.0, 0.0, 0.0],
    quaternion=[0.0, 0.0, 0.0, 1.0],
    source_frame="camera",
    target_frame="world",
)
projector = fo.CameraProjector(intrinsics, cam_to_world)
world_points = np.array([[0, 0, 10]])
pixels = projector.project(world_points, in_camera_frame=False)

Methods:

from_reference_to_camera(intrinsics, ...[, ...])

Creates a CameraProjector from reference-to-camera transform.

project(points_3d[, in_camera_frame])

Projects 3D points to 2D image coordinates.

unproject(points_2d, depth[, in_camera_frame])

Unprojects 2D image points to 3D given depth.
classmethod from_reference_to_camera(intrinsics: CameraIntrinsics, reference_to_camera: StaticTransform, camera_convention: str = 'opencv') → CameraProjector#

Creates a CameraProjector from reference-to-camera transform.

Use this constructor if your transform converts points FROM the reference frame TO the camera frame (e.g., world-to-camera). This is common when loading from some datasets or calibration tools.

This method automatically inverts the transform to the expected camera-to-reference format.

Parameters:

  • intrinsics – a CameraIntrinsics instance

  • reference_to_camera – a StaticTransform that transforms points from the reference frame to the camera frame

  • camera_convention – “opencv” or “opengl”

Returns:

a CameraProjector instance

Example:

# If you have world-to-camera transform:
world_to_cam = fo.StaticTransform(
    translation=[...],
    quaternion=[...],
    source_frame="world",
    target_frame="camera",
)
projector = fo.CameraProjector.from_reference_to_camera(
    intrinsics, world_to_cam
)
project(points_3d: ndarray, in_camera_frame: bool = False) → ndarray#

Projects 3D points to 2D image coordinates.

Parameters:

  • points_3d – (N, 3) array of 3D points

  • in_camera_frame – if True, points are already in camera frame; if False and camera_to_reference is provided, points are transformed from the reference frame to camera frame

Returns:

(N, 2) array of 2D pixel coordinates

unproject(points_2d: ndarray, depth: float | ndarray, in_camera_frame: bool = False) → ndarray#

Unprojects 2D image points to 3D given depth.

Note: For monocular cameras, depth must be provided from an external source (e.g., stereo, LiDAR, depth sensor, or depth estimation).

The depth is interpreted as z-depth in the camera coordinate frame (not Euclidean distance from camera center).

Parameters:

  • points_2d – (N, 2) array of 2D pixel coordinates

  • depth – scalar or (N,) array of z-depth values in camera frame

  • in_camera_frame – if True, returns points in camera frame; if False and camera_to_reference is provided, transforms to the reference frame

Returns:

(N, 3) array of 3D points

IN THIS ARTICLE