fiftyone.utils.utils3d#

3D utilities.

Copyright 2017-2022, Voxel51, Inc.
voxel51.com

Functions:

`compute_cuboid_iou`(gt, pred[, gt_crowd])	Computes the IoU between the given ground truth and predicted cuboids.
`rpy_to_rotation`(euler_rpy)	Converts Euler angles in roll-pitch-yaw order to a scipy.spatial.transform Rotation.
`multiple_coordinate_transform`(points, ...[, ...])	Applies a sequence of 3D coordinate frame transformations to a point and its orientation.
`single_coordinate_transform`(points, rot, ...)	Applies a single 3D coordinate frame transformation to a point and its orientation.
`corners_from_euler`(location, rotation, dimension)	Computes the 3D corners of a cuboid given its location, rotation, and dimensions.
`pinhole_projector`(points, cam_params[, ...])	Projects 3D detection points to 2D using the given camera intrinsics assuming a pinhole camera.
`point_in_front_of_camera`(corners_img, ...[, ...])	Checks if the input corners are visible in the image and in front of the camera.
`get_scene_asset_paths`(scene_paths[, ...])	Extracts all asset paths for the specified 3D scenes.
`compute_orthographic_projection_images`(...)	Computes orthographic projection images for the point clouds in the given collection.
`compute_orthographic_projection_image`(...[, ...])	Generates an orthographic projection image for the given PCD file onto the specified plane (default xy plane).
`pcd_to_3d`(dataset[, slices, output_dir, ...])	Converts the point cloud samples in the given dataset to 3D samples.

Classes:

OrthographicProjectionMetadata(*args, **kwargs)

Class for storing metadata about orthographic projections.

fiftyone.utils.utils3d.compute_cuboid_iou(gt, pred, gt_crowd=False)#

Computes the IoU between the given ground truth and predicted cuboids.

Parameters:

gt – a fiftyone.core.labels.Detection
pred – a fiftyone.core.labels.Detection
gt_crowd (False) – whether the ground truth cuboid is a crowd

Returns:

the IoU, in [0, 1]

fiftyone.utils.utils3d.rpy_to_rotation(euler_rpy: List[float])#

Converts Euler angles in roll-pitch-yaw order to a scipy.spatial.transform Rotation.

Parameters:: euler_rpy – a list of Euler angles in roll-pitch-yaw order
Returns:: A scipy.spatial.transform Rotation.

fiftyone.utils.utils3d.multiple_coordinate_transform(points: List[float], euler_rpy: List[float], transformation_sequence: List[Tuple[list[float] | ndarray, list[list[float]] | ndarray]], forward_transform_flags: List[bool] = None) → Tuple[List[float], List[float]]#

Applies a sequence of 3D coordinate frame transformations to a point and its orientation. Each transformation consists of a translation vector and a rotation matrix, applied in the order provided. The orientation is updated at each step using quaternion multiplication.

Parameters:

points – A 3-element list/array representing the (x, y, z) coordinates of the point
euler_rpy – A 3-element list of Euler angles [roll, pitch, yaw] in radians
transformation_sequence – A list of (translation, rotation) tuples: - translation: 3-element vector (tx, ty, tz) - rotation: (3, 3) rotation matrix
forward_transform_flags (None) – One per transformation True means apply the transform source → target False means apply the inverse (target → source). Defaults to all True

Returns:

Transformed 3D point [x, y, z].
Updated orientation as Euler angles [roll, pitch, yaw] in radians.

fiftyone.utils.utils3d.single_coordinate_transform(points: ndarray, rot: Any, transformation: Tuple[ndarray, ndarray], forward_transform: bool = True) → Tuple[ndarray, Any]#

Applies a single 3D coordinate frame transformation to a point and its orientation.

The transformation consists of a translation vector and a rotation matrix. The orientation is updated using quaternion multiplication.

Parameters:

points – A 3-element np.ndarray representing the (x, y, z) coordinates of the point
rot – A scipy.spatial.transform.Rotation representing the current orientation
transformation – A tuple containing: - translation: 3-element array (tx, ty, tz) - rotation_matrix: (3, 3) rotation matrix
forward_transform (True) – If True, applies the forward transform. If False, applies the inverse transform

Returns:

Transformed 3D point [x, y, z]
Updated orientation

fiftyone.utils.utils3d.corners_from_euler(location: List[float], rotation: List[float], dimension: List[float]) → ndarray#

Computes the 3D corners of a cuboid given its location, rotation, and dimensions.

Parameters:

location – a 3-element list or np.ndarray representing the (x, y, z) location of the cuboid
rotation – a 3-element list or np.ndarray representing the (roll, pitch, yaw) rotation in radians
dimension – a 3-element list or np.ndarray representing the (length, width, height) of the cuboid

Returns:

A 3x8 np.ndarray containing the 3D coordinates of the cuboid’s corners.

fiftyone.utils.utils3d.pinhole_projector(points: ndarray, cam_params: Dict, normalize=True) → ndarray#

Projects 3D detection points to 2D using the given camera intrinsics assuming a pinhole camera.

The following orientation is assumed- x axis points to the right in the image plane, y axis points down in the image plane and z axis points forward from the camera.

Parameters:

points – a 3xN np.ndarray containing the 3D coordinates of the points
cam_params – a dict containing the key ‘intrinsics’ that maps to a 3x3 or 4x4 np.ndarray representing the camera intrinsics matrix
normalize (True) – whether to normalize the projected points by their z-coordinate

Returns:

A 2xN np.ndarray containing the projected 2D coordinates of the points. If normalize is True, the points are normalized by their z-coordinate.

fiftyone.utils.utils3d.point_in_front_of_camera(corners_img: ndarray, corners_3d: ndarray, imsize: Tuple[int, int], distance_threshold: float = 0.1, safety_threshold: float = 0.1) → bool#

Checks if the input corners are visible in the image and in front of the camera.

Parameters:

corners_img – a 2x8 np.ndarray containing the projected 2D coordinates of a cuboid’s corners
corners_3d – a 3x8 np.ndarray containing the 3D coordinates of the cuboid’s corners
imsize – a tuple (width, height) of the image dimensions
distance_threshold (0.1) – a float representing the minimum distance in meters for a corner to be considered in front of the camera
safety_threshold (0.1) – a float representing the minimum safety distance in meters for a corner to be considered safe

Returns:

True if all corners are visible in the image and all corners are in front of the camera, False otherwise.

class fiftyone.utils.utils3d.OrthographicProjectionMetadata(*args, **kwargs)#

Bases: DynamicEmbeddedDocument, _HasMedia

Class for storing metadata about orthographic projections.

Parameters:

filepath (None) – the path to the orthographic projection on disk
min_bound (None) – the [xmin, ymin] of the image in the projection plane
max_bound (None) – the [xmax, ymax] of the image in the projection plane
normal (None) – the normal vector of the plane onto which the projection was performed. If not specified, [0, 0, 1] is assumed
width – the width of the image, in pixels
height – the height of the image, in pixels

Attributes:

`filepath`	A unicode string field.
`min_bound`	A list field that wraps a standard `Field`, allowing multiple instances of the field to be stored as a list in the database.
`max_bound`	A list field that wraps a standard `Field`, allowing multiple instances of the field to be stored as a list in the database.
`normal`	A list field that wraps a standard `Field`, allowing multiple instances of the field to be stored as a list in the database.
`width`	A 32 bit integer field.
`height`	A 32 bit integer field.
`STRICT`
`field_names`	An ordered tuple of the public fields of this document.

Methods:

`clean`()	Hook for doing document level data cleaning (usually validation or assignment) before validation is run.
`clear_field`(field_name)	Clears the field from the document.
`copy`()	Returns a deep copy of the document.
`fancy_repr`([class_name, select_fields, ...])	Generates a customizable string representation of the document.
`field_to_mongo`(field_name)
`field_to_python`(field_name, value)
`from_dict`(d[, extended])	Loads the document from a BSON/JSON dictionary.
`from_json`(s)	Loads the document from a JSON string.
`get_field`(field_name)	Gets the field of the document.
`get_text_score`()	Get text score from text query
`has_field`(field_name)	Determines whether the document has a field of the given name.
`iter_fields`()	Returns an iterator over the `(name, value)` pairs of the public fields of the document.
`merge`(doc[, merge_lists, merge_dicts, overwrite])	Merges the contents of the given document into this document.
`set_field`(field_name, value[, create])	Sets the value of a field of the document.
`to_dict`([extended])	Serializes this document to a BSON/JSON dictionary.
`to_json`([pretty_print])	Serializes the document to a JSON string.
`to_mongo`(args, *kwargs)	Return as SON data ready for use with MongoDB.
`validate`([clean])	Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass

filepath#

A unicode string field.

Parameters:

description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created

min_bound#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:

field (None) – an optional Field instance describing the type of the list elements
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created

max_bound#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:

field (None) – an optional Field instance describing the type of the list elements
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created

normal#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:

field (None) – an optional Field instance describing the type of the list elements
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created

width#

A 32 bit integer field.

Parameters:

description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created

height#

A 32 bit integer field.

Parameters:

description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created

STRICT = False#

clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:: field_name – the field name
Raises:: ValueError – if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:: a SerializableDocument

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:

class_name (None) – optional class name to use
select_fields (None) – iterable of field names to restrict to
exclude_fields (None) – iterable of field names to exclude
**kwargs – additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#: An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#

field_to_python(field_name, value)#

classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:

d – a dictionary
extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:: a SerializableDocument

get_field(field_name)#

Gets the field of the document.

Parameters:: field_name – the field name
Returns:: the field value
Raises:: AttributeError – if the field does not exist

get_text_score()#: Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:: field_name – the field name
Returns:: True/False

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:: an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:

doc – a SerializableDocument of same type as this document
merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value
merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value
overwrite (True) – whether to overwrite (True) or skip (False) existing fields

my_metaclass#: alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:

field_name – the field name
value – the field value
create (True) – whether to create the field if it does not exist

Raises:

ValueError – if field_name is not an allowed field name or does not exist and create == False

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:: extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format
Returns:: a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:: pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations
Returns:: a JSON string

to_mongo(*args, **kwargs)#: Return as SON data ready for use with MongoDB.

validate(clean=True)#

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

fiftyone.utils.utils3d.get_scene_asset_paths(scene_paths, abs_paths=False, skip_failures=True, progress=None)#

Extracts all asset paths for the specified 3D scenes.

Parameters:

scene_paths – an iterable of .fo3d paths
abs_paths (False) – whether to return absolute paths
skip_failures (True) – whether to gracefully continue without raising an error if metadata cannot be computed for a file
progress (None) – whether to render a progress bar (True/False), use the default value fiftyone.config.show_progress_bars (None), or a progress callback function to invoke instead

Returns:

a dict mapping scene paths to lists of asset paths

fiftyone.utils.utils3d.compute_orthographic_projection_images(samples, size, output_dir, rel_dir=None, in_group_slice=None, out_group_slice=None, metadata_field='orthographic_projection_metadata', shading_mode=None, colormap=None, subsampling_rate=None, projection_normal=None, bounds=None, padding=None, overwrite=False, skip_failures=False, progress=None)#

Computes orthographic projection images for the point clouds in the given collection.

This operation will populate OrthographicProjectionMetadata instances for each projection in the metadata_field of each sample.

Examples:

import fiftyone as fo
import fiftyone.utils.utils3d as fou3d
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-groups")
view = dataset.select_group_slices("pcd")

fou3d.compute_orthographic_projection_images(view, (-1, 512), "/tmp/proj")

session = fo.launch_app(view)

Parameters:

samples – a fiftyone.core.collections.SampleCollection
size – the desired (width, height) for the generated maps. Either dimension can be None or negative, in which case the appropriate aspect-preserving value is used
output_dir – an output directory in which to store the images/maps
rel_dir (None) – an optional relative directory to strip from each input filepath to generate a unique identifier that is joined with output_dir to generate an output path for each image. This argument allows for populating nested subdirectories in output_dir that match the shape of the input paths
in_group_slice (None) – the name of the group slice containing the point cloud data. Only applicable if samples is a grouped collection. If samples is a grouped collection and this parameter is not provided, the first point cloud slice will be used
out_group_slice (None) – the name of a group slice to which to add new samples containing the feature images/maps. Only applicable if samples is a grouped collection
metadata_field ("orthographic_projection_metadata") – the name of the field in which to store OrthographicProjectionMetadata instances for each projection
shading_mode (None) – an optional shading algorithm for the points. Supported values are ("intensity", "rgb", or "height"). The "intensity" and "rgb" options are only valid if the PCD’s header contains the "rgb" flag. By default, all points are shaded white
colormap (None) –
an optional colormap to use when shading gradients, formatted as either:
- a dict mapping values in [0, 1] to (R, G, B) tuples in [0, 255]
- a list of (R, G, B) tuples in [0, 255] that cover [0, 1] linearly spaced
subsampling_rate (None) – an optional unsigned int that, if provided, defines a uniform subsampling rate. The selected point indices are [0, k, 2k, …], where k = subsampling_rate
projection_normal (None) – the normal vector of the plane onto which to perform the projection. By default, (0, 0, 1) is used
bounds (None) – an optional ((xmin, ymin, zmin), (xmax, ymax, zmax)) tuple defining the field of view in the projected plane for which to generate each map. Either element of the tuple or any/all of its values can be None, in which case a tight crop of the point cloud along the missing dimension(s) are used
padding (None) – a relative padding(s) in [0, 1]] to apply to the field of view bounds prior to projection. Can either be a single value to apply in all directions or a [padx, pady, padz]. For example, pass padding=0.25 with no bounds to project onto a tight crop of each point cloud with 25% padding around it
overwrite (False) – whether to overwrite existing images
skip_failures (False) – whether to gracefully continue without raising an error if a projection fails
progress (None) – whether to render a progress bar (True/False), use the default value fiftyone.config.show_progress_bars (None), or a progress callback function to invoke instead

fiftyone.utils.utils3d.compute_orthographic_projection_image(filepath, size, shading_mode=None, colormap=None, subsampling_rate=None, projection_normal=None, bounds=None, padding=None)#

Generates an orthographic projection image for the given PCD file onto the specified plane (default xy plane).

The returned image is a three-channel array encoding the intensity, height, and density of the point cloud.

Parameters:

filepath – the path to the .pcd file
size – the desired (width, height) for the generated maps. Either dimension can be None or negative, in which case the appropriate aspect-preserving value is used
shading_mode (None) – an optional shading algorithm for the points. Supported values are ("intensity", "rgb", or "height"). The "intensity" and "rgb" options are only valid if the PCD’s header contains the "rgb" flag. By default, all points are shaded white
colormap (None) –
an optional colormap to use when shading gradients, formatted as either:
- a dict mapping values in [0, 1] to (R, G, B) tuples in [0, 255]
- a list of (R, G, B) tuples in [0, 255] that cover [0, 1] linearly spaced
subsampling_rate (None) – an unsigned int that, if defined, defines a uniform subsampling rate. The selected point indices are [0, k, 2k, …], where k = subsampling_rate
projection_normal (None) – the normal vector of the plane onto which to perform the projection. By default, (0, 0, 1) is used
bounds (None) – an optional ((xmin, ymin, zmin), (xmax, ymax, zmax)) tuple defining the field of view for which to generate each map in the projected plane. Either element of the tuple or any/all of its values can be None, in which case a tight crop of the point cloud along the missing dimension(s) are used
padding (None) – a relative padding(s) in [0, 1]] to apply to the field of view bounds prior to projection. Can either be a single value to apply in all directions or a [padx, pady, padz]. For example, pass padding=0.25 with no bounds to project onto a tight crop of the point cloud with 25% padding around it

Returns:

a tuple of

the orthographic projection image
an OrthographicProjectionMetadata instance

fiftyone.utils.utils3d.pcd_to_3d(dataset, slices=None, output_dir=None, assets_dir=None, rel_dir=None, abs_paths=False, progress=None)#

Converts the point cloud samples in the given dataset to 3D samples.

Parameters:

dataset – a fiftyone.core.dataset.Dataset containing point clouds
slices (None) –
point cloud slice(s) to convert. Only applicable when the dataset is grouped, in which case you can provide:
- a slice or iterable of point cloud slices to convert in-place
- a dict mapping point cloud slices to desired 3D slice names
- None (default): all point cloud slices are converted in-place
output_dir (None) – an optional output directory for the .fo3d files
assets_dir (None) – an optional directory to copy the .pcd files into. Can be either an absolute directory, a subdirectory of output_dir, or None if you do not wish to copy point clouds
rel_dir (None) – an optional relative directory to strip from each point cloud path to generate a unique identifier for each scene, which is joined with output_dir to generate an output path for each .fo3d file. This argument allows for populating nested subdirectories that match the shape of the input paths. The path is converted to an absolute path (if necessary) via fiftyone.core.storage.normalize_path()
abs_paths (False) – whether to store absolute paths to the point cloud files in the exported .fo3d files
progress (None) – whether to render a progress bar (True/False), use the default value fiftyone.config.show_progress_bars (None), or a progress callback function to invoke instead