fiftyone.utils.utils3d#

3D utilities.

Copyright 2017-2022, Voxel51, Inc.

Functions:

compute_cuboid_iou(gt,ย pred[,ย gt_crowd])

Computes the IoU between the given ground truth and predicted cuboids.

get_scene_asset_paths(scene_paths[,ย ...])

Extracts all asset paths for the specified 3D scenes.

compute_orthographic_projection_images(...)

Computes orthographic projection images for the point clouds in the given collection.

compute_orthographic_projection_image(...[,ย ...])

Generates an orthographic projection image for the given PCD file onto the specified plane (default xy plane).

pcd_to_3d(dataset[,ย slices,ย output_dir,ย ...])

Converts the point cloud samples in the given dataset to 3D samples.

Classes:

OrthographicProjectionMetadata(*args,ย **kwargs)

Class for storing metadata about orthographic projections.

fiftyone.utils.utils3d.compute_cuboid_iou(gt, pred, gt_crowd=False)#

Computes the IoU between the given ground truth and predicted cuboids.

Parameters:
Returns:

the IoU, in [0, 1]

class fiftyone.utils.utils3d.OrthographicProjectionMetadata(*args, **kwargs)#

Bases: DynamicEmbeddedDocument, _HasMedia

Class for storing metadata about orthographic projections.

Parameters:
  • filepath (None) โ€“ the path to the orthographic projection on disk

  • min_bound (None) โ€“ the [xmin, ymin] of the image in the projection plane

  • max_bound (None) โ€“ the [xmax, ymax] of the image in the projection plane

  • normal (None) โ€“ the normal vector of the plane onto which the projection was performed. If not specified, [0, 0, 1] is assumed

  • width โ€“ the width of the image, in pixels

  • height โ€“ the height of the image, in pixels

Attributes:

filepath

A unicode string field.

min_bound

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

max_bound

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

normal

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

width

A 32 bit integer field.

height

A 32 bit integer field.

STRICT

field_names

An ordered tuple of the public fields of this document.

Methods:

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

clear_field(field_name)

Clears the field from the document.

copy()

Returns a deep copy of the document.

fancy_repr([class_name,ย select_fields,ย ...])

Generates a customizable string representation of the document.

field_to_mongo(field_name)

field_to_python(field_name,ย value)

from_dict(d[,ย extended])

Loads the document from a BSON/JSON dictionary.

from_json(s)

Loads the document from a JSON string.

get_field(field_name)

Gets the field of the document.

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

merge(doc[,ย merge_lists,ย merge_dicts,ย overwrite])

Merges the contents of the given document into this document.

set_field(field_name,ย value[,ย create])

Sets the value of a field of the document.

to_dict([extended])

Serializes this document to a BSON/JSON dictionary.

to_json([pretty_print])

Serializes the document to a JSON string.

to_mongo(*args,ย **kwargs)

Return as SON data ready for use with MongoDB.

validate([clean])

Ensure that all fields' values are valid and that required fields are present.

Classes:

my_metaclass

alias of DocumentMetaclass

filepath#

A unicode string field.

Parameters:
  • description (None) โ€“ an optional description

  • info (None) โ€“ an optional info dict

  • read_only (False) โ€“ whether the field is read-only

  • created_at (None) โ€“ the datetime the field was created

min_bound#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:
  • field (None) โ€“ an optional Field instance describing the type of the list elements

  • description (None) โ€“ an optional description

  • info (None) โ€“ an optional info dict

  • read_only (False) โ€“ whether the field is read-only

  • created_at (None) โ€“ the datetime the field was created

max_bound#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:
  • field (None) โ€“ an optional Field instance describing the type of the list elements

  • description (None) โ€“ an optional description

  • info (None) โ€“ an optional info dict

  • read_only (False) โ€“ whether the field is read-only

  • created_at (None) โ€“ the datetime the field was created

normal#

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters:
  • field (None) โ€“ an optional Field instance describing the type of the list elements

  • description (None) โ€“ an optional description

  • info (None) โ€“ an optional info dict

  • read_only (False) โ€“ whether the field is read-only

  • created_at (None) โ€“ the datetime the field was created

width#

A 32 bit integer field.

Parameters:
  • description (None) โ€“ an optional description

  • info (None) โ€“ an optional info dict

  • read_only (False) โ€“ whether the field is read-only

  • created_at (None) โ€“ the datetime the field was created

height#

A 32 bit integer field.

Parameters:
  • description (None) โ€“ an optional description

  • info (None) โ€“ an optional info dict

  • read_only (False) โ€“ whether the field is read-only

  • created_at (None) โ€“ the datetime the field was created

STRICT = False#
clean()#

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)#

Clears the field from the document.

Parameters:

field_name โ€“ the field name

Raises:

ValueError โ€“ if the field does not exist

copy()#

Returns a deep copy of the document.

Returns:

a SerializableDocument

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)#

Generates a customizable string representation of the document.

Parameters:
  • class_name (None) โ€“ optional class name to use

  • select_fields (None) โ€“ iterable of field names to restrict to

  • exclude_fields (None) โ€“ iterable of field names to exclude

  • **kwargs โ€“ additional key-value pairs to include in the string representation

Returns:

a string representation of the document

property field_names#

An ordered tuple of the public fields of this document.

field_to_mongo(field_name)#
field_to_python(field_name, value)#
classmethod from_dict(d, extended=False)#

Loads the document from a BSON/JSON dictionary.

Parameters:
  • d โ€“ a dictionary

  • extended (False) โ€“ whether the input dictionary may contain serialized extended JSON constructs

Returns:

a SerializableDocument

classmethod from_json(s)#

Loads the document from a JSON string.

Returns:

a SerializableDocument

get_field(field_name)#

Gets the field of the document.

Parameters:

field_name โ€“ the field name

Returns:

the field value

Raises:

AttributeError โ€“ if the field does not exist

get_text_score()#

Get text score from text query

has_field(field_name)#

Determines whether the document has a field of the given name.

Parameters:

field_name โ€“ the field name

Returns:

True/False

iter_fields()#

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns:

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)#

Merges the contents of the given document into this document.

Parameters:
  • doc โ€“ a SerializableDocument of same type as this document

  • merge_lists (True) โ€“ whether to merge the elements of top-level list fields rather than treating the list as a single value

  • merge_dicts (True) โ€“ whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

  • overwrite (True) โ€“ whether to overwrite (True) or skip (False) existing fields

my_metaclass#

alias of DocumentMetaclass

set_field(field_name, value, create=True)#

Sets the value of a field of the document.

Parameters:
  • field_name โ€“ the field name

  • value โ€“ the field value

  • create (True) โ€“ whether to create the field if it does not exist

Raises:

ValueError โ€“ if field_name is not an allowed field name or does not exist and create == False

to_dict(extended=False)#

Serializes this document to a BSON/JSON dictionary.

Parameters:

extended (False) โ€“ whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns:

a dict

to_json(pretty_print=False)#

Serializes the document to a JSON string.

Parameters:

pretty_print (False) โ€“ whether to render the JSON in human readable format with newlines and indentations

Returns:

a JSON string

to_mongo(*args, **kwargs)#

Return as SON data ready for use with MongoDB.

validate(clean=True)#

Ensure that all fieldsโ€™ values are valid and that required fields are present.

Raises ValidationError if any of the fieldsโ€™ values are found to be invalid.

fiftyone.utils.utils3d.get_scene_asset_paths(scene_paths, abs_paths=False, skip_failures=True, progress=None)#

Extracts all asset paths for the specified 3D scenes.

Parameters:
  • scene_paths โ€“ an iterable of .fo3d paths

  • abs_paths (False) โ€“ whether to return absolute paths

  • skip_failures (True) โ€“ whether to gracefully continue without raising an error if metadata cannot be computed for a file

  • progress (None) โ€“ whether to render a progress bar (True/False), use the default value fiftyone.config.show_progress_bars (None), or a progress callback function to invoke instead

Returns:

a dict mapping scene paths to lists of asset paths

fiftyone.utils.utils3d.compute_orthographic_projection_images(samples, size, output_dir, rel_dir=None, in_group_slice=None, out_group_slice=None, metadata_field='orthographic_projection_metadata', shading_mode=None, colormap=None, subsampling_rate=None, projection_normal=None, bounds=None, padding=None, overwrite=False, skip_failures=False, progress=None)#

Computes orthographic projection images for the point clouds in the given collection.

This operation will populate OrthographicProjectionMetadata instances for each projection in the metadata_field of each sample.

Examples:

import fiftyone as fo
import fiftyone.utils.utils3d as fou3d
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-groups")
view = dataset.select_group_slices("pcd")

fou3d.compute_orthographic_projection_images(view, (-1, 512), "/tmp/proj")

session = fo.launch_app(view)
Parameters:
  • samples โ€“ a fiftyone.core.collections.SampleCollection

  • size โ€“ the desired (width, height) for the generated maps. Either dimension can be None or negative, in which case the appropriate aspect-preserving value is used

  • output_dir โ€“ an output directory in which to store the images/maps

  • rel_dir (None) โ€“ an optional relative directory to strip from each input filepath to generate a unique identifier that is joined with output_dir to generate an output path for each image. This argument allows for populating nested subdirectories in output_dir that match the shape of the input paths

  • in_group_slice (None) โ€“ the name of the group slice containing the point cloud data. Only applicable if samples is a grouped collection. If samples is a grouped collection and this parameter is not provided, the first point cloud slice will be used

  • out_group_slice (None) โ€“ the name of a group slice to which to add new samples containing the feature images/maps. Only applicable if samples is a grouped collection

  • metadata_field ("orthographic_projection_metadata") โ€“ the name of the field in which to store OrthographicProjectionMetadata instances for each projection

  • shading_mode (None) โ€“ an optional shading algorithm for the points. Supported values are ("intensity", "rgb", or "height"). The "intensity" and "rgb" options are only valid if the PCDโ€™s header contains the "rgb" flag. By default, all points are shaded white

  • colormap (None) โ€“

    an optional colormap to use when shading gradients, formatted as either:

    • a dict mapping values in [0, 1] to (R, G, B) tuples in [0, 255]

    • a list of of (R, G, B) tuples in [0, 255] that cover [0, 1] linearly spaced

  • subsampling_rate (None) โ€“ an optional unsigned int that, if provided, defines a uniform subsampling rate. The selected point indices are [0, k, 2k, โ€ฆ], where k = subsampling_rate

  • projection_normal (None) โ€“ the normal vector of the plane onto which to perform the projection. By default, (0, 0, 1) is used

  • bounds (None) โ€“ an optional ((xmin, ymin, zmin), (xmax, ymax, zmax)) tuple defining the field of view in the projected plane for which to generate each map. Either element of the tuple or any/all of its values can be None, in which case a tight crop of the point cloud along the missing dimension(s) are used

  • padding (None) โ€“ a relative padding(s) in [0, 1]] to apply to the field of view bounds prior to projection. Can either be a single value to apply in all directions or a [padx, pady, padz]. For example, pass padding=0.25 with no bounds to project onto a tight crop of each point cloud with 25% padding around it

  • overwrite (False) โ€“ whether to overwrite existing images

  • skip_failures (False) โ€“ whether to gracefully continue without raising an error if a projection fails

  • progress (None) โ€“ whether to render a progress bar (True/False), use the default value fiftyone.config.show_progress_bars (None), or a progress callback function to invoke instead

fiftyone.utils.utils3d.compute_orthographic_projection_image(filepath, size, shading_mode=None, colormap=None, subsampling_rate=None, projection_normal=None, bounds=None, padding=None)#

Generates an orthographic projection image for the given PCD file onto the specified plane (default xy plane).

The returned image is a three-channel array encoding the intensity, height, and density of the point cloud.

Parameters:
  • filepath โ€“ the path to the .pcd file

  • size โ€“ the desired (width, height) for the generated maps. Either dimension can be None or negative, in which case the appropriate aspect-preserving value is used

  • shading_mode (None) โ€“ an optional shading algorithm for the points. Supported values are ("intensity", "rgb", or "height"). The "intensity" and "rgb" options are only valid if the PCDโ€™s header contains the "rgb" flag. By default, all points are shaded white

  • colormap (None) โ€“

    an optional colormap to use when shading gradients, formatted as either:

    • a dict mapping values in [0, 1] to (R, G, B) tuples in [0, 255]

    • a list of of (R, G, B) tuples in [0, 255] that cover [0, 1] linearly spaced

  • subsampling_rate (None) โ€“ an unsigned int that, if defined, defines a uniform subsampling rate. The selected point indices are [0, k, 2k, โ€ฆ], where k = subsampling_rate

  • projection_normal (None) โ€“ the normal vector of the plane onto which to perform the projection. By default, (0, 0, 1) is used

  • bounds (None) โ€“ an optional ((xmin, ymin, zmin), (xmax, ymax, zmax)) tuple defining the field of view for which to generate each map in the projected plane. Either element of the tuple or any/all of its values can be None, in which case a tight crop of the point cloud along the missing dimension(s) are used

  • padding (None) โ€“ a relative padding(s) in [0, 1]] to apply to the field of view bounds prior to projection. Can either be a single value to apply in all directions or a [padx, pady, padz]. For example, pass padding=0.25 with no bounds to project onto a tight crop of the point cloud with 25% padding around it

Returns:

a tuple of

fiftyone.utils.utils3d.pcd_to_3d(dataset, slices=None, output_dir=None, assets_dir=None, rel_dir=None, abs_paths=False, progress=None)#

Converts the point cloud samples in the given dataset to 3D samples.

Parameters:
  • dataset โ€“ a fiftyone.core.dataset.Dataset containing point clouds

  • slices (None) โ€“

    point cloud slice(s) to convert. Only applicable when the dataset is grouped, in which case you can provide:

    • a slice or iterable of point cloud slices to convert in-place

    • a dict mapping point cloud slices to desired 3D slice names

    • None (default): all point cloud slices are converted in-place

  • output_dir (None) โ€“ an optional output directory for the .fo3d files

  • assets_dir (None) โ€“ an optional directory to copy the .pcd files into. Can be either an absolute directory, a subdirectory of output_dir, or None if you do not wish to copy point clouds

  • rel_dir (None) โ€“ an optional relative directory to strip from each point cloud path to generate a unique identifier for each scene, which is joined with output_dir to generate an output path for each .fo3d file. This argument allows for populating nested subdirectories that match the shape of the input paths. The path is converted to an absolute path (if necessary) via fiftyone.core.storage.normalize_path()

  • abs_paths (False) โ€“ whether to store absolute paths to the point cloud files in the exported .fo3d files

  • progress (None) โ€“ whether to render a progress bar (True/False), use the default value fiftyone.config.show_progress_bars (None), or a progress callback function to invoke instead