fiftyone.utils.data.ingestors¶
Dataset ingestors.
Classes:
|
Mixin for |
|
Dataset importer that ingests labeled images into the provided |
|
Dataset importer that ingests labeled videos into the provided |
|
Dataset importer that ingests unlabeled images into the provided |
|
Dataset importer that ingests unlabeled videos into the provided |
|
Mixin for |
-
class
fiftyone.utils.data.ingestors.
ImageIngestor
(dataset_dir, image_format=None)¶ Bases:
object
Mixin for
fiftyone.utils.data.importers.DatasetImporter
instances that ingest images into the provideddataset_dir
during import.- Parameters
dataset_dir – the directory where input images will be ingested into
image_format (None) – the image format to use when writing in-memory images to disk. By default,
fiftyone.config.default_image_ext
is used
-
class
fiftyone.utils.data.ingestors.
UnlabeledImageDatasetIngestor
(dataset_dir, samples, sample_parser, image_format=None, max_samples=None)¶ Bases:
fiftyone.utils.data.importers.UnlabeledImageDatasetImporter
,fiftyone.utils.data.ingestors.ImageIngestor
Dataset importer that ingests unlabeled images into the provided
dataset_dir
during import.The source images are parsed from the provided
samples
using the providedfiftyone.utils.data.parsers.UnlabeledImageSampleParser
.If an image path is available via
fiftyone.utils.data.parsers.UnlabeledImageSampleParser.get_image_path()
, then the image is directly copied from its source location intodataset_dir
. In this case, the original filename is maintained, unless a name conflict would occur, in which case an index of the form"-%d" % count
is appended to the base filename.If no image path is available, the image is read in-memory via
fiftyone.utils.data.parsers.UnlabeledImageSampleParser.get_image()
and written todataset_dir
in the following format:<dataset_dir>/<image_count><image_format>
where
image_count
is the number of files indataset_dir
.- Parameters
dataset_dir – the directory where input images will be ingested into
samples – an iterable of samples that can be parsed by
sample_parser
sample_parser – an
fiftyone.utils.data.parsers.UnlabeledImageSampleParser
to use to parse the samplesimage_format (None) – the image format to use when writing in-memory images to disk. By default,
fiftyone.config.default_image_ext
is usedmax_samples (None) – a maximum number of samples to import. By default, all samples are imported
Methods:
close
(*args)Performs any necessary actions after the last sample has been imported.
Returns the dataset info for the dataset.
setup
()Performs any necessary setup before importing the first sample in the dataset.
Attributes:
Whether this importer produces a dataset info dictionary.
Whether this importer produces
fiftyone.core.metadata.ImageMetadata
instances for each image.-
property
has_dataset_info
¶ Whether this importer produces a dataset info dictionary.
-
property
has_image_metadata
¶ Whether this importer produces
fiftyone.core.metadata.ImageMetadata
instances for each image.
-
setup
()¶ Performs any necessary setup before importing the first sample in the dataset.
This method is called when the importer’s context manager interface is entered,
DatasetImporter.__enter__()
.
-
close
(*args)¶ Performs any necessary actions after the last sample has been imported.
This method is called when the importer’s context manager interface is exited,
DatasetImporter.__exit__()
.- Parameters
*args – the arguments to
DatasetImporter.__exit__()
-
get_dataset_info
()¶ Returns the dataset info for the dataset.
By convention, this method should be called after all samples in the dataset have been imported.
- Returns
a dict of dataset info
-
class
fiftyone.utils.data.ingestors.
LabeledImageDatasetIngestor
(dataset_dir, samples, sample_parser, image_format=None, max_samples=None)¶ Bases:
fiftyone.utils.data.importers.LabeledImageDatasetImporter
,fiftyone.utils.data.ingestors.ImageIngestor
Dataset importer that ingests labeled images into the provided
dataset_dir
during import.The source images and labels are parsed from the provided
samples
using the providedfiftyone.utils.data.parsers.LabeledImageSampleParser
.If an image path is available via
fiftyone.utils.data.parsers.LabeledImageSampleParser.get_image_path()
, then the image is directly copied from its source location intodataset_dir
. In this case, the original filename is maintained, unless a name conflict would occur, in which case an index of the form"-%d" % count
is appended to the base filename.If no image path is available, the image is read in-memory via
fiftyone.utils.data.parsers.LabeledImageSampleParser.get_image()
and written todataset_dir
in the following format:<dataset_dir>/<image_count><image_format>
where
image_count
is the number of files indataset_dir
.- Parameters
dataset_dir – the directory where input images will be ingested into
samples – an iterable of samples that can be parsed by
sample_parser
sample_parser – an
fiftyone.utils.data.parsers.LabeledImageSampleParser
to use to parse the samplesimage_format (None) – the image format to use when writing in-memory images to disk. By default,
fiftyone.config.default_image_ext
is usedmax_samples (None) – a maximum number of samples to import. By default, all samples are imported
Methods:
close
(*args)Performs any necessary actions after the last sample has been imported.
Returns the dataset info for the dataset.
setup
()Performs any necessary setup before importing the first sample in the dataset.
Attributes:
Whether this importer produces a dataset info dictionary.
Whether this importer produces
fiftyone.core.metadata.ImageMetadata
instances for each image.The
fiftyone.core.labels.Label
class(es) returned by this importer.-
property
has_dataset_info
¶ Whether this importer produces a dataset info dictionary.
-
property
has_image_metadata
¶ Whether this importer produces
fiftyone.core.metadata.ImageMetadata
instances for each image.
-
property
label_cls
¶ The
fiftyone.core.labels.Label
class(es) returned by this importer.This can be any of the following:
a
fiftyone.core.labels.Label
class. In this case, the importer is guaranteed to return labels of this typea list or tuple of
fiftyone.core.labels.Label
classes. In this case, the importer can produce a single label field of any of these typesa dict mapping keys to
fiftyone.core.labels.Label
classes. In this case, the importer will return label dictionaries with keys and value-types specified by this dictionary. Not all keys need be present in the imported labelsNone
. In this case, the importer makes no guarantees about the labels that it may return
-
setup
()¶ Performs any necessary setup before importing the first sample in the dataset.
This method is called when the importer’s context manager interface is entered,
DatasetImporter.__enter__()
.
-
close
(*args)¶ Performs any necessary actions after the last sample has been imported.
This method is called when the importer’s context manager interface is exited,
DatasetImporter.__exit__()
.- Parameters
*args – the arguments to
DatasetImporter.__exit__()
-
get_dataset_info
()¶ Returns the dataset info for the dataset.
By convention, this method should be called after all samples in the dataset have been imported.
- Returns
a dict of dataset info
-
class
fiftyone.utils.data.ingestors.
VideoIngestor
(dataset_dir)¶ Bases:
object
Mixin for
fiftyone.utils.data.importers.DatasetImporter
instances that ingest videos into the provideddataset_dir
during import.- Parameters
dataset_dir – the directory where input videos will be ingested into
-
class
fiftyone.utils.data.ingestors.
UnlabeledVideoDatasetIngestor
(dataset_dir, samples, sample_parser, max_samples=None)¶ Bases:
fiftyone.utils.data.importers.UnlabeledVideoDatasetImporter
,fiftyone.utils.data.ingestors.VideoIngestor
Dataset importer that ingests unlabeled videos into the provided
dataset_dir
during import.The source videos are parsed from the provided
samples
using the providedfiftyone.utils.data.parsers.UnlabeledVideoSampleParser
.The source videos are directly copied from their source locations into
dataset_dir
, maintaining the original filenames, unless a name conflict would occur, in which case an index of the form"-%d" % count
is appended to the base filename.- Parameters
dataset_dir – the directory where input videos will be ingested into
samples – an iterable of samples that can be parsed by
sample_parser
sample_parser – an
fiftyone.utils.data.parsers.UnlabeledVideoSampleParser
to use to parse the samplesmax_samples (None) – a maximum number of samples to import. By default, all samples are imported
Methods:
close
(*args)Performs any necessary actions after the last sample has been imported.
Returns the dataset info for the dataset.
setup
()Performs any necessary setup before importing the first sample in the dataset.
Attributes:
Whether this importer produces a dataset info dictionary.
Whether this importer produces
fiftyone.core.metadata.VideoMetadata
instances for each video.-
property
has_dataset_info
¶ Whether this importer produces a dataset info dictionary.
-
property
has_video_metadata
¶ Whether this importer produces
fiftyone.core.metadata.VideoMetadata
instances for each video.
-
setup
()¶ Performs any necessary setup before importing the first sample in the dataset.
This method is called when the importer’s context manager interface is entered,
DatasetImporter.__enter__()
.
-
close
(*args)¶ Performs any necessary actions after the last sample has been imported.
This method is called when the importer’s context manager interface is exited,
DatasetImporter.__exit__()
.- Parameters
*args – the arguments to
DatasetImporter.__exit__()
-
get_dataset_info
()¶ Returns the dataset info for the dataset.
By convention, this method should be called after all samples in the dataset have been imported.
- Returns
a dict of dataset info
-
class
fiftyone.utils.data.ingestors.
LabeledVideoDatasetIngestor
(dataset_dir, samples, sample_parser, max_samples=None)¶ Bases:
fiftyone.utils.data.importers.LabeledVideoDatasetImporter
,fiftyone.utils.data.ingestors.VideoIngestor
Dataset importer that ingests labeled videos into the provided
dataset_dir
during import.The source videos and labels are parsed from the provided
samples
using the providedfiftyone.utils.data.parsers.LabeledVideoSampleParser
.The source videos are directly copied from their source locations into
dataset_dir
, maintaining the original filenames, unless a name conflict would occur, in which case an index of the form"-%d" % count
is appended to the base filename.- Parameters
dataset_dir – the directory where input videos will be ingested into
samples – an iterable of samples that can be parsed by
sample_parser
sample_parser – an
fiftyone.utils.data.parsers.LabeledVideoSampleParser
to use to parse the samplesmax_samples (None) – a maximum number of samples to import. By default, all samples are imported
Methods:
close
(*args)Performs any necessary actions after the last sample has been imported.
Returns the dataset info for the dataset.
setup
()Performs any necessary setup before importing the first sample in the dataset.
Attributes:
The
fiftyone.core.labels.Label
class(es) returned by this importer within the frame labels that it produces.Whether this importer produces a dataset info dictionary.
Whether this importer produces
fiftyone.core.metadata.VideoMetadata
instances for each video.The
fiftyone.core.labels.Label
class(es) returned by this importer within the sample-level labels that it produces.-
property
has_dataset_info
¶ Whether this importer produces a dataset info dictionary.
-
property
has_video_metadata
¶ Whether this importer produces
fiftyone.core.metadata.VideoMetadata
instances for each video.
-
property
label_cls
¶ The
fiftyone.core.labels.Label
class(es) returned by this importer within the sample-level labels that it produces.This can be any of the following:
a
fiftyone.core.labels.Label
class. In this case, the importer is guaranteed to return sample-level labels of this typea list or tuple of
fiftyone.core.labels.Label
classes. In this case, the importer can produce a single sample-level label field of any of these typesa dict mapping keys to
fiftyone.core.labels.Label
classes. In this case, the importer will return sample-level label dictionaries with keys and value-types specified by this dictionary. Not all keys need be present in the imported labelsNone
. In this case, the importer makes no guarantees about the sample-level labels that it may return
-
property
frame_labels_cls
¶ The
fiftyone.core.labels.Label
class(es) returned by this importer within the frame labels that it produces.This can be any of the following:
a
fiftyone.core.labels.Label
class. In this case, the importer is guaranteed to return frame labels of this typea list or tuple of
fiftyone.core.labels.Label
classes. In this case, the importer can produce a single frame label field of any of these typesa dict mapping keys to
fiftyone.core.labels.Label
classes. In this case, the importer will return frame label dictionaries with keys and value-types specified by this dictionary. Not all keys need be present in each frameNone
. In this case, the importer makes no guarantees about the frame labels that it may return
-
setup
()¶ Performs any necessary setup before importing the first sample in the dataset.
This method is called when the importer’s context manager interface is entered,
DatasetImporter.__enter__()
.
-
close
(*args)¶ Performs any necessary actions after the last sample has been imported.
This method is called when the importer’s context manager interface is exited,
DatasetImporter.__exit__()
.- Parameters
*args – the arguments to
DatasetImporter.__exit__()
-
get_dataset_info
()¶ Returns the dataset info for the dataset.
By convention, this method should be called after all samples in the dataset have been imported.
- Returns
a dict of dataset info