fiftyone.core.map.process¶
Multiprocessing utilities.
Classes:
|
Executes map_samples using multiprocessing. |
-
class
fiftyone.core.map.process.ProcessMapper(batch_cls: Type[fiftyone.core.map.batcher.batch.SampleBatch], num_workers: int, batch_size: Optional[int] = None)¶ Bases:
fiftyone.core.map.mapper.LocalMapperExecutes map_samples using multiprocessing.
Methods:
create(*, config, batch_cls[, num_workers, …])Create a new mapper instance
map_samples(sample_collection, map_fcn, *[, …])Applies map function to each sample and returns an iterator of the results.
Attributes:
Number of samples per worker batch
Number of workers to use
-
classmethod
create(*, config: fiftyone.core.config.FiftyOneConfig, batch_cls: Type[fiftyone.core.map.batcher.batch.SampleBatch], num_workers: Optional[int] = None, batch_size: Optional[int] = None, **__)¶ Create a new mapper instance
-
property
batch_size¶ Number of samples per worker batch
-
map_samples(sample_collection: fiftyone.core.map.typing.SampleCollection[T], map_fcn: Callable[[T], R], *, progress: Optional[Union[bool, Literal[workers]]] = None, save: bool = False, skip_failures: bool = True) → Iterator[Tuple[bson.objectid.ObjectId, R]]¶ Applies map function to each sample and returns an iterator of the results.
- Parameters
sample_collection (SampleCollection[T]) – The sample collection to map.
map_fcn (Callable[[T], R]) – The map function to apply to each sample.
progress (Union[bool, Literal["workers"]]) – Whether or not and how to render progress.
save (bool, optional) – Whether to save mutated samples mutated in the map function. Defaults to False.
skip_failures (bool, optional) – Whether to gracefully continue without raising an error if the map function raises an exception for a sample. Defaults to True.
- Yields
Iterator[Tuple[bson.ObjectId, R]] –
- The sample ID and the result of
the map function for the sample.
-
property
num_workers¶ Number of workers to use
-
classmethod