fiftyone.utils.eval¶
Module contents¶
Evaluation utilities.
Functions:
|
Evaluates the classification predictions in the given collection with respect to the specified ground truth labels. |
|
Evaluates the predicted detections in the given samples with respect to the specified ground truth detections. |
|
Evaluates the regression predictions in the given collection with respect to the specified ground truth values. |
|
Evaluates the specified semantic segmentation masks in the given collection with respect to the specified ground truth masks. |
Classes:
|
Class that stores the results of a classification evaluation. |
|
Class that stores the results of a binary classification evaluation. |
|
Class that stores the results of a detection evaluation. |
|
Class that stores the results of a regression evaluation. |
|
Class that stores the results of a segmentation evaluation. |
-
fiftyone.utils.eval.
evaluate_classifications
(samples, pred_field, gt_field='ground_truth', eval_key=None, classes=None, missing=None, method='simple', **kwargs)¶ Evaluates the classification predictions in the given collection with respect to the specified ground truth labels.
By default, this method simply compares the ground truth and prediction for each sample, but other strategies such as binary evaluation and top-k matching can be configured via the
method
parameter.You can customize the evaluation method by passing additional parameters for the method’s config class as
kwargs
.The supported
method
values and their associated configs are:"simple"
:SimpleEvaluationConfig
"top-k"
:TopKEvaluationConfig
"binary"
:BinaryEvaluationConfig
If an
eval_key
is specified, then this method will record some statistics on each sample:When evaluating sample-level fields, an
eval_key
field will be populated on each sample recording whether that sample’s prediction is correct.When evaluating frame-level fields, an
eval_key
field will be populated on each frame recording whether that frame’s prediction is correct. In addition, aneval_key
field will be populated on each sample that records the average accuracy of the frame predictions of the sample.
- Parameters
samples – a
fiftyone.core.collections.SampleCollection
pred_field – the name of the field containing the predicted
fiftyone.core.labels.Classification
instancesgt_field ("ground_truth") – the name of the field containing the ground truth
fiftyone.core.labels.Classification
instanceseval_key (None) – an evaluation key to use to refer to this evaluation
classes (None) – the list of possible classes. If not provided, the observed ground truth/predicted labels are used
missing (None) – a missing label string. Any None-valued labels are given this label for results purposes
method ("simple") – a string specifying the evaluation method to use. Supported values are
("simple", "binary", "top-k")
**kwargs – optional keyword arguments for the constructor of the
ClassificationEvaluationConfig
being used
- Returns
-
class
fiftyone.utils.eval.
ClassificationResults
(samples, config, eval_key, ytrue, ypred, confs=None, weights=None, ytrue_ids=None, ypred_ids=None, classes=None, missing=None, backend=None)¶ Bases:
fiftyone.utils.eval.base.BaseEvaluationResults
Class that stores the results of a classification evaluation.
- Parameters
samples – the
fiftyone.core.collections.SampleCollection
usedconfig – the
ClassificationEvaluationConfig
usedeval_key – the evaluation key
ytrue – a list of ground truth labels
ypred – a list of predicted labels
confs (None) – an optional list of confidences for the predictions
weights (None) – an optional list of sample weights
ytrue_ids (None) – a list of IDs for the ground truth labels
ypred_ids (None) – a list of IDs for the predicted labels
classes (None) – the list of possible classes. If not provided, the observed ground truth/predicted labels are used
missing (None) – a missing label string. Any None-valued labels are given this label for evaluation purposes
backend (None) – a
ClassificationEvaluation
backend
Methods:
Returns the list of class attributes that will be serialized by
serialize()
.confusion_matrix
([classes, include_other])Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
from_dict
(d, samples, config, key)Builds a
RunResults
from a JSON dict representation of it.from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
metrics
([classes, average, beta])Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
plot_confusion_matrix
([classes, …])Plots a confusion matrix for the evaluation results.
print_report
([classes, digits])Prints a classification report for the results via
sklearn.metrics.classification_report()
.report
([classes])Generates a classification report for the results via
sklearn.metrics.classification_report()
.save
()Saves the results to the database.
Saves these results config to the database.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
Attributes:
The
Run
for these results.The fully-qualified name of this
RunResults
class.The
RunConfig
for these results.The run key for these results.
The
fiftyone.core.collections.SampleCollection
associated with these results.-
attributes
()¶ Returns the list of class attributes that will be serialized by
serialize()
.- Returns
a list of attributes
-
property
backend
¶ The
Run
for these results.
-
property
cls
¶ The fully-qualified name of this
RunResults
class.
-
property
config
¶ The
RunConfig
for these results.
-
confusion_matrix
(classes=None, include_other=False)¶ Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.The rows of the confusion matrix represent ground truth and the columns represent predictions.
- Parameters
classes (None) – an optional list of classes to include in the confusion matrix. Include
self.missing
in this list if you would like to include a row/column for unmatched examplesinclude_other (False) – whether to include an extra row/column at the end of the matrix for labels that do not appear in
classes
. Only applicable ifclasses
are provided
- Returns
a
num_classes x num_classes
confusion matrix
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
classmethod
from_dict
(d, samples, config, key)¶ Builds a
RunResults
from a JSON dict representation of it.- Parameters
d – a JSON dict
samples – the
fiftyone.core.collections.SampleCollection
for the runconfig – the
RunConfig
for the runkey – the run key
- Returns
a
RunResults
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
property
key
¶ The run key for these results.
-
metrics
(classes=None, average='micro', beta=1.0)¶ Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
See
sklearn.metrics.precision_recall_fscore_support()
for details.- Parameters
classes (None) – an optional list of classes to include in the calculations
average ("micro") – the averaging strategy to use
beta (1.0) – the F-beta value to use
- Returns
a dict
-
plot_confusion_matrix
(classes=None, include_other=None, include_missing=None, other_label='(other)', backend='plotly', **kwargs)¶ Plots a confusion matrix for the evaluation results.
If you are working in a notebook environment with the default plotly backend, this method returns an interactive
fiftyone.core.plots.plotly.InteractiveHeatmap
that you can attach to an App session via itsfiftyone.core.session.Session.plots
attribute, which will automatically sync the session’s view with the currently selected cells in the confusion matrix.- Parameters
classes (None) – an optional list of classes to include in the confusion matrix
include_other (None) –
whether to include a row/column for examples whose label is in
classes
but are matched to labels that do not appear inclasses
. Only applicable ifclasses
are provided. The supported values are:None (default): only include a row/column for other labels if there are any
True: do include a row/column for other labels
False: do not include a row/column for other labels
include_missing (None) –
whether to include a row/column for missing ground truth/predictions in the confusion matrix. The supported values are:
None (default): only include a row/column for missing labels if there are any
True: do include a row/column for missing labels
False: do not include a row/column for missing labels
other_label ("(other)") – the label to use for “other” predictions
backend ("plotly") – the plotting backend to use. Supported values are
("plotly", "matplotlib")
**kwargs –
keyword arguments for the backend plotting method:
”plotly” backend:
fiftyone.core.plots.plotly.plot_confusion_matrix()
”matplotlib” backend:
fiftyone.core.plots.matplotlib.plot_confusion_matrix()
- Returns
a
fiftyone.core.plots.plotly.InteractiveHeatmap
, if the plotly backend is useda matplotlib figure, otherwise
- Return type
one of the following
-
print_report
(classes=None, digits=2)¶ Prints a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
digits (2) – the number of digits of precision to print
-
report
(classes=None)¶ Generates a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
- Returns
a dict
-
property
samples
¶ The
fiftyone.core.collections.SampleCollection
associated with these results.
-
save
()¶ Saves the results to the database.
-
save_config
()¶ Saves these results config to the database.
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
class
fiftyone.utils.eval.
BinaryClassificationResults
(samples, config, eval_key, ytrue, ypred, confs, classes, weights=None, ytrue_ids=None, ypred_ids=None, backend=None)¶ Bases:
fiftyone.utils.eval.classification.ClassificationResults
Class that stores the results of a binary classification evaluation.
Any missing ground truth or prediction labels are assumed to be examples of the negative class (with zero confidence, for predictions).
- Parameters
samples – the
fiftyone.core.collections.SampleCollection
usedconfig – the
ClassificationEvaluationConfig
usedeval_key – the evaluation key
ytrue – a list of ground truth labels
ypred – a list of predicted labels
confs – a list of confidences for the predictions
classes – the
(neg_label, pos_label)
label strings for the taskweights (None) – an optional list of sample weights
ytrue_ids (None) – a list of IDs for the ground truth labels
ypred_ids (None) – a list of IDs for the predicted labels
backend (None) – a
ClassificationEvaluation
backend
Methods:
Returns the list of class attributes that will be serialized by
serialize()
.average_precision
([average])Computes the average precision for the results via
sklearn.metrics.average_precision_score()
.confusion_matrix
([classes, include_other])Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
from_dict
(d, samples, config, key)Builds a
RunResults
from a JSON dict representation of it.from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
metrics
([classes, average, beta])Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
plot_confusion_matrix
([classes, …])Plots a confusion matrix for the evaluation results.
plot_pr_curve
([average, backend])Plots a precision-recall (PR) curve for the results.
plot_roc_curve
([backend])Plots a receiver operating characteristic (ROC) curve for the results.
print_report
([classes, digits])Prints a classification report for the results via
sklearn.metrics.classification_report()
.report
([classes])Generates a classification report for the results via
sklearn.metrics.classification_report()
.save
()Saves the results to the database.
Saves these results config to the database.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
Attributes:
The
Run
for these results.The fully-qualified name of this
RunResults
class.The
RunConfig
for these results.The run key for these results.
The
fiftyone.core.collections.SampleCollection
associated with these results.-
average_precision
(average='micro')¶ Computes the average precision for the results via
sklearn.metrics.average_precision_score()
.- Parameters
average ("micro") – the averaging strategy to use
- Returns
the average precision
-
plot_pr_curve
(average='micro', backend='plotly', **kwargs)¶ Plots a precision-recall (PR) curve for the results.
- Parameters
average ("micro") – the averaging strategy to use when computing average precision
backend ("plotly") – the plotting backend to use. Supported values are
("plotly", "matplotlib")
**kwargs –
keyword arguments for the backend plotting method:
”plotly” backend:
fiftyone.core.plots.plotly.plot_pr_curve()
”matplotlib” backend:
fiftyone.core.plots.matplotlib.plot_pr_curve()
- Returns
a
fiftyone.core.plots.plotly.PlotlyNotebookPlot
, if you are working in a notebook context and the plotly backend is useda plotly or matplotlib figure, otherwise
- Return type
one of the following
-
plot_roc_curve
(backend='plotly', **kwargs)¶ Plots a receiver operating characteristic (ROC) curve for the results.
- Parameters
backend ("plotly") – the plotting backend to use. Supported values are
("plotly", "matplotlib")
**kwargs –
keyword arguments for the backend plotting method:
”plotly” backend:
fiftyone.core.plots.plotly.plot_roc_curve()
”matplotlib” backend:
fiftyone.core.plots.matplotlib.plot_roc_curve()
- Returns
a
fiftyone.core.plots.plotly.PlotlyNotebookPlot
, if you are working in a notebook context and the plotly backend is useda plotly or matplotlib figure, otherwise
- Return type
one of the following
-
attributes
()¶ Returns the list of class attributes that will be serialized by
serialize()
.- Returns
a list of attributes
-
property
backend
¶ The
Run
for these results.
-
property
cls
¶ The fully-qualified name of this
RunResults
class.
-
property
config
¶ The
RunConfig
for these results.
-
confusion_matrix
(classes=None, include_other=False)¶ Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.The rows of the confusion matrix represent ground truth and the columns represent predictions.
- Parameters
classes (None) – an optional list of classes to include in the confusion matrix. Include
self.missing
in this list if you would like to include a row/column for unmatched examplesinclude_other (False) – whether to include an extra row/column at the end of the matrix for labels that do not appear in
classes
. Only applicable ifclasses
are provided
- Returns
a
num_classes x num_classes
confusion matrix
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
classmethod
from_dict
(d, samples, config, key)¶ Builds a
RunResults
from a JSON dict representation of it.- Parameters
d – a JSON dict
samples – the
fiftyone.core.collections.SampleCollection
for the runconfig – the
RunConfig
for the runkey – the run key
- Returns
a
RunResults
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
property
key
¶ The run key for these results.
-
metrics
(classes=None, average='micro', beta=1.0)¶ Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
See
sklearn.metrics.precision_recall_fscore_support()
for details.- Parameters
classes (None) – an optional list of classes to include in the calculations
average ("micro") – the averaging strategy to use
beta (1.0) – the F-beta value to use
- Returns
a dict
-
plot_confusion_matrix
(classes=None, include_other=None, include_missing=None, other_label='(other)', backend='plotly', **kwargs)¶ Plots a confusion matrix for the evaluation results.
If you are working in a notebook environment with the default plotly backend, this method returns an interactive
fiftyone.core.plots.plotly.InteractiveHeatmap
that you can attach to an App session via itsfiftyone.core.session.Session.plots
attribute, which will automatically sync the session’s view with the currently selected cells in the confusion matrix.- Parameters
classes (None) – an optional list of classes to include in the confusion matrix
include_other (None) –
whether to include a row/column for examples whose label is in
classes
but are matched to labels that do not appear inclasses
. Only applicable ifclasses
are provided. The supported values are:None (default): only include a row/column for other labels if there are any
True: do include a row/column for other labels
False: do not include a row/column for other labels
include_missing (None) –
whether to include a row/column for missing ground truth/predictions in the confusion matrix. The supported values are:
None (default): only include a row/column for missing labels if there are any
True: do include a row/column for missing labels
False: do not include a row/column for missing labels
other_label ("(other)") – the label to use for “other” predictions
backend ("plotly") – the plotting backend to use. Supported values are
("plotly", "matplotlib")
**kwargs –
keyword arguments for the backend plotting method:
”plotly” backend:
fiftyone.core.plots.plotly.plot_confusion_matrix()
”matplotlib” backend:
fiftyone.core.plots.matplotlib.plot_confusion_matrix()
- Returns
a
fiftyone.core.plots.plotly.InteractiveHeatmap
, if the plotly backend is useda matplotlib figure, otherwise
- Return type
one of the following
-
print_report
(classes=None, digits=2)¶ Prints a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
digits (2) – the number of digits of precision to print
-
report
(classes=None)¶ Generates a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
- Returns
a dict
-
property
samples
¶ The
fiftyone.core.collections.SampleCollection
associated with these results.
-
save
()¶ Saves the results to the database.
-
save_config
()¶ Saves these results config to the database.
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
fiftyone.utils.eval.
evaluate_detections
(samples, pred_field, gt_field='ground_truth', eval_key=None, classes=None, missing=None, method=None, iou=0.5, use_masks=False, use_boxes=False, classwise=True, dynamic=True, **kwargs)¶ Evaluates the predicted detections in the given samples with respect to the specified ground truth detections.
This method supports evaluating the following spatial data types:
Object detections in
fiftyone.core.labels.Detections
formatInstance segmentations in
fiftyone.core.labels.Detections
format with theirmask
attributes populatedPolygons in
fiftyone.core.labels.Polylines
formatKeypoints in
fiftyone.core.labels.Keypoints
formatTemporal detections in
fiftyone.core.labels.TemporalDetections
format
For spatial object detection evaluation, this method uses COCO-style evaluation by default.
When evaluating keypoints, “IoUs” are computed via object keypoint similarity.
For temporal segment detection, this method uses ActivityNet-style evaluation by default.
You can use the
method
parameter to select a different method, and you can optionally customize the method by passing additional parameters for the method’s config class askwargs
.The supported
method
values and their associated configs are:"open-images"
:fiftyone.utils.eval.openimages.OpenImagesEvaluationConfig
"activitynet"
:fiftyone.utils.eval.activitynet.ActivityNetEvaluationConfig
If an
eval_key
is provided, a number of fields are populated at the object- and sample-level recording the results of the evaluation:True positive (TP), false positive (FP), and false negative (FN) counts for each sample are saved in top-level fields of each sample:
TP: sample.<eval_key>_tp FP: sample.<eval_key>_fp FN: sample.<eval_key>_fn
In addition, when evaluating frame-level objects, TP/FP/FN counts are recorded for each frame:
TP: frame.<eval_key>_tp FP: frame.<eval_key>_fp FN: frame.<eval_key>_fn
The fields listed below are populated on each individual object; these fields tabulate the TP/FP/FN status of the object, the ID of the matching object (if any), and the matching IoU:
TP/FP/FN: object.<eval_key> ID: object.<eval_key>_id IoU: object.<eval_key>_iou
- Parameters
samples – a
fiftyone.core.collections.SampleCollection
pred_field – the name of the field containing the predicted
fiftyone.core.labels.Detections
,fiftyone.core.labels.Polylines
,fiftyone.core.labels.Keypoints
, orfiftyone.core.labels.TemporalDetections
gt_field ("ground_truth") – the name of the field containing the ground truth
fiftyone.core.labels.Detections
,fiftyone.core.labels.Polylines
,fiftyone.core.labels.Keypoints
, orfiftyone.core.labels.TemporalDetections
eval_key (None) – an evaluation key to use to refer to this evaluation
classes (None) – the list of possible classes. If not provided, the observed ground truth/predicted labels are used
missing (None) – a missing label string. Any unmatched objects are given this label for results purposes
method (None) – a string specifying the evaluation method to use. For spatial object detection, the supported values are
("coco", "open-images")
and the default is"coco"
. For temporal segment detection, the supported values are("activitynet")
and the default is"activitynet"
iou (0.50) – the IoU threshold to use to determine matches
use_masks (False) – whether to compute IoUs using the instances masks in the
mask
attribute of the provided objects, which must befiftyone.core.labels.Detection
instancesuse_boxes (False) – whether to compute IoUs using the bounding boxes of the provided
fiftyone.core.labels.Polyline
instances rather than using their actual geometriesclasswise (True) – whether to only match objects with the same class label (True) or allow matches between classes (False)
dynamic (True) – whether to declare the dynamic object-level attributes that are populated on the dataset’s schema
**kwargs – optional keyword arguments for the constructor of the
DetectionEvaluationConfig
being used
- Returns
-
class
fiftyone.utils.eval.
DetectionResults
(samples, config, eval_key, matches, classes=None, missing=None, backend=None)¶ Bases:
fiftyone.utils.eval.base.BaseEvaluationResults
Class that stores the results of a detection evaluation.
- Parameters
samples – the
fiftyone.core.collections.SampleCollection
usedconfig – the
DetectionEvaluationConfig
usedeval_key – the evaluation key
matches – a list of
(gt_label, pred_label, iou, pred_confidence, gt_id, pred_id)
matches. Either label can beNone
to indicate an unmatched objectclasses (None) – the list of possible classes. If not provided, the observed ground truth/predicted labels are used
missing (None) – a missing label string. Any unmatched objects are given this label for evaluation purposes
backend (None) – a
DetectionEvaluation
backend
Methods:
Returns the list of class attributes that will be serialized by
serialize()
.confusion_matrix
([classes, include_other])Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
from_dict
(d, samples, config, key)Builds a
RunResults
from a JSON dict representation of it.from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
metrics
([classes, average, beta])Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
plot_confusion_matrix
([classes, …])Plots a confusion matrix for the evaluation results.
print_report
([classes, digits])Prints a classification report for the results via
sklearn.metrics.classification_report()
.report
([classes])Generates a classification report for the results via
sklearn.metrics.classification_report()
.save
()Saves the results to the database.
Saves these results config to the database.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
Attributes:
The
Run
for these results.The fully-qualified name of this
RunResults
class.The
RunConfig
for these results.The run key for these results.
The
fiftyone.core.collections.SampleCollection
associated with these results.-
attributes
()¶ Returns the list of class attributes that will be serialized by
serialize()
.- Returns
a list of attributes
-
property
backend
¶ The
Run
for these results.
-
property
cls
¶ The fully-qualified name of this
RunResults
class.
-
property
config
¶ The
RunConfig
for these results.
-
confusion_matrix
(classes=None, include_other=False)¶ Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.The rows of the confusion matrix represent ground truth and the columns represent predictions.
- Parameters
classes (None) – an optional list of classes to include in the confusion matrix. Include
self.missing
in this list if you would like to include a row/column for unmatched examplesinclude_other (False) – whether to include an extra row/column at the end of the matrix for labels that do not appear in
classes
. Only applicable ifclasses
are provided
- Returns
a
num_classes x num_classes
confusion matrix
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
classmethod
from_dict
(d, samples, config, key)¶ Builds a
RunResults
from a JSON dict representation of it.- Parameters
d – a JSON dict
samples – the
fiftyone.core.collections.SampleCollection
for the runconfig – the
RunConfig
for the runkey – the run key
- Returns
a
RunResults
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
property
key
¶ The run key for these results.
-
metrics
(classes=None, average='micro', beta=1.0)¶ Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
See
sklearn.metrics.precision_recall_fscore_support()
for details.- Parameters
classes (None) – an optional list of classes to include in the calculations
average ("micro") – the averaging strategy to use
beta (1.0) – the F-beta value to use
- Returns
a dict
-
plot_confusion_matrix
(classes=None, include_other=None, include_missing=None, other_label='(other)', backend='plotly', **kwargs)¶ Plots a confusion matrix for the evaluation results.
If you are working in a notebook environment with the default plotly backend, this method returns an interactive
fiftyone.core.plots.plotly.InteractiveHeatmap
that you can attach to an App session via itsfiftyone.core.session.Session.plots
attribute, which will automatically sync the session’s view with the currently selected cells in the confusion matrix.- Parameters
classes (None) – an optional list of classes to include in the confusion matrix
include_other (None) –
whether to include a row/column for examples whose label is in
classes
but are matched to labels that do not appear inclasses
. Only applicable ifclasses
are provided. The supported values are:None (default): only include a row/column for other labels if there are any
True: do include a row/column for other labels
False: do not include a row/column for other labels
include_missing (None) –
whether to include a row/column for missing ground truth/predictions in the confusion matrix. The supported values are:
None (default): only include a row/column for missing labels if there are any
True: do include a row/column for missing labels
False: do not include a row/column for missing labels
other_label ("(other)") – the label to use for “other” predictions
backend ("plotly") – the plotting backend to use. Supported values are
("plotly", "matplotlib")
**kwargs –
keyword arguments for the backend plotting method:
”plotly” backend:
fiftyone.core.plots.plotly.plot_confusion_matrix()
”matplotlib” backend:
fiftyone.core.plots.matplotlib.plot_confusion_matrix()
- Returns
a
fiftyone.core.plots.plotly.InteractiveHeatmap
, if the plotly backend is useda matplotlib figure, otherwise
- Return type
one of the following
-
print_report
(classes=None, digits=2)¶ Prints a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
digits (2) – the number of digits of precision to print
-
report
(classes=None)¶ Generates a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
- Returns
a dict
-
property
samples
¶ The
fiftyone.core.collections.SampleCollection
associated with these results.
-
save
()¶ Saves the results to the database.
-
save_config
()¶ Saves these results config to the database.
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
fiftyone.utils.eval.
evaluate_regressions
(samples, pred_field, gt_field='ground_truth', eval_key=None, missing=None, method='simple', **kwargs)¶ Evaluates the regression predictions in the given collection with respect to the specified ground truth values.
You can customize the evaluation method by passing additional parameters for the method’s config class as
kwargs
.The supported
method
values and their associated configs are:"simple"
:SimpleEvaluationConfig
If an
eval_key
is specified, then this method will record some statistics on each sample:When evaluating sample-level fields, an
eval_key
field will be populated on each sample recording the error of that sample’s prediction.When evaluating frame-level fields, an
eval_key
field will be populated on each frame recording the error of that frame’s prediction. In addition, aneval_key
field will be populated on each sample that records the average error of the frame predictions of the sample.
- Parameters
samples – a
fiftyone.core.collections.SampleCollection
pred_field – the name of the field containing the predicted
fiftyone.core.labels.Regression
instancesgt_field ("ground_truth") – the name of the field containing the ground truth
fiftyone.core.labels.Regression
instanceseval_key (None) – a string key to use to refer to this evaluation
missing (None) – a missing value. Any None-valued regressions are given this value for results purposes
method ("simple") – a string specifying the evaluation method to use. Supported values are
("simple")
**kwargs – optional keyword arguments for the constructor of the
RegressionEvaluationConfig
being used
- Returns
-
class
fiftyone.utils.eval.
RegressionResults
(samples, config, eval_key, ytrue, ypred, confs=None, ids=None, missing=None, backend=None)¶ Bases:
fiftyone.core.evaluation.EvaluationResults
Class that stores the results of a regression evaluation.
- Parameters
samples – the
fiftyone.core.collections.SampleCollection
usedconfig – the
RegressionEvaluationConfig
usedeval_key (None) – the evaluation key
ytrue – a list of ground truth values
ypred – a list of predicted values
confs (None) – an optional list of confidences for the predictions
eval_key – the evaluation key of the evaluation
gt_field (None) – the name of the ground truth field
pred_field (None) – the name of the predictions field
ids (None) – a list of sample or frame IDs corresponding to the regressions
missing (None) – a missing value. Any None-valued regressions are given this value for results purposes
backend (None) – a
RegressionEvaluation
backend
Methods:
Returns the list of class attributes that will be serialized by
serialize()
.copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
from_dict
(d, samples, config, key)Builds a
RunResults
from a JSON dict representation of it.from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
metrics
([weights])Computes various popular regression metrics for the results.
plot_results
([labels, sizes, backend])Plots the regression results.
print_metrics
([weights, digits])Prints the regression metrics computed via
metrics()
.save
()Saves the results to the database.
Saves these results config to the database.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
Attributes:
The
Run
for these results.The fully-qualified name of this
RunResults
class.The
RunConfig
for these results.The run key for these results.
The
fiftyone.core.collections.SampleCollection
associated with these results.-
metrics
(weights=None)¶ Computes various popular regression metrics for the results.
The computed metrics are:
Mean squared error:
sklearn.metrics.mean_squared_error()
Root mean squared error:
sklearn.metrics.mean_squared_error()
Mean absolute error:
sklearn.metrics.mean_absolute_error()
Median absolute error:
sklearn.metrics.median_absolute_error()
R^2 score:
sklearn.metrics.r2_score()
Explained variance score:
sklearn.metrics.explained_variance_score()
Max error:
sklearn.metrics.max_error()
Support: the number of examples
- Parameters
weights (None) – an optional list of weights for each example
- Returns
a dict
-
print_metrics
(weights=None, digits=2)¶ Prints the regression metrics computed via
metrics()
.- Parameters
weights (None) – an optional list of weights for each example
digits (2) – the number of digits of precision to print
-
plot_results
(labels=None, sizes=None, backend='plotly', **kwargs)¶ Plots the regression results.
You can use the
labels
parameters to define a coloring for the points, and you can use thesizes
parameter to scale the sizes of the points.You can attach plots generated by this method to an App session via its
fiftyone.core.session.Session.plots
attribute, which will automatically sync the session’s view with the currently selected points in the plot.- Parameters
labels (None) –
data to use to color the points. Can be any of the following:
the name of a sample field or
embedded.field.name
of from which to extract numeric or string valuesa
fiftyone.core.expressions.ViewExpression
defining numeric or string values to extract viafiftyone.core.collections.SampleCollection.values()
a list or array-like of numeric or string values (or lists of lists for frame-level regressions)
sizes (None) –
data to use to scale the sizes of the points. Can be any of the following:
the name of a sample field or
embedded.field.name
from which to extract numeric valuesa
fiftyone.core.expressions.ViewExpression
defining numeric values to extract viafiftyone.core.collections.SampleCollection.values()
a list or array-like of numeric values (or lists of lists for frame-level regressions)
backend ("plotly") – the plotting backend to use. Supported values are
("plotly", "matplotlib")
**kwargs –
keyword arguments for the backend plotting method:
”plotly” backend:
fiftyone.core.plots.plotly.plot_regressions()
”matplotlib” backend:
fiftyone.core.plots.matplotlib.plot_regressions()
- Returns
-
attributes
()¶ Returns the list of class attributes that will be serialized by
serialize()
.- Returns
a list of attributes
-
property
backend
¶ The
Run
for these results.
-
property
cls
¶ The fully-qualified name of this
RunResults
class.
-
property
config
¶ The
RunConfig
for these results.
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
classmethod
from_dict
(d, samples, config, key)¶ Builds a
RunResults
from a JSON dict representation of it.- Parameters
d – a JSON dict
samples – the
fiftyone.core.collections.SampleCollection
for the runconfig – the
RunConfig
for the runkey – the run key
- Returns
a
RunResults
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
property
key
¶ The run key for these results.
-
property
samples
¶ The
fiftyone.core.collections.SampleCollection
associated with these results.
-
save
()¶ Saves the results to the database.
-
save_config
()¶ Saves these results config to the database.
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()
-
fiftyone.utils.eval.
evaluate_segmentations
(samples, pred_field, gt_field='ground_truth', eval_key=None, mask_targets=None, method='simple', **kwargs)¶ Evaluates the specified semantic segmentation masks in the given collection with respect to the specified ground truth masks.
If the size of a predicted mask does not match the ground truth mask, it is resized to match the ground truth.
By default, this method simply performs pixelwise evaluation of the full masks, but other strategies such as boundary-only evaluation can be configured by passing additional parameters for the method’s config class as
kwargs
.The supported
method
values and their associated configs are:"simple"
:SimpleEvaluationConfig
If an
eval_key
is provided, the accuracy, precision, and recall of each sample is recorded in top-level fields of each sample:Accuracy: sample.<eval_key>_accuracy Precision: sample.<eval_key>_precision Recall: sample.<eval_key>_recall
In addition, when evaluating frame-level masks, the accuracy, precision, and recall of each frame if recorded in the following frame-level fields:
Accuracy: frame.<eval_key>_accuracy Precision: frame.<eval_key>_precision Recall: frame.<eval_key>_recall
Note
The mask values
0
and#000000
are treated as a background class for the purposes of computing evaluation metrics like precision and recall.- Parameters
samples – a
fiftyone.core.collections.SampleCollection
pred_field – the name of the field containing the predicted
fiftyone.core.labels.Segmentation
instancesgt_field ("ground_truth") – the name of the field containing the ground truth
fiftyone.core.labels.Segmentation
instanceseval_key (None) – an evaluation key to use to refer to this evaluation
mask_targets (None) – a dict mapping pixel values or RGB hex strings to labels. If not provided, the observed values are used as labels
method ("simple") – a string specifying the evaluation method to use. Supported values are
("simple")
**kwargs – optional keyword arguments for the constructor of the
SegmentationEvaluationConfig
being used
- Returns
-
class
fiftyone.utils.eval.
SegmentationResults
(samples, config, eval_key, pixel_confusion_matrix, classes, missing=None, backend=None)¶ Bases:
fiftyone.utils.eval.base.BaseEvaluationResults
Class that stores the results of a segmentation evaluation.
- Parameters
samples – the
fiftyone.core.collections.SampleCollection
usedconfig – the
SegmentationEvaluationConfig
usedeval_key – the evaluation key
pixel_confusion_matrix – a pixel value confusion matrix
classes – a list of class labels corresponding to the confusion matrix
missing (None) – a missing (background) class
backend (None) – a
SegmentationEvaluation
backend
Methods:
Returns the list of class attributes that will be serialized by
serialize()
.confusion_matrix
([classes, include_other])Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.copy
()Returns a deep copy of the object.
custom_attributes
([dynamic, private])Returns a customizable list of class attributes.
Computes the Dice score across all samples in the evaluation.
from_dict
(d, samples, config, key)Builds a
RunResults
from a JSON dict representation of it.from_json
(path, *args, **kwargs)Constructs a Serializable object from a JSON file.
from_str
(s, *args, **kwargs)Constructs a Serializable object from a JSON string.
Returns the fully-qualified class name string of this object.
metrics
([classes, average, beta])Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
plot_confusion_matrix
([classes, …])Plots a confusion matrix for the evaluation results.
print_report
([classes, digits])Prints a classification report for the results via
sklearn.metrics.classification_report()
.report
([classes])Generates a classification report for the results via
sklearn.metrics.classification_report()
.save
()Saves the results to the database.
Saves these results config to the database.
serialize
([reflective])Serializes the object into a dictionary.
to_str
([pretty_print])Returns a string representation of this object.
write_json
(path[, pretty_print])Serializes the object and writes it to disk.
Attributes:
The
Run
for these results.The fully-qualified name of this
RunResults
class.The
RunConfig
for these results.The run key for these results.
The
fiftyone.core.collections.SampleCollection
associated with these results.-
attributes
()¶ Returns the list of class attributes that will be serialized by
serialize()
.- Returns
a list of attributes
-
dice_score
()¶ Computes the Dice score across all samples in the evaluation.
- Returns
the Dice score in
[0, 1]
-
property
backend
¶ The
Run
for these results.
-
property
cls
¶ The fully-qualified name of this
RunResults
class.
-
property
config
¶ The
RunConfig
for these results.
-
confusion_matrix
(classes=None, include_other=False)¶ Generates a confusion matrix for the results via
sklearn.metrics.confusion_matrix()
.The rows of the confusion matrix represent ground truth and the columns represent predictions.
- Parameters
classes (None) – an optional list of classes to include in the confusion matrix. Include
self.missing
in this list if you would like to include a row/column for unmatched examplesinclude_other (False) – whether to include an extra row/column at the end of the matrix for labels that do not appear in
classes
. Only applicable ifclasses
are provided
- Returns
a
num_classes x num_classes
confusion matrix
-
copy
()¶ Returns a deep copy of the object.
- Returns
a Serializable instance
-
custom_attributes
(dynamic=False, private=False)¶ Returns a customizable list of class attributes.
By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).
- Parameters
dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False
- Returns
a list of class attributes
-
classmethod
from_dict
(d, samples, config, key)¶ Builds a
RunResults
from a JSON dict representation of it.- Parameters
d – a JSON dict
samples – the
fiftyone.core.collections.SampleCollection
for the runconfig – the
RunConfig
for the runkey – the run key
- Returns
a
RunResults
-
classmethod
from_json
(path, *args, **kwargs)¶ Constructs a Serializable object from a JSON file.
Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.
- Parameters
path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
from_str
(s, *args, **kwargs)¶ Constructs a Serializable object from a JSON string.
Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.
- Parameters
s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()
- Returns
an instance of the Serializable class
-
classmethod
get_class_name
()¶ Returns the fully-qualified class name string of this object.
-
property
key
¶ The run key for these results.
-
metrics
(classes=None, average='micro', beta=1.0)¶ Computes classification metrics for the results, including accuracy, precision, recall, and F-beta score.
See
sklearn.metrics.precision_recall_fscore_support()
for details.- Parameters
classes (None) – an optional list of classes to include in the calculations
average ("micro") – the averaging strategy to use
beta (1.0) – the F-beta value to use
- Returns
a dict
-
plot_confusion_matrix
(classes=None, include_other=None, include_missing=None, other_label='(other)', backend='plotly', **kwargs)¶ Plots a confusion matrix for the evaluation results.
If you are working in a notebook environment with the default plotly backend, this method returns an interactive
fiftyone.core.plots.plotly.InteractiveHeatmap
that you can attach to an App session via itsfiftyone.core.session.Session.plots
attribute, which will automatically sync the session’s view with the currently selected cells in the confusion matrix.- Parameters
classes (None) – an optional list of classes to include in the confusion matrix
include_other (None) –
whether to include a row/column for examples whose label is in
classes
but are matched to labels that do not appear inclasses
. Only applicable ifclasses
are provided. The supported values are:None (default): only include a row/column for other labels if there are any
True: do include a row/column for other labels
False: do not include a row/column for other labels
include_missing (None) –
whether to include a row/column for missing ground truth/predictions in the confusion matrix. The supported values are:
None (default): only include a row/column for missing labels if there are any
True: do include a row/column for missing labels
False: do not include a row/column for missing labels
other_label ("(other)") – the label to use for “other” predictions
backend ("plotly") – the plotting backend to use. Supported values are
("plotly", "matplotlib")
**kwargs –
keyword arguments for the backend plotting method:
”plotly” backend:
fiftyone.core.plots.plotly.plot_confusion_matrix()
”matplotlib” backend:
fiftyone.core.plots.matplotlib.plot_confusion_matrix()
- Returns
a
fiftyone.core.plots.plotly.InteractiveHeatmap
, if the plotly backend is useda matplotlib figure, otherwise
- Return type
one of the following
-
print_report
(classes=None, digits=2)¶ Prints a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
digits (2) – the number of digits of precision to print
-
report
(classes=None)¶ Generates a classification report for the results via
sklearn.metrics.classification_report()
.- Parameters
classes (None) – an optional list of classes to include in the report
- Returns
a dict
-
property
samples
¶ The
fiftyone.core.collections.SampleCollection
associated with these results.
-
save
()¶ Saves the results to the database.
-
save_config
()¶ Saves these results config to the database.
-
serialize
(reflective=False)¶ Serializes the object into a dictionary.
Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.
- Parameters
reflective – whether to include reflective attributes when serializing the object. By default, this is False
- Returns
a JSON dictionary representation of the object
-
to_str
(pretty_print=True, **kwargs)¶ Returns a string representation of this object.
- Parameters
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()
- Returns
a string representation of the object
-
write_json
(path, pretty_print=False, **kwargs)¶ Serializes the object and writes it to disk.
- Parameters
path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()