fiftyone.core.odm.sample¶
Backing document classes for fiftyone.core.sample.Sample
instances.
Class hierarchy:
SerializableDocument
├── NoDatasetSampleDocument
└── DatasetSampleDocument
├── my_custom_dataset
├── another_dataset
└── ...
Design invariants:
A
fiftyone.core.sample.Sample
always has a backing_doc
that is an instance offiftyone.core.odm.document.SerializableDocument
A
fiftyone.core.dataset.Dataset
always has a backing _sample_doc_cls that is a subclass ofDatasetSampleDocument
Implementation details
When a new fiftyone.core.sample.Sample
is created, its _doc
attribute is an instance of NoDatasetSampleDocument
:
import fiftyone as fo
sample = fo.Sample()
sample._doc # NoDatasetSampleDocument
When a new fiftyone.core.dataset.Dataset
is created, its
_sample_doc_cls
attribute holds a dynamically created subclass of
DatasetSampleDocument
whose name is the name of the dataset’s sample
collection:
dataset = fo.Dataset(name="my_dataset")
dataset._sample_doc_cls # my_dataset(DatasetSampleDocument)
When a sample is added to a dataset, its _doc
attribute is changed from
type NoDatasetSampleDocument
to type dataset._sample_doc_cls
:
dataset.add_sample(sample)
sample._doc # my_dataset(DatasetSampleDocument)
Classes:
|
Base class for sample documents backing samples in datasets. |
|
Backing document for samples that have not been added to a dataset. |
-
class
fiftyone.core.odm.sample.
DatasetSampleDocument
(**kwargs)¶ Bases:
fiftyone.core.odm.mixins.DatasetMixin
,fiftyone.core.odm.document.Document
Base class for sample documents backing samples in datasets.
All
fiftyone.core.dataset.Dataset._sample_doc_cls
classes inherit from this class.Attributes:
An Object ID field.
A unicode string field.
A list field that wraps a standard
Field
, allowing multiple instances of the field to be stored as a list in the database.A field that stores instances of a given type of
fiftyone.core.odm.BaseEmbeddedDocument
object.A datetime field.
A datetime field.
An ordered tuple of the public fields of this document.
Whether the document has been inserted into the database.
Get the primary key.
Methods:
add_field
(path, ftype[, embedded_doc_type, …])Adds a new field or embedded field to the document, if necessary.
add_implied_field
(path, value[, …])Adds the field or embedded field to the document, if necessary, inferring the field type from the provided value.
cascade_save
(**kwargs)Recursively save any references and generic references on the document.
clean
()Hook for doing document level data cleaning (usually validation or assignment) before validation is run.
clear_field
(field_name)Clears the field from the document.
Compares the indexes defined in MongoEngine with the ones existing in the database.
copy
([new_id])Returns a deep copy of the document.
create_index
(keys[, background])Creates the given indexes if required.
delete
([signal_kwargs])Delete the
Document
from the database.Drops the entire collection associated with this
Document
type from the database.ensure_index
(key_or_list[, background])Ensure that the given indexes are in place.
Checks the document meta data and ensures all the indexes exist.
fancy_repr
([class_name, select_fields, …])Generates a customizable string representation of the document.
field_to_mongo
(field_name)field_to_python
(field_name, value)from_dict
(d[, extended])Loads the document from a BSON/JSON dictionary.
from_json
(s)Loads the document from a JSON string.
get_field
(field_name)Gets the field of the document.
get_field_schema
([ftype, embedded_doc_type, …])Returns a schema dictionary describing the fields of this document.
Get text score from text query
has_field
(field_name)Determines whether the document has a field of the given name.
Returns an iterator over the
(name, value)
pairs of the public fields of the document.Lists all indexes that should be created for the Document collection.
merge
(doc[, merge_lists, merge_dicts, overwrite])Merges the contents of the given document into this document.
merge_field_schema
(schema[, expand_schema, …])Merges the field schema into this document.
modify
([query])Perform an atomic update of the document in the database and reload the document object using updated version.
register_delete_rule
(document_cls, …)This method registers the delete rules to apply when removing this object.
reload
(*fields, **kwargs)Reloads the document from the database.
save
([upsert, validate, safe])Saves the document to the database.
select_related
([max_depth])Handles dereferencing of
DBRef
objects to a maximum depth in order to cut down the number queries to mongodb.set_field
(field_name, value[, create, …])Sets the value of a field of the document.
switch_collection
(collection_name[, …])Temporarily switch the collection for a document instance.
switch_db
(db_alias[, keep_created])Temporarily switch the database for a document instance.
to_dbref
()Returns an instance of
DBRef
useful in __raw__ queries.to_dict
([extended])Serializes this document to a BSON/JSON dictionary.
to_json
([pretty_print])Serializes the document to a JSON string.
to_mongo
(*args, **kwargs)Return as SON data ready for use with MongoDB.
update
(**kwargs)Performs an update on the
Document
A convenience wrapper toupdate()
.validate
([clean])Ensure that all fields’ values are valid and that required fields are present.
Classes:
alias of
mongoengine.base.metaclasses.TopLevelDocumentMetaclass
-
id
¶ An Object ID field.
- Parameters
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created
-
filepath
¶ A unicode string field.
- Parameters
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created
A list field that wraps a standard
Field
, allowing multiple instances of the field to be stored as a list in the database.If this field is not set, its default value is
[]
.- Parameters
field (None) – an optional
Field
instance describing the type of the list elementsdescription (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created
-
metadata
¶ A field that stores instances of a given type of
fiftyone.core.odm.BaseEmbeddedDocument
object.- Parameters
document_type – the
fiftyone.core.odm.BaseEmbeddedDocument
type stored in this fielddescription (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created
-
created_at
¶ A datetime field.
- Parameters
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created
-
last_modified_at
¶ A datetime field.
- Parameters
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field is read-only
created_at (None) – the datetime the field was created
-
property
media_type
¶
-
STRICT
= False¶
-
classmethod
add_field
(path, ftype, embedded_doc_type=None, subfield=None, fields=None, description=None, info=None, read_only=False, expand_schema=True, recursive=True, validate=True, **kwargs)¶ Adds a new field or embedded field to the document, if necessary.
- Parameters
path – the field name or
embedded.field.name
ftype – the field type to create. Must be a subclass of
fiftyone.core.fields.Field
embedded_doc_type (None) – the
fiftyone.core.odm.BaseEmbeddedDocument
type of the field. Only applicable whenftype
isfiftyone.core.fields.EmbeddedDocumentField
subfield (None) – the
fiftyone.core.fields.Field
type of the contained field. Only applicable whenftype
isfiftyone.core.fields.ListField
orfiftyone.core.fields.DictField
fields (None) – a list of
fiftyone.core.fields.Field
instances defining embedded document attributes. Only applicable whenftype
isfiftyone.core.fields.EmbeddedDocumentField
description (None) – an optional description
info (None) – an optional info dict
read_only (False) – whether the field should be read-only
expand_schema (True) – whether to add new fields to the schema (True) or simply validate that the field already exists with a consistent type (False)
recursive (True) – whether to recursively add embedded document fields
validate (True) – whether to validate the field against an existing field at the same path
- Returns
True/False whether one or more fields or embedded fields were added to the document or its children
- Raises
ValueError – if a field in the schema is not compliant with an existing field of the same name
-
classmethod
add_implied_field
(path, value, expand_schema=True, dynamic=False, recursive=True, validate=True)¶ Adds the field or embedded field to the document, if necessary, inferring the field type from the provided value.
- Parameters
path – the field name or
embedded.field.name
value – the field value
expand_schema (True) – whether to add new fields to the schema (True) or simply validate that the field already exists with a consistent type (False)
dynamic (False) – whether to declare dynamic embedded document fields
recursive (True) – whether to recursively add embedded document fields
validate (True) – whether to validate the field against an existing field at the same path
- Returns
True/False whether one or more fields or embedded fields were added to the document or its children
- Raises
ValueError – if a field in the schema is not compliant with an existing field of the same name
-
cascade_save
(**kwargs)¶ Recursively save any references and generic references on the document.
-
clean
()¶ Hook for doing document level data cleaning (usually validation or assignment) before validation is run.
Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.
-
clear_field
(field_name)¶ Clears the field from the document.
- Parameters
field_name – the field name
- Raises
ValueError – if the field does not exist
-
property
collection_name
¶
-
classmethod
compare_indexes
()¶ Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.
-
copy
(new_id=False)¶ Returns a deep copy of the document.
- Parameters
new_id (False) – whether to generate a new ID for the copied document. By default, the ID is left as
None
and will be automatically populated when the document is added to the database
-
copy_with_new_id
()¶
-
classmethod
create_index
(keys, background=False, **kwargs)¶ Creates the given indexes if required.
- Parameters
keys – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering
background – Allows index creation in the background
-
delete
(signal_kwargs=None, **write_concern)¶ Delete the
Document
from the database. This will only take effect if the document has been previously saved.- Parameters
signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.
write_concern – Extra keyword arguments are passed down which will be used as options for the resultant
getLastError
command. For example,save(..., w: 2, fsync: True)
will wait until at least two servers have recorded the write and will force an fsync on the primary server.
-
classmethod
drop_collection
()¶ Drops the entire collection associated with this
Document
type from the database.Raises
OperationError
if the document has no collection set (i.g. if it is abstract)
-
classmethod
ensure_index
(key_or_list, background=False, **kwargs)¶ Ensure that the given indexes are in place. Deprecated in favour of create_index.
- Parameters
key_or_list – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering
background – Allows index creation in the background
-
classmethod
ensure_indexes
()¶ Checks the document meta data and ensures all the indexes exist.
Global defaults can be set in the meta - see Defining documents
By default, this will get called automatically upon first interaction with the Document collection (query, save, etc) so unless you disabled auto_create_index, you shouldn’t have to call this manually.
Note
You can disable automatic index creation by setting auto_create_index to False in the documents meta data
-
fancy_repr
(class_name=None, select_fields=None, exclude_fields=None, **kwargs)¶ Generates a customizable string representation of the document.
- Parameters
class_name (None) – optional class name to use
select_fields (None) – iterable of field names to restrict to
exclude_fields (None) – iterable of field names to exclude
**kwargs – additional key-value pairs to include in the string representation
- Returns
a string representation of the document
-
property
field_names
¶ An ordered tuple of the public fields of this document.
-
field_to_mongo
(field_name)¶
-
field_to_python
(field_name, value)¶
-
classmethod
from_dict
(d, extended=False)¶ Loads the document from a BSON/JSON dictionary.
- Parameters
d – a dictionary
extended (False) – whether the input dictionary may contain serialized extended JSON constructs
- Returns
a
SerializableDocument
-
classmethod
from_json
(s)¶ Loads the document from a JSON string.
- Returns
a
SerializableDocument
-
get_field
(field_name)¶ Gets the field of the document.
- Parameters
field_name – the field name
- Returns
the field value
- Raises
AttributeError – if the field does not exist
-
classmethod
get_field_schema
(ftype=None, embedded_doc_type=None, read_only=None, info_keys=None, created_after=None, include_private=False, flat=False, mode=None)¶ Returns a schema dictionary describing the fields of this document.
If the document belongs to a dataset, the schema will apply to all documents in the collection.
- Parameters
ftype (None) – an optional field type or iterable of field types to which to restrict the returned schema. Must be subclass(es) of
fiftyone.core.fields.Field
embedded_doc_type (None) – an optional embedded document type or iterable of types to which to restrict the returned schema. Must be subclass(es) of
fiftyone.core.odm.BaseEmbeddedDocument
read_only (None) – whether to restrict to (True) or exclude (False) read-only fields. By default, all fields are included
info_keys (None) – an optional key or list of keys that must be in the field’s
info
dictcreated_after (None) – an optional
datetime
specifying a minimum creation dateinclude_private (False) – whether to include fields that start with
_
in the returned schemaflat (False) – whether to return a flattened schema where all embedded document fields are included as top-level keys
mode (None) – whether to apply the above constraints before and/or after flattening the schema. Only applicable when ``flat` is True. Supported values are
("before", "after", "both")
. The default is"after"
- Returns
a dict mapping field names to
fiftyone.core.fields.Field
instances
-
get_text_score
()¶ Get text score from text query
-
has_field
(field_name)¶ Determines whether the document has a field of the given name.
- Parameters
field_name – the field name
- Returns
True/False
-
property
in_db
¶ Whether the document has been inserted into the database.
-
iter_fields
()¶ Returns an iterator over the
(name, value)
pairs of the public fields of the document.- Returns
an iterator that emits
(name, value)
tuples
-
classmethod
list_indexes
()¶ Lists all indexes that should be created for the Document collection. It includes all the indexes from super- and sub-classes.
Note that it will only return the indexes’ fields, not the indexes’ options
-
merge
(doc, merge_lists=True, merge_dicts=True, overwrite=True)¶ Merges the contents of the given document into this document.
- Parameters
doc – a
SerializableDocument
of same type as this documentmerge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value
merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value
overwrite (True) – whether to overwrite (True) or skip (False) existing fields
-
classmethod
merge_field_schema
(schema, expand_schema=True, recursive=True, validate=True, overwrite=False)¶ Merges the field schema into this document.
- Parameters
schema – a dict mapping field names or
embedded.field.names
tofiftyone.core.fields.Field
instancesexpand_schema (True) – whether to add new fields to the schema (True) or simply validate that fields already exist with consistent types (False)
recursive (True) – whether to recursively merge embedded document fields
validate (True) – whether to validate fields against existing fields at the same path
overwrite (False) – whether to overwrite the editable metadata of existing fields
- Returns
True/False whether any new fields were added
- Raises
ValueError – if a field in the schema is not compliant with an existing field of the same name or a new field is found but
expand_schema == False
-
modify
(query=None, **update)¶ Perform an atomic update of the document in the database and reload the document object using updated version.
Returns True if the document has been updated or False if the document in the database doesn’t match the query.
Note
All unsaved changes that have been made to the document are rejected if the method returns True.
- Parameters
query – the update will be performed only if the document in the database matches the query
update – Django-style update keyword arguments
-
my_metaclass
¶ alias of
mongoengine.base.metaclasses.TopLevelDocumentMetaclass
Methods:get_auto_id_names
(new_class)Find a name for the automatic ID field for the given new class.
mro
()Return a type’s method resolution order.
-
property
pk
¶ Get the primary key.
-
classmethod
register_delete_rule
(document_cls, field_name, rule)¶ This method registers the delete rules to apply when removing this object.
-
reload
(*fields, **kwargs)¶ Reloads the document from the database.
- Parameters
*fields – an optional args list of specific fields to reload
-
save
(upsert=False, validate=True, safe=False, **kwargs)¶ Saves the document to the database.
If the document already exists, it will be updated, otherwise it will be created.
- Parameters
upsert (False) – whether to insert the document if it has an
id
populated but no document with that ID exists in the databasevalidate (True) – whether to validate the document
safe (False) – whether to
reload()
the document before raising any errors
- Returns
self
Handles dereferencing of
DBRef
objects to a maximum depth in order to cut down the number queries to mongodb.
-
set_field
(field_name, value, create=True, validate=True, dynamic=False, _enforce_read_only=True)¶ Sets the value of a field of the document.
- Parameters
field_name – the field name
value – the field value
create (True) – whether to create the field if it does not exist
- Raises
ValueError – if
field_name
is not an allowed field name or does not exist andcreate == False
-
switch_collection
(collection_name, keep_created=True)¶ Temporarily switch the collection for a document instance.
Only really useful for archiving off data and calling save():
user = User.objects.get(id=user_id) user.switch_collection('old-users') user.save()
- Parameters
collection_name (str) – The database alias to use for saving the document
keep_created (bool) – keep self._created value after switching collection, else is reset to True
See also
Use
switch_db
if you need to read from another database
-
switch_db
(db_alias, keep_created=True)¶ Temporarily switch the database for a document instance.
Only really useful for archiving off data and calling save():
user = User.objects.get(id=user_id) user.switch_db('archive-db') user.save()
- Parameters
db_alias (str) – The database alias to use for saving the document
keep_created (bool) – keep self._created value after switching db, else is reset to True
See also
Use
switch_collection
if you need to read from another collection
-
to_dict
(extended=False)¶ Serializes this document to a BSON/JSON dictionary.
- Parameters
extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format
- Returns
a dict
-
to_json
(pretty_print=False)¶ Serializes the document to a JSON string.
- Parameters
pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations
- Returns
a JSON string
-
to_mongo
(*args, **kwargs)¶ Return as SON data ready for use with MongoDB.
-
update
(**kwargs)¶ Performs an update on the
Document
A convenience wrapper toupdate()
.Raises
OperationError
if called on an object that has not yet been saved.
-
validate
(clean=True)¶ Ensure that all fields’ values are valid and that required fields are present.
Raises
ValidationError
if any of the fields’ values are found to be invalid.
-
-
class
fiftyone.core.odm.sample.
NoDatasetSampleDocument
(**kwargs)¶ Bases:
fiftyone.core.odm.mixins.NoDatasetMixin
,fiftyone.core.odm.document.SerializableDocument
Backing document for samples that have not been added to a dataset.
Attributes:
An ordered tuple of the public fields of this document.
Methods:
clear_field
(field_name)Clears the field from the document.
copy
()Returns a deep copy of the document.
delete
()fancy_repr
([class_name, select_fields, …])Generates a customizable string representation of the document.
from_dict
(d[, extended])Loads the document from a BSON/JSON dictionary.
from_json
(s)Loads the document from a JSON string.
get_field
(field_name)Gets the field of the document.
has_field
(field_name)Determines whether the document has a field of the given name.
Returns an iterator over the
(name, value)
pairs of the public fields of the document.merge
(doc[, merge_lists, merge_dicts, overwrite])Merges the contents of the given document into this document.
reload
()save
()set_field
(field_name, value[, create, …])Sets the value of a field of the document.
to_dict
([extended])Serializes this document to a BSON/JSON dictionary.
to_json
([pretty_print])Serializes the document to a JSON string.
-
default_fields
= {'_dataset_id': <fiftyone.core.fields.ObjectIdField object>, '_media_type': <fiftyone.core.fields.StringField object>, '_rand': <fiftyone.core.fields.FloatField object>, 'created_at': <fiftyone.core.fields.DateTimeField object>, 'filepath': <fiftyone.core.fields.StringField object>, 'id': <fiftyone.core.fields.ObjectIdField object>, 'last_modified_at': <fiftyone.core.fields.DateTimeField object>, 'metadata': <fiftyone.core.fields.EmbeddedDocumentField object>, 'tags': <fiftyone.core.fields.ListField object>}¶
-
default_fields_ordered
= ('id', 'filepath', 'tags', 'metadata', 'created_at', 'last_modified_at', '_media_type', '_rand', '_dataset_id')¶
-
property
media_type
¶
-
clear_field
(field_name)¶ Clears the field from the document.
- Parameters
field_name – the field name
- Raises
ValueError – if the field does not exist
-
copy
()¶ Returns a deep copy of the document.
- Returns
a
SerializableDocument
-
delete
()¶
-
fancy_repr
(class_name=None, select_fields=None, exclude_fields=None, **kwargs)¶ Generates a customizable string representation of the document.
- Parameters
class_name (None) – optional class name to use
select_fields (None) – iterable of field names to restrict to
exclude_fields (None) – iterable of field names to exclude
**kwargs – additional key-value pairs to include in the string representation
- Returns
a string representation of the document
-
property
field_names
¶ An ordered tuple of the public fields of this document.
-
classmethod
from_dict
(d, extended=False)¶ Loads the document from a BSON/JSON dictionary.
- Parameters
d – a dictionary
extended (False) – whether the input dictionary may contain serialized extended JSON constructs
- Returns
a
SerializableDocument
-
classmethod
from_json
(s)¶ Loads the document from a JSON string.
- Returns
a
SerializableDocument
-
get_field
(field_name)¶ Gets the field of the document.
- Parameters
field_name – the field name
- Returns
the field value
- Raises
AttributeError – if the field does not exist
-
has_field
(field_name)¶ Determines whether the document has a field of the given name.
- Parameters
field_name – the field name
- Returns
True/False
-
property
in_db
¶
-
iter_fields
()¶ Returns an iterator over the
(name, value)
pairs of the public fields of the document.- Returns
an iterator that emits
(name, value)
tuples
-
merge
(doc, merge_lists=True, merge_dicts=True, overwrite=True)¶ Merges the contents of the given document into this document.
- Parameters
doc – a
SerializableDocument
of same type as this documentmerge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value
merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value
overwrite (True) – whether to overwrite (True) or skip (False) existing fields
-
reload
()¶
-
save
()¶
-
set_field
(field_name, value, create=True, validate=True, dynamic=False)¶ Sets the value of a field of the document.
- Parameters
field_name – the field name
value – the field value
create (True) – whether to create the field if it does not exist
- Raises
ValueError – if
field_name
is not an allowed field name or does not exist andcreate == False
-
to_dict
(extended=False)¶ Serializes this document to a BSON/JSON dictionary.
- Parameters
extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format
- Returns
a dict
-
to_json
(pretty_print=False)¶ Serializes the document to a JSON string.
- Parameters
pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations
- Returns
a JSON string
-