dal.data package

Subpackages

dal.data.shared package

Submodules

dal.data.archive module

Developers: - Alexandre Pires (alexandre.pires@mov.ai) - 2020

class dal.data.archive.RemoteArchive(remote_uri: str, user: str, password: str)

Bases: object

A helper class that encapsulates the async code

backup(objs: object, dst: str): Backup scopes and dependencies objs - a list of the objects to backup

get_scope(path: str): Get a scope

list_scopes(workspace_id: str): Get a list of all scopes in a workspace

list_workspaces(): Get a list of all workspaces

restore(source: str): Restore scopes source - the restore file

class dal.data.archive.RemoteArchiveAsync(remote_uri: URL, user: str, password: str)

Bases: object

This class implements a remote archive client It’s used for the command line tool and also for the filesystem to fetch missing data

async backup(objs: object, destination: str): Download a scope

async get_scope(path: str): List available workspaces

async list_scopes(workspace_id: str): List scopes in a workspace

async list_workspaces(): List available workspaces

async restore(source: str): Upload a restore file to the server

dal.data.mixins module

class dal.data.mixins.ChildrenCmpMixin

Bases: object

Implement _children comparison functions

class dal.data.mixins.ValueCmpMixin

Bases: object

Implement _value comparison functions

dal.data.schema module

Developers: - Alexandre Pires (alexandre.pires@mov.ai) - 2020

class dal.data.schema.SchemaDeserializer(version: str = '1.0')

Bases: ObjectDeserializer, PluginManager

Deserializer through a dict and convert to a tree

deserialize(root: TreeNode, data: dict): Abstract method to run the data deserializer

classmethod plugin_class(): Get current class plugin

property version: Get the version of this schema deserializer

class dal.data.schema.SchemaNode(schema: str)

Bases: DictNode

Implements a schema node

property node_type: the node type

property path: get the tree path

property schema: the schema name

validate(node: PropertyNode): Validate a property Node against this schema

property version: Return the current version

class dal.data.schema.SchemaObjectNode(name: str)

Bases: ObjectNode

Implements a schema object node

validate(node: PropertyNode): Validate a property Node against this schema

class dal.data.schema.SchemaPropertyNode(name: str, value_type: type)

Bases: PropertyNode

Implements a property node

DEFAULT_MAPPING = {<class 'str'>: '', <class 'int'>: 0, <class 'float'>: 0.0, <class 'dict'>: {}, <class 'bool'>: False, <class 'object'>: None}

validate(node: PropertyNode): Validate a property Node against this schema

property value: return the current value

class dal.data.schema.SchemaVersionNode(version)

Bases: VersionNode

A version node of schemas

class dal.data.schema.SchemasTree

Bases: CallableNode

Implements a Schema Tree

property node_type: the node type

property path: get the tree path

dal.data.serialization module

Developers: - Alexandre Pires (alexandre.pires@mov.ai) - 2020

class dal.data.serialization.ObjectDeserializer

Bases: ABC

A base data deserializer

abstract deserialize(root: TreeNode, data: dict): Abstract method to run the data deserializer

class dal.data.serialization.ObjectSerializer

Bases: ABC

A base data serializer

abstract serialize(root: TreeNode): Abstract method to run the data deserializer

class dal.data.serialization.SerializableObject

Bases: ABC

A serializable object should implement this inteface

abstract serialize(**kwargs): for internal use only, this method is not supposed to be called by the user

class dal.data.serialization.SimpleDeserializer

Bases: ObjectDeserializer

Deserializer through a dict and convert to a tree

deserialize(root: TreeNode, data: dict): Abstract method to run the data deserializer

class dal.data.serialization.SimpleSerializer

Bases: ObjectSerializer

Serializer through a tree and convert to a dict

serialize(root: TreeNode): Abstract method to run the data serializer

dal.data.tree module

Developers: - Alexandre Pires (alexandre.pires@mov.ai) - 2020

class dal.data.tree.CallableNode

Bases: DictNode

Implements an attribute tree node

property children: children list

property count: number of children

sort(): sort this tree node

class dal.data.tree.DictNode

Bases: TreeNode[VT], ChildrenCmpMixin

Implements a dict tree node

add_child(node: Tuple[str, VT] | VT): add a ne child to this node

property children: children list

contains(key: str): check if dict contains key

property count: number of children

get(key: str, default: VT | None = None): return the value of the element with key

items(): access to this dict item

keys(): access to this dict keys

remove_child(node): remove child to this node

sort(): sort this tree node

values(): acess to this dict values

class dal.data.tree.ListNode

Bases: TreeNode[VT], ChildrenCmpMixin

Implements a listed tree node

add_child(node: VT): add child to this node

property children: children list

property count: number of children

remove_child(node: VT): remove child to this node

class dal.data.tree.ObjectNode(name: str)

Bases: TreeNode[VT], ChildrenCmpMixin

Implements a object node

add_child(node: Tuple[str, VT] | VT): add a child node to the node

property children: children list

property count: number of children

property name: the object name

property node_type: the node type

property path: get the tree path

remove_child(node: str | VT): remove child in the node

sort(): sort this tree node

class dal.data.tree.PropertyNode(name: str, value: str)

Bases: TreeNode, ValueCmpMixin

Implements a property node

property children: children list

property count: number of children

property name: the property name

property node_type: the node type

property path: get the tree path

property value: return the current value

property value_type: return the type of the current value

class dal.data.tree.TreeNode

Bases: ABC, Generic[VT]

Implements an abstract tree node

property attributes

Schema custom attributes, this is a dict, and should be used to store important information in the beahviour when processing this node.

One use case is on the schema tree there is an attribute value_on_key that informs the persistent layer that the value should also be present when creating the key for storing in redis.

Another use case is the “is_hash”, which is used for telling what schema objects are DictNodes.

static cached_attribute(method): A decorator to set use the self.attributes as a cache, when using this decorator the property will be cached on the attributes of this node

abstract property children: List[VT]: children list

abstract property count: int: number of children

property depth: int: return the depth of this node

from_path(path): get the tree node from a path

get_first_parent(node_type: str): get the first parent of type node_type:

property is_root: is_root_node

abstract property node_type: the node type

property parent: the object reference

abstract property path: get the tree path

sort(): sort this tree node

property sorted: children list

to_path(): get a path representation of this Node

dal.data.version module

Developers: - Alexandre Pires (alexandre.pires@mov.ai) - 2020

class dal.data.version.VersionNode(version)

Bases: CallableNode, VersionObject

Implements a Schema Versions Node

property node_type: the node type

property path: get the tree path

property version: The current version

class dal.data.version.VersionObject

Bases: ABC

A versionable object, it handles different versions of the same object

abstract property version: the current version

dal.data.workspace module

Developers: - Alexandre Pires (alexandre.pires@mov.ai) - 2020

class dal.data.workspace.WorkspaceManager

Bases: object

A static class that holds all methods to operate with workspaces

static create_workspace(workspace: str, **kwargs): create a new workspace

static delete_workspace(workspace: str): create a new workspace

static list_workspaces(): get a list of available workspaces

static workspace_info(workspace: str): get information about a workspace

class dal.data.workspace.WorkspaceNode(workspace: str, plugin, readonly: bool = False)

Bases: DictNode, WorkspaceObject

Implements a workspace tree node, to define a workspace we must provide a name, and a underlying plugin

backup(**kwargs): backup object from this workspace to a zip file

create_version(version_tag, **kwargs)

delete(data: object | None = None, **kwargs): delete object from this workspace

property node_type: the node type

property path: get the tree path

property plugin: Get the current plugin for this workspace

prev_version(**kwargs)

pull(**kwargs)

push(**kwargs)

read(**kwargs): read object from this workspace

property readonly: The current version

reload(**kwargs): Reload the cached data on this workspace

restore(**kwargs): restore a object from a zip file to this workspace

unload(**kwargs): Unload the cached data on this workspace

property workspace: The current version

write(data: object, **kwargs): write object to this workspace

class dal.data.workspace.WorkspaceObject

Bases: ABC

This class represents a interface for a workspace object A workspace object is a object that belongs to a specific Workspacw

abstract property workspace: the object workspace

Module contents

Developers: - Alexandre Pires (alexandre.pires@mov.ai) - 2020

class dal.data.ObjectDeserializer

Bases: ABC

A base data deserializer

abstract deserialize(root: TreeNode, data: dict): Abstract method to run the data deserializer

class dal.data.ObjectSerializer

Bases: ABC

A base data serializer

abstract serialize(root: TreeNode): Abstract method to run the data deserializer

class dal.data.SchemaNode(schema: str)

Bases: DictNode

Implements a schema node

property node_type: the node type

property path: get the tree path

property schema: the schema name

validate(node: PropertyNode): Validate a property Node against this schema

property version: Return the current version

class dal.data.SchemaObjectNode(name: str)

Bases: ObjectNode

Implements a schema object node

validate(node: PropertyNode): Validate a property Node against this schema

class dal.data.SchemaPropertyNode(name: str, value_type: type)

Bases: PropertyNode

Implements a property node

DEFAULT_MAPPING = {<class 'str'>: '', <class 'int'>: 0, <class 'float'>: 0.0, <class 'dict'>: {}, <class 'bool'>: False, <class 'object'>: None}

validate(node: PropertyNode): Validate a property Node against this schema

property value: return the current value

class dal.data.SchemasTree

Bases: CallableNode

Implements a Schema Tree

property node_type: the node type

property path: get the tree path

class dal.data.TreeNode

Bases: ABC, Generic[VT]

Implements an abstract tree node

property attributes

Schema custom attributes, this is a dict, and should be used to store important information in the beahviour when processing this node.

One use case is on the schema tree there is an attribute value_on_key that informs the persistent layer that the value should also be present when creating the key for storing in redis.

Another use case is the “is_hash”, which is used for telling what schema objects are DictNodes.

static cached_attribute(method): A decorator to set use the self.attributes as a cache, when using this decorator the property will be cached on the attributes of this node

abstract property children: List[VT]: children list

abstract property count: int: number of children

property depth: int: return the depth of this node

from_path(path): get the tree node from a path

get_first_parent(node_type: str): get the first parent of type node_type:

property is_root: is_root_node

abstract property node_type: the node type

property parent: the object reference

abstract property path: get the tree path

sort(): sort this tree node

property sorted: children list

to_path(): get a path representation of this Node

class dal.data.VersionObject

Bases: ABC

A versionable object, it handles different versions of the same object

abstract property version: the current version

class dal.data.WorkspaceManager

Bases: object

A static class that holds all methods to operate with workspaces

static create_workspace(workspace: str, **kwargs): create a new workspace

static delete_workspace(workspace: str): create a new workspace

static list_workspaces(): get a list of available workspaces

static workspace_info(workspace: str): get information about a workspace

class dal.data.WorkspaceObject

Bases: ABC

This class represents a interface for a workspace object A workspace object is a object that belongs to a specific Workspacw

abstract property workspace: the object workspace