# Component

machinable.Component(config:dict=None, flags:dict=None, node=None)

Component base class. All machinable components must inherit from this class.

TIP

All components are Mixins by default. However, if you want to explicitly mark your components for Mixin-only use consider inheriting from the Mixin base class machinable.Mixin.

# components property

List of sub components or None

# config property

Component configuration

# flags property

Component flags

# log property

Log writer instance

# node property

Node component or None

# record property

Record writer instance

# bind

bind(self, mixin, attribute)

Binds a mixin to the component

# Arguments

  • mixin: Mixin module or class
  • attribute: Attribute name, e.g. my_mixin

# create

create(self, components=None)

Creates the components

TIP

This method is triggered automatically. However, sub-component creation can be suppressed in the on_create event of the node component. See on_create for details.

Triggers create events of the lifecycle

# Arguments

  • components: Optional list of sub components instances that are used by the components

# destroy

destroy(self)

Destroys the components

Triggers destroy events

TIP

This method is triggered automatically.

# execute

execute(self)

Executes the components

Triggers execution events and writes execution meta-data

TIP

Execution is triggered automatically for node components only.

# get_url

get_url(self, append='')

Returns the storage URL of the component

# local_directory

local_directory(self, append='')

Returns the local storage filesystem path, or False if non-local

# Returns

Local filesystem path, or False if non-local

# on_after_create

on_after_create(self)

Lifecycle event triggered after components creation

# on_after_destroy

on_after_destroy(self)

Lifecycle event triggered after components destruction

# on_after_execute

on_after_execute(self)

Lifecycle event triggered after execution

# on_after_execute_iteration

on_after_execute_iteration(self, iteration:int)

Lifecycle event triggered after execution iteration

# on_after_init

on_after_init(self)

Lifecycle event triggered after components initialisation

# on_after_init_components

on_after_init_components(self, components, index:Union[int, NoneType]=None)

Lifecycle event triggered after components initialisation

# on_before_create

on_before_create(self)

Lifecycle event triggered before components creation

# on_before_destroy

on_before_destroy(self)

Lifecycle event triggered before components destruction

# on_before_execute

on_before_execute(self)

Lifecycle event triggered before components execution

# on_before_execute_iteration

on_before_execute_iteration(self, iteration:int)

Lifecycle event triggered before an execution iteration

# on_before_init

on_before_init(self, config=None, flags=None, node=None)

Lifecycle event triggered before components initialisation

# on_create

on_create(self)

Lifecycle event triggered during components creation

The method can declare arguments to handle components explicitly. For example, the signature on_create(self, node, alias_of_child_1, alias_of_child_2=None) declares that components accepts two sub components with alias alias_of_child_1 and alias_of_child_2 where the latter is declared optional. If the alias starts with an underscore _ the components lifecycle will not be triggered automatically.

The function signature may also be used to annotate expected types, for example:

on_create(self, node: DistributedExperiment, model: DistributedModel = None):
    self.experiment = node
    self.model = model

Note, however, that the type annotation will not be enforced.

# on_destroy

on_destroy(self)

Lifecycle event triggered at components destruction

# on_execute

on_execute(self)

Lifecycle event triggered at components execution

TIP

This event and all other execution events are triggered in node components only

# on_execute_iteration

on_execute_iteration(self, iteration:int)

Lifecycle event triggered at execution iteration

Allows to implement repeating iterations

The method is repeatedly called until StopIteration is returned or raised; iteration is increased on each call The iteration number is also available class-wide under self.flags.ITERATION

# Arguments

  • iteration: Integer, iteration number that automatically increases in every iteration starting from 0

# on_init

on_init(self, config=None, flags=None, node=None)

Lifecycle event triggered during components initialisation

Return False to prevent the default configuration and flag instantiation

# on_init_components

on_init_components(self, components_config, index:Union[int, NoneType]=None)

Lifecycle event triggered during components initialisation

Return False to prevent the components instantiation

# on_init_storage

on_init_storage(self, storage_config:Dict)

Lifecycle event triggered at write initialisation

# on_restore

on_restore(self, checkpoint)

Implements the restore checkpoint functionality of the components

# Arguments

  • checkpoint: Checkpoint specification
  • timestep: Integer, counting number of checkpoint

# Returns

Return False to indicate that checkpoint has not been restored.

# on_save

on_save(self, checkpoint_path:str, timestep:int) -> Union[bool, str]

Implements the checkpoint functionality of the components

# Arguments

  • checkpoint_path: String, target directory
  • timestep: Integer, counting number of checkpoint

# Returns

Returns the name of the checkpoint file or path. Return False to indicate that checkpoint has not been saved.

# on_seeding

on_seeding(self)

Lifecycle event to implement custom seeding

Return False to prevent the default seeding procedure

# refresh_status

refresh_status(self, log_errors=False)

Updates the status.json file with a heartbeat at the current time

# restore_checkpoint

restore_checkpoint(self, checkpoint)

Restores a checkpoint

# Arguments

  • filepath: Checkpoint filepath

# save_checkpoint

save_checkpoint(self, path:str=None, timestep=None) -> Union[bool, str]

Saves components to a checkpoint

# Arguments

  • path: Path where checkpoints should be saved
  • timestep: Optional timestep of checkpoint. If None, timestep will count up automatically

# Returns

Filepath of the saved checkpoint or False if checkpoint has not been saved

# set_seed

set_seed(self, seed=None) -> bool

Applies a global random seed

# Arguments

  • seed: Integer, random seed. If None, self.flags.SEED will be used

To prevent the automatic seeding, you can overwrite the on_seeding event and return False

# Store

machinable.store.Store(component, config)

Store interface

TIP

Becomes available as self.store

# Arguments

  • component: Component instance
  • config: dict, configuration options

# log property

Log interface

# Returns

machinable.store.log.Log

# record property

Record interface

# Returns

machinable.store.record.Record

# record_writers property

Returns a mapping of all record writers (scope => RecordWriter)

# get_path

get_path(self, append='', create=False)

Returns the write path

# Arguments

  • append: String, optional postfix that is appended to the path
  • create: Boolean, if True path is being created if not existing

# get_record_writer

get_record_writer(self, scope)

Creates or returns an instance of a record writer

# Arguments

  • scope: Name of the record writer

# Returns

machinable.store.record.Record

# get_stream

get_stream(self, path, mode='r', *args, **kwargs)

Returns a file stream on the store write

# Arguments

  • path: Relative file path
  • mode: Python file mode *args, **kwargs: Optional arguments passed into stream open()

# has_logs

has_logs(self)

Determines whether log writer exists

# Returns

True if log writer exists

# has_records

has_records(self, scope='default')

Determines whether record writer exists

# Arguments

  • scope: String, name of the record writer. Defaults to 'default'

# Returns

True if records have been written

# write

write(self, name, data, overwrite=False, _meta=False)

Stores a data object

# Arguments

  • name: String, name identifier. You can provide an extension to instruct machinable to write the data in its own file and not as part of a dictionary with other stored values. Supported formats are .json (JSON), .npy (numpy), .p (pickle), .txt (txt)
  • data: The data object
  • overwrite: Boolean, if True existing values will be overwritten

# Record

machinable.store.record.Record(store, config=None, scope='default')

Tabular record writer

TIP

Becomes available as self.store.record and self.record

# Arguments

  • store: machinable.store.Observer, parent store instance
  • config: dict, configuration options
  • scope: String, optional destination name

# empty

empty(self)

Whether the record writer is empty (len(self.record) == 0)

# save

save(self, echo=False, force=False)

Save the current row

# Arguments

  • echo: Boolean, if True the written row will be printed to the terminal
  • force: Boolean, if True the row will be written even if it contains no columns

# Returns

The row data

# timing

timing(self, mode='iteration', timestamp=None, return_type='seconds')

Get timing statistics about the records

# Arguments

  • mode: String, 'iteration', 'avg' or 'total' that determines whether the last iteration, the average iteration or the total time is collected
  • timestamp: Mixed, end of the timespan; if None, datetime.now() is used
  • return_type: String, specifying the return format 'seconds', 'words', or 'period' (pendulum.Period object)

Returns: See return_type

# update

update(self, dict_like=None, **kwargs)

Update record values using a dictionary. Equivalent to dict's update method.

# Arguments

  • dict_like: dict, update values

# write

write(self, key, value, fmt=None)

Writes a cell value

Note that you can use array notation to write value as well, for example

self.record.write('loss', 0.1)
#### is equivalent to
self.record['loss'] = 0.1

# Arguments

  • key: String, the column name
  • value: Value to write
  • fmt: String, optional formatter.

# Log

machinable.store.log.Log(store, config=None)

Logs to the console and the write

TIP

Becomes available as self.store.log and self.log

Exposes the standard interface of Python loggers, e.g. info, debug etc.

# Arguments

  • store: machinable.store.Observer, parent store instance
  • config: dict, configuration options