# Component

machinable.Component(config:dict=None, flags:dict=None, node=None)

Component base class. All machinable components must inherit from this class.

TIP

All components are Mixins by default. However, if you want to explicitly mark your component for Mixin-only use consider inheriting from the Mixin base class machinable.Mixin.

# children property

List of child components or None

# config property

Component configuration

# flags property

Component flags

# log property

Log writer instance

# node property

Node component or None

# record property

Record writer instance

# bind

bind(self, mixin, attribute)

Binds a mixin to the component

# Arguments

  • mixin: Mixin module or class
  • attribute: Attribute name, e.g. my_mixin

# create

create(self, children=None, observer=None)

Creates the component

TIP

This method is triggered automatically. However, child component creation can be suppressed in the on_create event of the node component. See on_create for details.

Triggers create events of the lifecycle and restores the component from provided checkpoint if any

# Arguments

  • children: Optional list of child component instances that are used by the component
  • observer: Optional observer instance to be used by the component

# destroy

destroy(self)

Destroys the component

Triggers destroy events

TIP

This method is triggered automatically.

# execute

execute(self)

Executes the component

Triggers execution events and writes execution meta-data

TIP

Execution is triggered automatically for node components only.

# on_after_create

on_after_create(self)

Lifecycle event triggered after component creation

# on_after_destroy

on_after_destroy(self)

Lifecycle event triggered after component destruction

# on_after_execute

on_after_execute(self)

Lifecycle event triggered after execution

# on_after_execute_iteration

on_after_execute_iteration(self, iteration:int)

Lifecycle event triggered after execution iteration

# on_after_init

on_after_init(self)

Lifecycle event triggered after component initialisation

# on_after_init_child

on_after_init_child(self, child, index:int)

Lifecycle event triggered after child component initialisation

# on_after_init_children

on_after_init_children(self, children:List)

Lifecycle event triggered after child component initialisation

# on_after_init_observer

on_after_init_observer(self, observer:machinable.observer.observer.Observer)

Lifecycle event triggered after observer initialisation

# on_before_create

on_before_create(self)

Lifecycle event triggered before component creation

# on_before_destroy

on_before_destroy(self)

Lifecycle event triggered before component destruction

# on_before_execute

on_before_execute(self)

Lifecycle event triggered before component execution

# on_before_execute_iteration

on_before_execute_iteration(self, iteration:int)

Lifecycle event triggered before an execution iteration

# on_before_init

on_before_init(self, config=None, flags=None, node=None)

Lifecycle event triggered before component initialisation

# on_create

on_create(self)

Lifecycle event triggered during component creation

The method can declare arguments to handle components explicitly. For example, the signature on_create(self, node, alias_of_child_1, alias_of_child_2=None) declares that component accepts two child components with alias alias_of_child_1 and alias_of_child_2 where the latter is declared optional. If the alias starts with an underscore _ the component lifecycle will not be triggered automatically.

The function signature may also be used to annotate expected types, for example:

on_create(self, node: DistributedExperiment, model: DistributedModel = None):
    self.experiment = node
    self.model = model

Note, however, that the type annotation will not be enforced.

# on_destroy

on_destroy(self)

Lifecycle event triggered at component destruction

# on_execute

on_execute(self)

Lifecycle event triggered at component execution

TIP

This event and all other execution events are triggered in node components only

# on_execute_iteration

on_execute_iteration(self, iteration:int)

Lifecycle event triggered at execution iteration

Allows to implement repeating iterations

The method is repeatedly called until StopIteration is returned or raised; iteration is increased on each call The iteration number is also available class-wide under self.flags.ITERATION

# Arguments

  • iteration: Integer, iteration number that automatically increases in every iteration starting from 0

# on_init

on_init(self, config=None, flags=None, node=None)

Lifecycle event triggered during component initialisation

Return False to prevent the default configuration and flag instantiation

# on_init_child

on_init_child(self, child_config:Dict, index:int)

Lifecycle event triggered during child component initialisation

Return False to prevent the child instantiation

# on_init_children

on_init_children(self, children_config:List[Dict])

Lifecycle event triggered at child component initialisation

Return False to prevent the children instantiation

# on_init_observer

on_init_observer(self, observer_config:Dict)

Lifecycle event triggered at observer initialisation

Return False to prevent the default observer instantiation

# on_restore

on_restore(self, checkpoint_file:str)

Implements the restore checkpoint functionality of the component

# Arguments

  • checkpoint_path: String, source directory
  • timestep: Integer, counting number of checkpoint

# Returns

Return False to indicate that checkpoint has not been restored.

# on_save

on_save(self, checkpoint_path:str, timestep:int) -> Union[bool, str]

Implements the checkpoint functionality of the component

# Arguments

  • checkpoint_path: String, target directory
  • timestep: Integer, counting number of checkpoint

# Returns

Returns the name of the checkpoint file or path. Return False to indicate that checkpoint has not been saved.

# on_seeding

on_seeding(self)

Lifecycle event to implement custom seeding

Return False to prevent the default seeding procedure

# restore_checkpoint

restore_checkpoint(self, filepath=None)

Restores a checkpoint

# Arguments

  • filepath: Checkpoint filepath

# save_checkpoint

save_checkpoint(self, path:str=None, timestep=None) -> Union[bool, str]

Saves component to a checkpoint

# Arguments

  • path: Path where checkpoints should be saved
  • timestep: Optional timestep of checkpoint. If None, timestep will count up automatically

# Returns

Filepath of the saved checkpoint or False if checkpoint has not been saved

# set_seed

set_seed(self, seed=None) -> bool

Applies a global random seed

# Arguments

  • seed: Integer, random seed. If None, self.flags.SEED will be used

To prevent the automatic seeding, you can overwrite the on_seeding event and return False

# Observer

machinable.observer.Observer(config=None, heartbeat=15)

Interface to observation storage

TIP

Becomes available as self.observer

# Arguments

  • config: dict, configuration options
  • heartbeat: Integer, seconds between two heartbeat events

# log property

Log interface

# Returns

machinable.observer.log.Log

# record property

Record interface

# Returns

machinable.observer.record.Record

# destroy

destroy(self)

Destroys the observer instance

# directory

directory(self)

Returns the storage directory

# get_path

get_path(self, append='', create=False)

Returns the storage path

# Arguments

  • append: String, optional postfix that is appended to the path
  • create: Boolean, if True path is being created if not existing

# get_record_writer

get_record_writer(self, scope)

Creates or returns an instance of a record writer

# Arguments

  • scope: Name of the record writer

# Returns

machinable.observer.record.Record

# get_stream

get_stream(self, path, mode='r', *args, **kwargs)

Returns a file stream on the observer storage

# Arguments

  • path: Relative file path
  • mode: Python file mode *args, **kwargs: Optional arguments passed into stream open()

# has_logs

has_logs(self)

Determines whether log writer exists

# Returns

True if log writer exists

# has_records

has_records(self, scope='default')

Determines whether record writer exists

# Arguments

  • scope: String, name of the record writer. Defaults to 'default'

# Returns

True if records have been written

# local_directory

local_directory(self)

Returns the local filesystem path, or False if non-local

# Returns

Local filesystem path, or False if non-local

# refresh_meta_data

refresh_meta_data(self, node=None, children=None, flags=None)

Updates the observation's meta data

# Arguments

  • node: Node config
  • children: Children config
  • flags: Flags

# refresh_status

refresh_status(self)

Updates the status.json file with a heartbeat at the current time

# store

store(self, name, data, overwrite=False, _meta=False)

Stores a data object

# Arguments

  • name: String, name identifier. You can provide an extension to instruct machinable to store the data in its own file and not as part of a dictionary with other stored values. Supported formats are .json (JSON), .npy (numpy), .p (pickle), .txt (txt)
  • data: The data object
  • overwrite: Boolean, if True existing values will be overwritten

# Record

machinable.observer.record.Record(observer, config=None, scope='default')

Tabular record writer

TIP

Becomes available as self.observer.record and self.record

# Arguments

  • observer: machinable.observer.Observer, parent observer instance
  • config: dict, configuration options
  • scope: String, optional destination name

# empty

empty(self)

Whether the record writer is empty (len(self.record) == 0)

# save

save(self, echo=False, force=False)

Save the current row

# Arguments

  • echo: Boolean, if True the written row will be printed to the terminal
  • force: Boolean, if True the row will be written even if it contains no columns

# Returns

The row data

# timing

timing(self, mode='iteration', timestamp=None, return_type='seconds')

Get timing statistics about the records

# Arguments

  • mode: String, 'iteration', 'avg' or 'total' that determines whether the last iteration, the average iteration or the total time is collected
  • timestamp: Mixed, end of the timespan; if None, datetime.now() is used
  • return_type: String, specifying the return format 'seconds', 'words', or 'period' (pendulum.Period object)

Returns: See return_type

# update

update(self, dict_like=None, **kwargs)

Update record values using a dictionary. Equivalent to dict's update method.

# Arguments

  • dict_like: dict, update values

# write

write(self, key, value, fmt=None)

Writes a cell value

Note that you can use array notation to write value as well, for example

self.record.write('loss', 0.1)
#### is equivalent to
self.record['loss'] = 0.1

# Arguments

  • key: String, the column name
  • value: Value to write
  • fmt: String, optional formatter.

# Log

machinable.observer.log.Log(observer, config=None)

Logs to the console and the storage

TIP

Becomes available as self.observer.log and self.log

Exposes the standard interface of Python loggers, e.g. info, debug etc.

# Arguments

  • observer: machinable.observer.Observer, parent observer instance
  • config: dict, configuration options