# Components

Components are the core interface to implement functionality in a machinable project. Technically, they are simply classes that inherit from the base class machinable.Component as defined in the python module that is specified in the machinable.yaml. For instance, to implement a components that encapsulates some optimization problem, we could create the following source file:

# optimization.py
from machinable import Component

class DummyOptimization(Component):
    def on_create(self):
            "Creating the optimization model with the following configuration: ",

    def on_execute(self):
        for i in range(3):
            print("Training step", i)

Note that it does not matter how you name the class as long as the class inherits from the components base class and is registered in the machinable.yaml, for instance:

 - optimization: 
     learning_rate: 0.1

The components base provides a variety of interfaces that bootstrap the implementation of the components and are described below.

# Life cycle

Components expose a number of life cycle events that can be overwritten to hook into the execution cycle at a certain point. All event methods start with on_ and are documented in the event reference. In the example above, the on_create and on_execute events are implemented and will thus be triggered during execution.

The components life cycle allows you to implement using any framework and standard python methods without worrying about the execution logic (i.e. configuration parsing, parallel execution, etc.). Moreover, the event paradigm provides a clear semantic while the object orientation enables flexible code sharing mechanisms (e.g. inheritance, mixins, etc.).

# self.config

Components can consume their configuration via the self.config object:

from machinable import Component

class MyComponent(Component):
    def on_create(self):

>>> 1

For convenience, the dict interface can be accessed using the . object notation and provides a few helper methods like pretty-printing pprint etc.

# self.flags

Flags are configuration values that are associated with the particular execution, for example the random seeds or worker IDs. They are accessible via the self.flags object, that supports the . object notation. You can add your own flags through basic assignment, e.g. self.flags.counter = 1. To avoid name collision, all native machinable flags use UPPERCASE (e.g. self.flags.SEED).

# self.store

The interface self.store allows for the storing of data and results of the components. Note that you don't have to specify where the data is being stored. machinable will manage unique directories automatically. The data can later be retrieved using the Storage interface.


self.store.log or self.log provides a standard logger interface that outputs to the console and a log file.

self.log.info('Component created')
self.log.debug('Component initialized')


self.store.record or self.record provides an interface for tabular logging, that is, storing recurring data points at each iteration. The results become available as a table where each row represents each iteration.

for iteration in range(10):
    self.record['iteration'] = iteration
    loss, acc = ...
    # write column values
    self.record['accuracy'] = acc
    self.record['loss'] = loss

    # save at the end of the iteration to start a new row

If you use the on_execute_iteration event, iteration information and record.save() will be triggered automatically at the end of each iteration.

Sometimes it is useful to have multiple tabular loggers, for example to record training and validation performance separately. You can create custom record loggers using self.store.get_record_writer(scope) which returns a new instance of a record writer that you can use just like the main record writer.


You can use self.store.write() to write any other Python object, for example:

self.store.write('final_accuracy', [0.85, 0.92])

Note that to protect unintended data loss, overwriting will fail unless the overwrite argument is explicitly set.

For larger data structures, it can be more suitable to write data in specific file formats by appending a file extension, i.e.:

self.store.write('data.txt', 'a string')
self.store.write('data.p', generic_object)
self.store.write('data.json', jsonable_object)
self.store.write('data.npy', numpy_array)

Refer to the store reference for more details.

# Config methods

While config references allow you to make static references, configuration values can be more complex. They might, for example, evolve during the course of execution or obey non-trivial conditions. Config methods allow you to implement such complex configuration values. To define a config method just add a regular Python method to the components class. The method name must start with config_. You can then 'call' the method directly in the machinable.yaml configuration, for example:

  - my_network:
      batch_size: 32
      learning_rate: base_learning_rate(2**-5)

Here, the learning rate parameter is defined as a config method that takes a base learning rate parameter. The config method config_base_learning_rate needs to be defined in the corresponding component:

from machinable import Component

class MyBaseModel(Component):
    def on_create(self):
        print('Training with lr=', self.config.learning_rate)

    def config_base_learning_rate(self, lr):
        return lr * self.config.batch_size

The method is executed whenever self.config.learning_rate is being accessed; as a result, the execution output prints:

>>> 'Training with lr=1'

Config methods hence allow for the expression of arbitrary configuration dependencies and are a powerful tool for implementing complex configuration patterns more efficiently. They can also be useful for parsing configuration values into Python objects. For instance, you might define a config method dtype where dtype: dtype('f32') returns np.float32 etc.

# Sub-components

In many cases, it can be useful to organise components in a hierarchical way. For example, your components may implement a certain prediction problem and you want to encapsulate different prediction strategies in sub-components.

machinable allows you to use components as sub-components, meaning they become available to a parent node components. Consider the following components:

from machinable import Component

# sub_component_example.py

class PredictionStrategy(Component):
    def on_create(self):
        self.model = ... # set up some model
    def predict(self, data):
        return self.model.predict(data)

# node_component_example.py

from machinable import Component

class PredictionBenchmark(Component):

    def on_create(self, prediction_strategy):
        self.prediction_strategy = prediction_strategy

        # load data
        self.data = ...


Here, the sub-component encapsulates the model while the node components implements the benchmark control flow. The sub-component becomes available as argument to the on_create event of the node components.

In general, the sub-components can access the parent node via self.node while the node components can access its sub-components via self.components.

To designate components as sub-components use the components argument of Experiment.components() that will be discussed in the following section.