# Experiment

# execute

machinable.execute(experiment:Union[machinable.experiment.experiment.Experiment, Any], storage:Union[dict, str]=None, engine:Union[machinable.engines.engine.Engine, str, dict, NoneType]=None, project:Union[machinable.project.project.Project, Callable, str, dict]=None, seed:Union[int, NoneType, str]=None) -> machinable.execution.execution.Execution

Executes a machinable experiment

Schedules the experiment for execution.

# Arguments

experiment: machinable.Experiment, specifies the execution experiment. For convenience, it can also be a string or tuple that defines the node argument, e.g. ml.execute('my_model') or ml.execute((('my_model', {'param': 1}),))
storage: String, URL of the write location for the results. If unspecified, the result write will be non-persistent. Remote locations like SSH or S3 are supported via pyFilesystem URLs
engine: machinable.Engine|Dict|String|None, engine that handles execution, e.g. 'local' or 'ray' etc.
project: Project|Dict|String|None, project used, defaults to current working directory
seed: Integer|String|None, determines the global random seed. If None, a random seed will be generated. To re-use the same random seed of a previous execution, you can pass in its experiment ID

# Example

import machinable as ml
ml.execute(ml.Experiment().components('iris', 'random_forest'), 's3://bucket', seed=42)

# Functional API

You can use this method as a function decorator to implement a custom execution. The decorated function is invoked with the configuration as well as an store object, for example:

@ml.execute
def custom_execute(component, components, store):
    store.log.info('Custom training with learning_rate=' + str(component.config.lr))

custom_execute(experiment, storage, seed) # invokes the decorated function

# Using the Execution

This method forms a wrapper around machinable.Execution. You can instantiate machinable.Executon directly with the same argument to benefit from more fine grained execution APIs like asynchronous executon etc.

# Returns

The execution returns an machinable.Execution object that contains the result of the execution

# Experiment

machinable.Experiment()

Defines an execution schedule for available components. The experiment interface is fluent, methods can be chained in arbitrary order.

# Arguments

components: Optional String or tuple components definition to add to the experiment

# Example

import machinable as ml
linear_regression = ml.Experiment().components('mnist').repeat(3)
#### or using the shorthand where components arguments are passed into the constructor
linear_regression = ml.Experiment('mnist').repeat(3)

# component

component(self, name, version=None, checkpoint=None, flags=None, resources=None)

Adds a component to the experiment

# Arguments

name: String, the name of the components as defined in the machinable.yaml
version: dict|String, a configuration update to override its default config
checkpoint: String, optional URL to a checkpoint file from which the components will be restored
flags: dict, optional flags to be passed to the component
resources: dict, specifies the resources that are available to the component. This can be computed by passing in a callable (see below)

# Examples

import machinable as ml
experiment = ml.Experiment().component(name='models.linear_regression', version={'alpha': 0.1})

# Dynamic resource computation

You can condition the resource specification on the configuration, for example:

resources = lambda(config): {'gpu': config.num_gpus }

The arguments of the callable are passed in only if requested by the signature. Options are: node - The full component specification config - alias for node.args flags - alias for node.flags components - List of sub-component specifications

# components

components(self, node, components=None, resources=None)

Adds a component with sub-components to the experiment

# Arguments

node: machinable.ExperimentComponent specifying the node components
components: optional list of machinable.Component, supplementary components for this node
resources: dict, specifies the resources that are available to the component

# Examples

import machinable as ml
experiment = ml.Experiment().components(
    node=('mnist', version='~shuffled'),                #### main component
    components=['resnet', ('resnext', {'lr': 3e-4})]    #### sub-components
    resources={'num_gpus': 2}
)

# copy

copy(self)

Returns a copy of the current experiment object

# directory

directory(self, path:str='&PROJECT/&MODULE')

Set the directory of the experiment

Relative path that gets appended to the storage directory of this experiment

# Arguments

path: String, defines the directory name as string which may contain the following variables:
- &MODULE will be replaced by the experiment module name (empty if it can not be determined)
- &PROJECT will be replaced by project name (empty if it can not be determined)
- %x expressions will be replaced by strftime

# repeat

repeat(self, k, name='REPEAT', mode='independent')

Repeats the experiment k times

Schedules the current experiment multiple times and injects the flags REPEAT_NUMBER and REPEAT_SEED to the node instances.

# Arguments

k: Integer, the number of repetitions
name: String, flag prefix, e.g. '{name}_SEED' etc. Defaults to REPEAT

# Examples

import machinable as ml
#### five independent runs of the same node
ml.Experiment().components('regression').repeat(5)

# split

split(self, k, name='SPLIT', mode='independent')

Splits the experiment k times

Schedules the current experiment k times and injects appropriate flags SPLIT_NUMBER and SPLIT_SEED to the node instances. Note that machinable does not split the nodes automatically. The user has to implement the algorithmic splitting based on the flag information. For example, to implement a cross-validation algorithm the node should split the data using the split seed in the flag SPLIT_SEED and use the split that is specified in the flag SPLIT_NUMBER.

# Arguments

k: Integer, the number of splits
name: String, flag prefix, e.g. '{name}_SEED' etc. Defaults to SPLIT

# Examples

import machinable as ml
#### five independent runs of the same node
ml.Experiment().components('regression').repeat(5)

# tune

tune(self, *args, **kwargs)

Schedules for hyperparameter tuning

Components need to implement on_execute_iteration event that becomes tune training step. The record writer can be used as usual. In particular, record fields may be used in stop conditions.

WARNING

Tuning integration is experimental. Please report any issues that you encounter.

# Arguments

The arguments differ based on the used engine.

Ray engine: Uses Ray tune (Argument reference)

# version

version(self, config=None, **kwargs)

Applies a configuration update

Selected components use the default configuration as specified in the machinable.yaml. Versioning allows for configuration overrides using dictionaries or defined version patterns. The versions here are applied globally to all components in this experiment. If you want to apply the version locally, use the version parameter in the local method.

# Arguments

config: Configuration update represented as dictionary

# Examples

import machinable as ml
experiment = ml.Experiment().components('evolution').components('sgd')
#### use a dictionary to override configuration values
experiment.version({'data': {'shuffle': True}, 'components': {'lr': 0.01}})
#### or kwargs
experiment.version(data={'shuffle': True}, components={'lr': 0.01})
#### use a specified version of the machinable.yaml
experiment.version(data='~shuffled')
#### use a mixin configuration
experiment.version(data='_mnist_')

← Component Storage →