# Experiment

# Experiment

machinable.Experiment()

Defines an execution schedule for available components. The experiment interface is fluent, methods can be chained in arbitrary order.

# Arguments

  • components: Optional String or tuple components definition to add to the experiment

# Example

import machinable as ml
linear_regression = ml.Experiment().components('mnist').repeat(3)
#### or using the shorthand where components arguments are passed into the constructor
linear_regression = ml.Experiment('mnist').repeat(3)

# component

component(self, name, version=None, checkpoint=None, flags=None, resources=None)

Adds a component to the experiment

# Arguments

  • name: String, the name of the components as defined in the machinable.yaml
  • version: dict|String, a configuration update to override its default config
  • checkpoint: String, optional URL to a checkpoint file from which the components will be restored
  • flags: dict, optional flags to be passed to the component
  • resources: dict, specifies the resources that are available to the component. This can be computed by passing in a callable (see below)

# Examples

import machinable as ml
experiment = ml.Experiment().component(name='models.linear_regression', version={'alpha': 0.1})

# Dynamic resource computation

You can condition the resource specification on the configuration, for example:

resources = lambda(engine, component, components): {'gpu': component.config.num_gpus }

The arguments of the callable are: engine - The engine instance component - The full component specification (config, flags, module) components - List of sub-component specifications

# components

components(self, node, components=None, resources=None)

Adds a component with sub-components to the experiment

# Arguments

  • node: machinable.ExperimentComponent specifying the node components
  • components: optional list of machinable.Component, supplementary components for this node
  • resources: dict, specifies the resources that are available to the component

# Examples

import machinable as ml
experiment = ml.Experiment().components(
    node=('mnist', version='~shuffled'),                #### main component
    components=['resnet', ('resnext', {'lr': 3e-4})]    #### sub-components
    resources={'num_gpus': 2}
)

# copy

copy(self)

Returns a copy of the current experiment object

# name

name(self, name:str)

Sets an experiment name

# Arguments

  • name: String, experiment name. Must be a valid Python variable name or path e.g. my_name or example.name etc.

# repeat

repeat(self, k, name='REPEAT', mode='independent')

Repeats the experiment k times

Schedules the current experiment multiple times and injects the flags REPEAT_NUMBER and REPEAT_SEED to the node instances.

# Arguments

  • k: Integer, the number of repetitions
  • name: String, flag prefix, e.g. '{name}_SEED' etc. Defaults to REPEAT

# Examples

import machinable as ml
#### five independent runs of the same node
ml.Experiment().components('regression').repeat(5)

# split

split(self, k, name='SPLIT', mode='independent')

Splits the experiment k times

Schedules the current experiment k times and injects appropriate flags SPLIT_NUMBER and SPLIT_SEED to the node instances. Note that machinable does not split the nodes automatically. The user has to implement the algorithmic splitting based on the flag information. For example, to implement a cross-validation algorithm the node should split the data using the split seed in the flag SPLIT_SEED and use the split that is specified in the flag SPLIT_NUMBER.

# Arguments

  • k: Integer, the number of splits
  • name: String, flag prefix, e.g. '{name}_SEED' etc. Defaults to SPLIT

# Examples

import machinable as ml
#### five independent runs of the same node
ml.Experiment().components('regression').repeat(5)

# tune

tune(self, *args, **kwargs)

Schedules for hyperparameter tuning

Components need to implement on_execute_iteration event that becomes tune training step. The record writer can be used as usual. In particular, record fields may be used in stop conditions.

WARNING

Tuning integration is experimental. Please report any issues that you encounter.

# Arguments

The arguments differ based on the used engine.

# version

version(self, config=None)

Applies a configuration update

Selected components use the default configuration as specified in the machinable.yaml. Versioning allows for configuration overrides using dictionaries or defined version patterns. The versions here are applied globally to all components in this experiment. If you want to apply the version locally, use the version parameter in the local method.

# Arguments

  • config: Configuration update dictionary

# Examples

from machinable import Experiment
experiment = Experiment().components('evolution').components('sgd')
#### use a dictionary to override configuration values
experiment.version({'data': {'shuffle': True}, 'lr': 0.01})