Running experiments #

Executing experiments #

Once implemented and configured, experiments can be executed by calling launch():

>>> from machinable import get
>>> experiment = get('estimate_gravity')
>>> experiment.launch()
Assuming height of 52 and time of 0.3
The gravity on the exoplanet is:  1155.5555555555557

If the execution is successful, the experiment is marked as finished.

>>> experiment.is_finished()
True

By design, experiment instances can only be executed once. They are automatically assigned a unique experiment ID, a random seed, as well as a timestamp for easy identification.

>>> gravity = get('estimate_gravity')
>>> gravity.experiment_id
'GDCN4d'
>>> gravity.timestamp
1673648406
>>> gravity.seed
1632827863

Invocations of launch() after successful execution, do not trigger another execution since the experiment is already finished. On the other hand, if the execution failed, calling launch() will resume the execution with the same configuration.

To replicate or reproduce an experiment, create a new experiment instance with the same configuration. Learn more about continuing and repeating experiments.

Implementing custom execution #

Experiments can be executed in different ways. You may, for example, like to run experiments using multiprocessing or execute on a cloud environment. However, instead of adding the execution logic directly to your experiment code, machinable makes it easy to separate concerns. You can encapsulate the execution implementation in its own execution class that can then be used to execute the experiment.

To implement an execution, create a module with a class that inherits from the Execution base class, for example:

from multiprocessing import Pool

from machinable import Execution


class Multiprocess(Execution):
    class Config:
        processes: int = 1

    def on_dispatch(self):
        pool = Pool(processes=self.config.processes, maxtasksperchild=1)
        try:
            pool.imap_unordered(
                lambda experiment: experiment(),
                self.experiments,
            )
            pool.close()
            pool.join()
        finally:
            pool.terminate()

Much like in the case of experiments, the execution class provides a Config dataclass and implements the on_dispatch event that handles the execution of the given self.experiments by calling them within a subprocess (experiment()).

Like before, we can instantiate this execution using the module convention:

from machinable import get

multiprocessing = get("multiprocess_execution", {'processes': 2})

Then, to use it, we can wrap the launch in the execution context:

with multiprocessing:
    experiment.launch()

Check out the execution guide to learn more about executions. You may also be interested in the execution examples that you may like to use in your projects.

Using the CLI #

machinable provides a powerful CLI out of the box that closely mirrors the Python interface. To run an experiment, type its module name and method name, optionally followed by the configuration options, for example:

machinable estimate_gravity time_dilation=1.5 --launch

To use multiprocessing, you can type:

machinable estimate_gravity time_dilation=1.5 \
  multiprocess_execution processes=4 --launch

To learn more refer to the CLI guide.