jla-gardner / digital-experiments Goto Github PK

Keeping track of the results of coding experiments can be a pain. Over time, code and dependencies can change, and without careful record-keeping, it becomes difficult to remember and reproduce optimal configurations and results.

digital-experiments automates such tracking. To enable this automation, wrap your experiment's main function with the @experiment decorator. Every time the function is called, the following information is saved to disk:

• the inputs (args, kwargs and defaults)
• the output/s (any, arbitrary object)
• the code of the function
• the current git information (if available)
• timing information
• python environment information

This information is available for analysis in the same or different python sessions, via the observations API.

To get started, see the basic use case below, or our example notebook.

pip install digital-experiments

1. Define your experiment as a pure-python function, and decorate it with @experiment:

from digital_experiments import experiment

@experiment
def my_experiment(a, b=2):
    return a ** b

2. Call the function as normal:

>>> my_experiment(2, 3)
8
>>> my_experiment(4)
16

3. Access the results of the experiment:

>>> my_experiment.observations()
[Observation(<id1>, {'a': 2, 'b': 3} → 8}),
Observation(<id2>, {'a': 4, 'b': 2} → 16})]

If you have pandas installed, you can also access these results as a DataFrame:

>>> my_experiment.to_dataframe()
      id  config.a  config.b  result
0  <id1>         2         3       8
1  <id2>         4         2      16

For more information, see the documentation.