Sacred
Sacred is a tool to configure, organize, log and reproduce computational experiments. It is designed to introduce only minimal overhead, while encouraging modularity and configurability of experiments.
The ability to conveniently make experiments configurable is at the heart of Sacred. If the parameters of an experiment are exposed in this way, it will help you to:
keep track of all the parameters of your experiment
easily run your experiment for different settings
save configurations for individual runs in files or a database
reproduce your results
Features
1. Config Scopes A very convenient way of the local variables in a function to define the parameters your experiment uses.
2. Config Injection: You can access all parameters of your configuration from every function. They are automatically injected by name.
3. Command-line interface: You get a powerful command-line interface for each experiment that you can use to change parameters and run different variants.
4. Observers: Sacred provides Observers that log all kinds of information about your experiment, its dependencies, the configuration you used, the machine it is run on, and of course the result. These can be saved to a MongoDB, for easy access later.
5. Automatic seeding helps controlling the randomness in your experiments, such that the results remain reproducible.