Code from paper 'Uncertainty in Neural Networks: Bayesian Ensembling' - https://arxiv.org/abs/1810.05546
Play with the interactive demo here. See how it compares to dropout here.
notebook_examples provides lightweight demo code for a Bayesian anchored ensemble in PyTorch / Keras / Tensorflow for regression and classification.
[FULL EXPERIMENTAL CODE TO BE UPLOADED SOON]
For queries, email me at tp424 [at] cam.ac.uk.
Folder 03_trimmed_down provides a carved out piece of code from the full experimental code (still to be uploaded).
- script_methods.py - main program, run this file to produce some graphs
- DataGen.py - handles data set generation or read in
- utils.py - handles some plotting and support functions
- module_gp.py - code behind the equivalent gaussian processes
- module_NN_ens.py - where the magic happens! - includes the regularisation around initialisation values (the ‘anchor’ method, line 155). Is only set up for single or double layer fully connected NN.
Other thoughts:
- Does NOT include modules for the other methods, such as variational inference and HMC
- I’m not sure how robust the code will be to deviations from the specific versions of packages (I had conflict issues between Tensorflow and Edward) and python (3.6) I used, so may require some tweaking depending on what you’re running
- code in module_NN_ens.py (and more generally everywhere) is much more complex than needed - I added a lot of extra functionality during development and testing, trying things out which aren’t necessarily needed in the final version.
- Hopefully variable names etc should be obvious enough, parameters I think you’ll want to vary are mainly controlled in the inputs section in script_methods.py