Comments (6)
This is a 6 page short summary on Stein Variational Gradient Descent:
http://www.cs.utexas.edu/~lqiang/PDF/svgd_aabi2016.pdf
This short paper gives a brief overview of the key idea of SVGD, and outline several directionsfor future work, including a new theoretical framework that interprets SVGD as a natural gradientdescent of the KL divergence functional on a Riemannian-like metric structure on the space ofdistributions, and extensions of SVGD that allow us to train neural networks to draw approximatesamples from given distributions, and develop new adaptive importance sampling methods withoutassuming parametric forms on the proposals.
from covid19_inference.
In principle one can use the ASVGD inference, however, one might have to play with the priors a bit to help the algorithm converge. Also, it is not yet recommended to use. What helped in my case to make it converge more robustly:
- mean prior for lambda_0 of 0.6.
- prior for I_0 of around 50.
- adam with learning rate 0.02
- 300 steps at least .. which takes around 6 minutes in a google colab notebook
If you know how to apply SVGD or a similar VI to boost computing time of the posteriors, let us know here.
from covid19_inference.
General comment: If you are hell bent on using variational methods, consider looking at pyro since that framework originally was conceived for exactly that. SVGD exists, but the implementation claims to be 'basic'.
https://pyro.ai/
http://pyro.ai/examples/svi_part_i.html
Any particular reason why variational stuff should be better? People use it to train bayesian neural networks and with those number of parameters sampling becomes unfeasible. But given the few parameters of the current model that should not be a problem.
from covid19_inference.
We are not hell bent on it. My thought was that variational methods could eventually be parallelized, when we will look at the level of Landkreise.
But that I find that interesting, intrinsically are Monte-Carlo methods scaling worse than variational methods with the number of parameters? My take was that the advanced methods like Hamiltonian MC scale pretty well with the number of parameters. But my knowledge is pretty superficial.
from covid19_inference.
One this issue, no one is actively working on. So if someone want to have a look...
Things learned so far:
- the pymc3.ASVGD needs very high temperature, around 2 to give more or less the correct posterior, one the 1 dimensional problem.
- It takes time until ASVGD and SVGD converges, mainly probably because of wide distribution of I_begin. Learning rates around 0.01 seems to work.
Next steps would be use pymc3.SVGD with about 100 particles and try to apply it to the example_bundeslaender, to see whether one gets faster to approximate good results. There one can also test whether theano uses multiprocessing.
from covid19_inference.
So basically how can people help @jdehning
A different question
how do we infer these rates for different countries?
Are they universal?
from covid19_inference.
Related Issues (20)
- Testing data class
- JHU-data has different format HOT 2
- Example SIR model with one german state has negative lambda values? HOT 9
- Advanced plotting in example_bundeslaender.ipynb is broken
- delay_cases function parameters
- cannot import name 'data_retrieval' HOT 4
- Import error when module covid19_inference is installed (instead of loaded from directory) HOT 1
- Region population size HOT 3
- AttributeError: 'RKI' object has no attribute 'get_new_confirmed' & 'get_confirmed' HOT 2
- Script for simple SIR and SEIR models HOT 2
- Offset in _new_cases_to_cum_cases seems to be broken
- Windows user strftime bug HOT 2
- plot._distributions does not acoomodate named models HOT 3
- Error with Trace(MCMC) in Example_bundeslaender HOT 1
- doubt about the posterior distributions HOT 2
- Plotting: AssertionError: var should be in trace.varnames HOT 6
- ERROR [pymc3] The rhat statistic is larger than 1.4 for some parameters. The sampler did not converge. HOT 6
- KeyError when using the "state" parameter in jhu "get_new" and "get_total_confirmed_deaths_recovered" methods HOT 3
- Nan errors in example notebooks HOT 3
- :noIssue: ❓ disappeared updates of divi json data HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from covid19_inference.