Exponential families and natural parameters about zhusuan HOT 9 OPEN

Great idea. Exponential families are always linked to conjugacy, which is very hard to have an automated design. I guess that's why existing frameworks don't have a good support (though some claim to ). But we may start from the easy case. Maybe first a general distribution named ExponentialFamily? And let all subclasses have the natural parameter attributes.

from zhusuan.

Just for some pointers, do you know any frameworks which have/claim to have some support of this as it would be good to see what people have come up in terms of ideas and what problems they got in the way?

from zhusuan.

Edward has some preliminary design of conjugacy. But they are doing this based on graph copying, which is a feature that TF people just don't like (see this PR by me). This design seems to be deprecated by the author of Edward. They are currently working on a new version based on function reuse like us, but it's not clear what they will do about the original conjugacy features.

from zhusuan.

I'm also interested in implementing some examples shown in the structured vae paper. That's a very neat paper. If you could outline some detailed needs, I'm happy to help with it.

from zhusuan.

Update: @liyr will start working on a general abstraction for exponential families.

from zhusuan.

I had some thoughts on this and I think essentially what is needed is something similar to the query, but which given some observations if the structure of the (remaining) graph is conjugate gives you back the parameters (natural or not) of the exact posterior or throws an error if it is not conjugate. The main challenges is basically how to tag things such that you can indeed detect if things are conjugate or not.

from zhusuan.

I like it. That's the ultimate goal. We are working at the very beginning though, to have some an ExponentialFamily abstraction, and something like Normal.from_natural_parameter(). Detecting conjugacy for single-particle models is easy, But it's getting hard as you have many plates in the graphical model.

from zhusuan.

Hi,

Any updates on this?

from zhusuan.

We have some attempts here, but they are far from perfect. @sameerkhurana10

from zhusuan.