compute the Hessian manually about celeste.jl HOT 6 CLOSED

I'm assigning myself this for the task of hard-coding the actual Hessian, not just the diagonal.

from celeste.jl.

Just curious, Jeff, do you have thoughts about what kind of data structure would be best for the Hessian object inside the SensitiveFloat? You know much more about the performance of different julia data types than I do.

I'm planning to treat the Hessian of the brightness (G in the derivations) differently from the Hessian of the ELBO, because the Hessian of G is only nonzero within a source. Also, we'll pretty much only write to the full Hessian once, whereas the G Hessians will need to be read again and again as we perform transforms on SensitiveFloats.

I'm currently just treating the G Hessians as length(ids) x length(ids)matrices. Most of the entries in these matrices are zeros most of the time. But I'm not sure if the lookup costs (and complexity) that would be associated with sparse matrices or some other custom data structure would be worth it.

Better to get your thoughts sooner rather than later!

from celeste.jl.

How about H::Vector{Matrix{Float64}}? The outer vector has one entry per
source. Each inner matrix is roughly 30x30, a dense matrix that stores the
block of the Hessian corresponding to a particular source.

An alternative is Array{Float64, 3}, but this is worse because until Julia
0.5 H[:,:,s] copies rather than returns a view without copying.

Another alternative is a sparse matrix. That's no good either since blocks
of it are dense, and we only want to compute the block diagonal anyway, I
think.

On Mon, Nov 16, 2015 at 3:35 PM Ryan [email protected] wrote:

Just curious, Jeff, do you have thoughts about what kind of data structure
would be best for the Hessian object inside the SensitiveFloat? You know
much more about the performance of different julia data types than I do.

I'm planning to treat the Hessian of the brightness (G in the derivations)
differently from the Hessian of the ELBO, because the Hessian of G is only
nonzero within a source. Also, we'll pretty much only write to the full
Hessian once, whereas the G Hessians will need to be read again and again
as we perform transforms on SensitiveFloats.

I'm currently just treating the G Hessians as length(ids) x length(ids)matrices.
Most of the entries in these matrices are zeros most of the time. But I'm
not sure if the lookup costs (and complexity) that would be associated with
sparse matrices or some other custom data structure would be worth it.

Better to get your thoughts sooner rather than later!

—
Reply to this email directly or view it on GitHub
#15 (comment)
.

from celeste.jl.

Cool, good to know. I had been using Array{Float64, 3} but I didn't realize that problem with copying. I'll do Vector{Matrix{Float64}}.

Could I also ask you to comment up the parameter stuff starting around line 400 of CelesteTypes.jl? I'm considering writing generic tools for taking functions of SensitiveFloats. To do that I will have to be able to define a SensitiveFloat of arbitrary size, but you've tied the SensitiveFloat type to the abstract ParamSet type which is used inside a bunch of meta-programming that I haven't managed to wrap my head around. (Sorry, I think it'll be quick for you to add a few comments and slow for me to try to figure out on my own what's going on in there. My julia meta programming chops are not so good.)

from celeste.jl.

Hey Ryan, I'll add some comments but I can't get to it today. That block of
metaprogramming didn't turn out to be a slick as I'd hoped: you should feel
free to modify/delete it as you see fit. My aim was to use the type system
in Julia to control how different types of derivatives related to each
other, without any run time penalty, but now I think there must have been a
better way

On Mon, Nov 16, 2015 at 3:54 PM Ryan [email protected] wrote:

Cool, good to know. I had been using Array{Float64, 3} but I didn't
realize that problem with copying. I'll do Vector{Matrix{Float64}}.

Could I also ask you to comment up the parameter stuff starting around
line 400 of CelesteTypes.jl? I'm considering writing generic tools for
taking functions of SensitiveFloats. To do that I will have to be able to
define a SensitiveFloat of arbitrary size, but you've tied the
SensitiveFloat type to the abstract ParamSet type which is used inside a
bunch of meta-programming that I haven't managed to wrap my head around.
(Sorry, I think it'll be quick for you to add a few comments and slow for
me to try to figure out on my own what's going on in there. My julia meta
programming chops are not so good.)

—
Reply to this email directly or view it on GitHub
#15 (comment)
.

from celeste.jl.

Done in 33ae77a.

whew

from celeste.jl.