Weighted loss function about proxtv HOT 4 CLOSED

Hi David,

What is the equation for the whole objective function you want to optimize? If you want to use the usual regularizers for fused lasso, and you are only changing the loss function, then you solve that fairly easily. You can use a proximal gradient or a FISTA algorithm, providing a way of computing the gradient for your loss, and the tools you can find in proxTV for the Total-Variation + L1 regularizer.

from proxtv.

Hi,

First to say is that I'm not an expert with the algorithms. The full function has the usual regularizers and looks like:

min ||Wi(Xi-Yi)||^2 + lambda Wi(Xi-Xj)

in two dimensions.

We are using now a graph fused lasso package from https://github.com/tansey/gfl which works well and can be used for different loss functions. The problem is that the algorithm used there ADMM seems to converge slowly and we are struggling to make it faster as we don't see how to parallelize it.
Can we parallelize FISTA?
If you have suggestions I'd appreciate.

Thanks

David

from proxtv.

ADMM is indeed quite slow unless you find good stepsize parameters, which is no easy task in general. But this problem should be solvable through a combination of FISTA and the 2D Total Variation proximity solver provided in proxTV.

You see, FISTA is a general algorithm for minimizing a sum of a differentiable function (your loss) and a non-differentiable function. For the differentiable part you need to provide a way to compute the gradient of that part, which at first glance should be simple for your problem. For the non-differentiable part you need to provide the prox operator, but you already have that coded in proxTV!

About parallelization, FISTA essentialy alternates between calls to the gradient function and the prox operator you provided. 2D Total Variation here alredy admits parallelization, so half of your problem is already solved.

FISTA is not implemented in this library, though I created some scripts using it for the related paper. I'm attaching a general implementation of FISTA in MATLAB. As you will see, you need to provide gradient and prox functions approriate for your problem, as discussed above.

FISTAB.txt

Hope this helps!

from proxtv.

It definitely helps a lot.
I'll try to implement it and compare with what we have.
Thanks

from proxtv.