hichamjanati / mutar Goto Github PK

Hi, it seems there is a problem when using 'set_params' on GroupLasso?

The code below should reproduce the error: some_coeffs is different from some_other_coeffs. I would think some_other_coeffs contains the right answer: when alpha is large they should be zero? Apparently, there is an issue with the 'specific coeffs' becoming non zero at some point? Am I right? These ones should always be zero?

`from mutar import GroupLasso
import numpy as np
X = np.array([[[3, 1], [1, 0], [1, 0]],[[0, 2], [2, 3], [2, 3]]], dtype=float)
y = X.sum(axis=2) + 2
glr = GroupLasso().fit(X, y)
glr.fit(X, y).coef_
some_coeffs = glr.set_params(alpha=1e4).fit(X, y).coef_

glr2 = GroupLasso(alpha=1e4)
some_other_coeffs = glr2.fit(X, y).coef_
`

I was wondering if in the future it might also be possible to support l1,linfinity penalization, which is known as the "simultaneous LASSO", see
Turlach 2005, https://www.tandfonline.com/doi/pdf/10.1198/004017005000000139
Liu et al 2009, https://www.cs.cmu.edu/afs/cs.cmu.edu/Web/People/fmri/papers/168-Blockwise-Coord-Descent.pdf (maybe the best en fastest algo, confusingly called multi-task LASSO here, but using l1,linfinity penalty unlike the l1,l2 penalty in sklearn's MultiTaskLasso)
Quattoni et al 2009, https://dspace.mit.edu/bitstream/handle/1721.1/59367/Collins_An%20efficient.pdf?sequence=1&isAllowed=y
Vogt & Roth 2010,
https://www.researchgate.net/profile/Volker_Roth/publication/262409253_The_group-lasso_l_1_regularization_versus_l_12_regularization/links/09e41512b178be6c04000000/The-group-lasso-l-1-regularization-versus-l-1-2-regularization.pdf

Or its generalization: l1-lq penalization, see
https://arxiv.org/pdf/1009.4766

This creates fits with greater sparsity than l1,l2 and fewer false positives, so could be useful for many multitask learning applications with a shared sparsity structure... But I haven't found any open implementations anywhere...

Great work, I have been using quite a lot and like it a lot! However, I have an error when using the warm start for the Group Lasso:
from mutar import GroupLasso
import numpy as np
X = np.array([[[3, 1], [1, 0], [1, 0]],[[0, 2], [2, 3], [2, 3]]], dtype=float)
y = X.sum(axis=2) + 2
grouplasso = GroupLasso(warm_start=True).fit(X, y)
grouplasso.fit(X, y).coef_

UnboundLocalError: local variable 'coef_shared_' referenced before assignment

By any chance do you have a quick workaround?