Giter Club home page Giter Club logo

pd3o's People

Contributors

mingyan08 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

e0189584

pd3o's Issues

The Prox of the Conjugate of `h()`

In the paper the algorithm steps are:

image

The step for ${s}^{+}$ requires the Prox operator of the conjugate of $h \left( \cdot \right)$.
In case only the Prox of $h \left( \cdot \right)$ is given then:

$$\text{prox}{\delta {h}^{\ast} \left( \cdot \right)} \left( y \right) = y - \delta \text{prox}{ \frac{ h \left( \cdot \right)}{\delta} } \left( \frac{y}{\delta} \right)$$

image

In the code what actually is done is $y - \text{prox}_{ \delta h \left( \cdot \right) } \left( y \right)$:

image

Is there a reason for that?

Remark: In the code the parameter delta is called mu.

Simple Example Where the Algortihm Will Not Converge for the Article γ and λ

I have a simple case where the algorithm won't converge.
The problem is given by:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} \boldsymbol{x}^{T} \boldsymbol{A} \boldsymbol{x} \quad \text{ subject to } \quad \boldsymbol{A} \boldsymbol{x} \in {\left[ a, b \right]}^{m} $$

For $\boldsymbol{A} \in \mathbb{R}^{m \times n}$ where in the case above $m = n$.

The model for the algorithm is:

$$ \frac{1}{2} \boldsymbol{x}^{T} \boldsymbol{A} \boldsymbol{x} + {\Pi}_{{\left[ a, b \right]}^{m}} \left( \boldsymbol{A} \boldsymbol{x} \right) $$

So $\beta = \frac{1}{ {\left| \boldsymbol{A} \boldsymbol{A}^{T} \right|} }$.
I took your defaults from the Fused LASSO demo: $\gamma = 0.5 \beta, \quad \lambda = \frac{3}{16}$.
Defining $\delta = \frac{\gamma}{\lambda}$ (The code uses $\mu$ for this) we indeed fulfill:

image

Yet when I run the code, it will break. Here the simple code (Based on Fuses Lasso code):

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%   min    1/2 x' * A * x + delta(A * x)   (1)
% Where delta(A * x) is Indicator function for \in [a, b]
%
% f(x) = 0.5 * x' * A * x
% g(x) = 0 -> Prox_G(y) = y
% h(A * x) = delta(A x) -> Prox_H(y) = max(min(y, b), a)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
close all
clear
clc;
addpath('fcns','data','output') 
numRows = 5;
numCols = numRows; %<! PSD Matrix
% ---------------------- Generate random data ----------------------
randNum     = 1;
%randn('state', (randNum - 1) * 3 + 1);  % old version
rng((randNum - 1) * 3 + 1, 'v5normal');
A           = rand(numRows, numCols);
A           = A' * A;

% Indicator boundaries: A * x \in [valA, valB]
valA = 0.25;
valB = 1.25;

%----------------------- Set parameters for the problem ------------------------
beta        = 1/normest(A*A');
obj         = PrimalDual;  % using the clas PrimalDual

%% Define all its functions:
% Create handles to functions F, G, H, A, and H*:
obj.myF = @(x) 0.5 * x' * A * x;
obj.myG = @(x) 0;
obj.myH = @(y) 1e6 * any((y > valB) | (y < valA)); %<! Indicator function
obj.myHA = @(y) 0; %<! No use
obj.myA = @(x) A * x;
% Create handles to adjoint/gradient/prox:
obj.myGradF = @(x) A * x;
obj.myProxG = @(x, t) x;
obj.myProxH = @(y, t) min(max(y, valA), valB);
obj.myAdjA  = @(y) A * y; %<! Symmetric

%% Define the parameters for the algorithm
obj.gamma   = 0.5 * beta; % two parameters for the primal-dual algorithms
obj.lambda  = 3/16;       % we will choose different gammas in this example

iter    = 4000;           % the number of iterations

obj.input.x     = A \ (((valA + valB) / 2) * ones(numCols, 1)); %<! Valid initialization
obj.input.s     = zeros(numCols, 1);
obj.input.iter  = iter;


%% Run the primal-dual codes
% PD3O %%%
j   = 1;
tic
[x_PD3O, s_PD3O, E_PD3O, out_PD3O]  = obj.minimize('PD3O', []);
time(j) = toc;

x_PD3O

This will result in NaN.
If I reduce the fectors to $\gamma = 0.1 \beta, \quad \lambda = \frac{1}{128}$ it will converge.
Is there an explanation?

Remark: There is a bug in PrimalDual.m. In case we use type = [], namely no checks, it will fails as not struct out is defined.
A simple fix is to add after line 86: out = struct();. Then it will work.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.