hiroyuki-kasai / sgdlibrary Goto Github PK
View Code? Open in Web Editor NEWMATLAB/Octave library for stochastic optimization algorithms: Version 1.0.20
License: MIT License
MATLAB/Octave library for stochastic optimization algorithms: Version 1.0.20
License: MIT License
Greetings Hiroyuki,
I'm so excited to have come across such a rich library in SGD, thank you for these great project.
I'm currently learning Python, Matlab and Machine Learning for data training and prediction.
I have collected data as bitmap files (400 files each containing 1000 ~ 1s and 0s), I wanted to do data training and prediction on single file first, and later a couple of files (5 files, 10 files, 20 files... and so on). These bitmap files are in .txt (text files) format.
Kindly advice, how I can use these bitmap files as input?
Lawrence.
Hello,
@hiroyuki-kasai Thank you for creating this project with a wide variety of algorithms.
I went through the code in linear_regression_data_generator.m and was not quite clear how the data is being generated. Also, on running the code I find that all my rows have the same number. Can you explain how the dataset is generated for linear regression?
% set number of dimensions
d = 50;
% set number of samples
n = 7000;
% generate data
std = 0.25
data = linear_regression_data_generator(n, d, std);
Attached below is the data(x_train) and label(y_train) generated
Thank you!
Hello, this library is really good. I am now learning machine learning algorithms. It helps me a lot and I have some problems. When I was learning this library, I saw the data in the file data. I wanted to know what the data meant, where it was collected, and what each data represented. Thank you very much
Thanks for developing such a library. When trying to use it, I found that when I change the values of "d" and "n" in the file "demo.m", there will be some errors in the convergence results. For instance, if I choose "d = 105", "n = 30000", the returned results are very strange as shown before:
SVRG: Epoch = 000, cost = Inf, optgap = Inf
SVRG: Epoch = 001, cost = NaN, optgap = NaN
Is there any problem with the code for the reusability?
Thanks for developing such a library. I am trying to learn the SGD through the demon, it helps me a lot. But when I try to run this demo(test_l1_linear_regression.m), the command line throw an error:
The method, attribute or field'prox_flag' of class'l1_linear_regression' is not recognized.
Dear Prof. Hiroyuki KASAI,
I have some confusion about the damping scheme in obfgs.
You gave a citation of Wang et al. 's "Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization," when mentioning the damping, but I found that the code is more likely to be the classical Powell's damping. In the limited memory setting, Wang's damping only requires one two-loop-recursion but in your implementation you call it twice. In addition, the threshold value indicating when to carray out the damping is set to 0.2 in your code, which is the same to Powell's method but different from the Wang's. I also found a probable bug, if it is indeed the Powell's damping scheme. In Line 144, the probablly correct code is "lbfgs_two_loop_recursion(s, s_array, y_array)" (otherwise you may directly reuse the variable calculated before). Is this a typo or something wrong with my opinion?
Thanks!
Greetings,
First, I would like to thank you for creating such a nice project with so many optimization algorythms!
I'm currently trying to port the online memory limited BFGS algorythm to python ( yes scipy has many algorythm coded in haskell but it's lacking some that I would like to test, namely this one.)
I've got most of it done but I'm confused about the gradient descent part using the weight on the input.
I'm not familiar with matlab and I want to know if calling .cost , .grad or anything else on the input (the variable problem
) are integrated method in the problem
code itself as a method or is it a matlab built-in?
P.S.:Thanks for all your hard work!
Hi @hiroyuki-kasai ,
Thanks for publishing this project. This looks great and I would like do some research with this toolbox.
My trouble is in the sgd_cm.m. It looks like containing two momentum schemes, the classic one ('CM') and the Nesterov's ('NAG'), but in the current implementain, they seem to differ only in the setting of the momentum coefficient. See lines 78, 80, and 82.
In my impression NAG should 'look one step ahead' before the gradient calculation, but in the code, the gradient is evaluated just in the current point. This seems to be inconsistent to the original paper. See equations 3 and 4 in Ilya Sutskever, James Martens, George Dahl and Geoffrey Hinton, "On the importance of initialization and momentum in deep learning," ICML, 2013.
Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.