In order to understand the behavior of machine learning optimized algorithms better, this project studied an application of Kaggle competition by several algorithms with different step-size and variants. Dealing with real world problem was pretty challenging since the dataset from Kaggle competition was taken from real-world including number of samples with various features. Also, the efficiency was important for practical problems. So the algorithms applied in machine learning were the key factor in deciding success. Aiming at finding the most suitable algorithm to provide predictions by applying several methods, namely, gradient descent method, gradient descent with momentum(including heavy ball method and Nesterov’s method), decomposition-type methods(stochastic gradient descent), quasiNewton algorithm(BFGS method) and BB (Barzilai-Borwein) Method (including LBB method and SBB method).

in this project, the database got from: https://www.kaggle.com/uciml/breast-cancer-wisconsin-data/data

first step:try to normalized data which get from kaggle with python as programming language

second step: try different optimization methods including gradient descent, heavy ball methed, coordinate descent and Explored the different convergence behavior in terms of convergence speed, efficiency.

finally, comparing with different behaviors with the same database when using these three kinds of optimization algorithms and make a conclusion with a summary of the best performance algorithm.