This is a centralized repository for all of the machine learning examples I've implemented. Not everything I've done is here, but I'm attempting to make this repository as clean as possible.

Each directory contains an algorithm that only depends on other code or data present in this repository. Most of the code is written in Python and MATLAB, and specific dependencies for each project are noted. Each example will have an associated blog post on my personal website (http://briandolhansky.com/).

The following items will be added to this repository (with associated blog posts) in the future:

Regression

• Ridge Regression
• LASSO

Logistic Regression

• Simple LR, batch gradient descent
• Simple LR, stochastic gradient descent
• Sparse LR (L1 regularized)
• LR for big data

Decision Trees

• Single decision tree
• Random forest
• Gradient tree boosting

Determinantal Point Processes

• Simple DPP on a 2D grid of points
• DPP sampling applied to real data

Neural Networks

• Simple feedforward network with explicit network function
• Feedforward network with matrix multiplies

Hidden Markov Models

• Simple HMM with forward/backward on a handwriting task

Conditional Random Fields

• Linear chain CRFs
• Grid CRFs with loopy belief propagation

Submodular Optimization

Algorithms and Techniques

• SAFE feature elimination
• AdaGrad
• Confidence Weighted Linear Classifiers and AROW

This is by no means a complete list, and more may be added in the future.

Brian Dolhansky [email protected]