Machine learning is how to get computers to automatically learn and improve with experience. Experience comes in the form of data, improvement is with respect to some performance metric, and learning is done by a learning algorithm. There are always computational constraints, such as the architecture, computation time, bandwidth limitations, and so on. So we can more precisely restate the goal thus: to construct learning algorithms that use data to improve with respect to a performance metric and do so under computational constraints.

We will focus on principles of statistical machine learning in the prediction problems, regression and classification. Conspicuously absent is most Bayesian methodology and advanced concepts such as reinforcement learning. This course is not a broad overview of all of machine learning, but rather a tour of the key ideas in machine learning as told through these prediction tasks. Typically, I have students tell me something along the lines of "I thought machine learning was about [insert random methodology here]". Machine learning is a field, like physical chemistry or creative literature. It is not defined by a couple of methods or a single task, and cannot be taught in a single quarter. With that said, I want this course to lay the foundation for a rich understanding of machine learning.

Instructions: The lectures will mostly be jupyter notebooks. To follow along with the slides use the following command in the lecture folder.

jupyter nbconvert lecture[# here].ipynb --to slides --post serve

Due to Covid-19, I have recorded all of my lectures and am uploading them to Youtube. They will be linked as they become available, and will be organized into playlists which correspond to a single lecture (between 1-2 hrs of content). I have structured the lecture so that there are exercises that you can do on your own as you listen to the lecture. Most of the exercises are at the end of a video and will be answered at the beginning of the next, but sometimes I will ask you to pause the video. After I ask the question you should take the time to complete the exercise, then go on to hear the answer.

Introduction to Machine Learning

Principles: Bias-Variance, training and testing, losses, OLS and KNN
Reading: ESL Chapter 2

Principles: Convex relaxation, computational intractability in subset selection
Reading: ESL Chapter 3, Boyd Chapter 1

Principles: Surrogate losses, generative and discriminative methods
Reading: ESL Chapter 4

Principles: HMMs, Clustering, Dimension Reduction Reading: ESL Chapter 14, "An Introduction to Hidden Markov Models and Bayesian Networks", Zoubin Ghahramani

Principles: basis expansion, kernel trick, bagging, boosting, neural nets
Reading: ESL Chapter 5, 7, 8