Quick links:

• Course syllabus
• Materials and software
• Exercises

The team:

• James Scott, instructor. Office hours MW 3:20 to 4:20, CBA 6.478.
• Molin Li, teaching assistant. Office hours Friday at 2PM, immediately after TA session.

This class is about gaining knowledge from data and about understanding random phenomena. We'll cover a mix of practice and principles:

• In part 1, you'll learn the mathematical foundations of probability, the language for expressing judgments about uncertain outcomes.
• In part 2, you'll learn some concrete data-crunching skills, using the R language.
• In part 3, you'll gain a solid technical understanding of some essential statistical ideas: estimation, hypothesis testing, and a few others.

There's also an important intermediate-term goal: prep you for Econometrics this upcoming academic year.

In this first part of the course, we'll learn the mathematical foundations of probability.

Slides: Introduction to Probability

R scripts and data sets:

• predimed_intro.R and predimed.csv

Two short pieces that illustrate the "fallacy of mistaken compounding":

• How likely is it that birth control could let you down? from the New York Times
• An excerpt from Chapter 7 of AIQ: How People and Machines are Smarter Together, by Nick Polson and James Scott.

Slides: Discrete random variables

R scripts:

• poisson_binomial.R

Slides: General random variables

R scripts:

• testmax_example.R
• normal_example.R and annual_returns_since1928.csv
• wellbore.R

Slides: Multivariate distributions

R scripts:

• bivariate_normal.R
• Case study on stocks and bonds

Midterm statistics: curved grades

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  65.00   83.50   86.80   86.25   92.30   98.90 

Midterm statistics: raw grades

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   35.0    70.0    76.0    74.9    86.0    98.0 

In this middle part of the course, we spend a lot more time hands-on with data sets and with R.

Topics: data visualization and practice with R. Bar plots; basic plots for numerical data (scatterplots, boxplots, histograms, line graphs); panel plots. Introduction to ggplot2.

Examples of bad graphics. And an example from the New York Times.

Slides: Introduction to Data Exploration

R scripts and data:

• mpg.R
• titanic.R and TitanicSurvival.csv
• toyimports_linegraph.R and toyimports.csv
• Green buildings case study

Inspiration:

• 50 ggplots
• A map of average ages in Swiss municipalities
• Low-income students in college
• The French presidential election
• LeBron James's playoff scoring record

Slides: Fitting equations

R scripts and data:

• afc_intro.R and afc.csv
• creatinine.csv
• utilities.R
• race_splines.R
• ebola.R

Slides: Introduction to the bootstrap

R scripts and data:

• creatinine_bootstrap.R and creatinine.csv
• predimed_bootstrap.R
• predimed_plugin.R

Slides: Introduction to hypothesis testing

R scripts and data:

• patriots.R
• power.R

Slides: Introduction to asymptotic theory

R scripts and data:

• clt.R
• normalCI_examples.R

Slides: Constructing estimators