In this lab, you'll practice your linear transformation skills!

You will be able to:

• Determine if a linear transformation would be useful for a specific model or set of data
• Identify an appropriate linear transformation technique for a specific model or set of data
• Apply linear transformations to independent and dependent variables in linear regression
• Interpret the coefficients of variables that have been transformed using a linear transformation

Let's look at the Ames Housing data, where each record represents a home sale:

import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('seaborn-v0_8-darkgrid')

ames = pd.read_csv('ames.csv', index_col=0)
ames

We'll use this subset of features. These are specifically the continuous numeric variables, which means that we'll hopefully have meaningful mean values.

From the data dictionary (data_description.txt):

LotArea: Lot size in square feet

MasVnrArea: Masonry veneer area in square feet

TotalBsmtSF: Total square feet of basement area

GrLivArea: Above grade (ground) living area square feet

GarageArea: Size of garage in square feet

ames = ames[[
    "LotArea",
    "MasVnrArea",
    "TotalBsmtSF",
    "GrLivArea",
    "GarageArea",
    "SalePrice"
]].copy()
ames

We'll also drop any records with missing values for any of these features:

ames.dropna(inplace=True)
ames

And plot the distributions of the un-transformed variables:

ames.hist(figsize=(15,10), bins="auto");

SalePrice should be the target, and all other columns in ames should be predictors.

# Your code here - build a linear regression model with un-transformed features

Describe the model performance overall and interpret the meaning of each predictor coefficient. Make sure to refer to the explanations of what each feature means from the data dictionary!

# Your written answer here

Your stakeholder gets back to you and says this is great, but they are interested in metric units.

Specifically they would like to measure area in square meters rather than square feet.

Report the same coefficients, except using square meters. You can do this by building a new model, or by transforming just the coefficients.

The conversion you can use is 1 square foot = 0.092903 square meters.

# Your code here - building a new model or transforming coefficients
# from initial model so that they are in square meters

# Your written answer here

Your stakeholder is happy with the metric results, but now they want to know what's happening with the intercept value. Negative \$17k for a home with zeros across the board...what does that mean?

Center the data so that the mean is 0, fit a new model, and report on the new intercept.

(It doesn't matter whether you use data that was scaled to metric units or not. The intercept should be the same either way.)

# Your code here - center data

# Your code here - build a new model

# Your written answer here - interpret the new intercept

Finally, either build a new model with transformed coefficients or transform the coefficients from the Step 4 model so that the most important feature can be identified.

Even though all of the features are measured in area, they are different kinds of area (e.g. lot area vs. masonry veneer area) that are not directly comparable as-is. So apply standardization (dividing predictors by their standard deviations) and identify the feature with the highest standardized coefficient as the "most important".

# Your code here - building a new model or transforming coefficients
# from centered model so that they are in standard deviations

# Your written answer here - identify the "most important" feature

Great! You've now got some hands-on practice transforming data and interpreting the results!