This short lesson summarizes the topics we covered in section 12 and why they'll be important to you as a data scientist.
You will be able to:
- Understand and explain what was covered in this section
- Understand and explain why this section will help you become a data scientist
The goal of section 12 was to give you some practice going through the end to end process of obtaining, scrubbing, exploring (EDA), and modeling a given data set to create a prediction. Some of the key takeaways include:
- It's important to think about providing value from your data science projects - not just modeling efficiently. Most of the business value comes from asking the right questions and persuasively communicating the results, and much of the predi ctive value comes from thoughtful selection, cleanup and exploration of the data set.
- The most common data science frameworks are CRISP-DM, KDD and OSEMiN
- The process of finding, filtering and loading the appropriate data to answer a question is non-trivial
- Decisions made in the data munging/scrubbing phase can have a huge impact on the accuracy of your predictions
- Visualization is a key phase in EDA.
- A Kernel Density Estimation (KDE) plot can be very useful in getting a sense for visualizing the distribution of a column
- A Joint plots can be very useful for bringing together multiple visualizations to (for example) check the linearity assumption between predictors and a target variable
- It's important to "hold out" a subset of your data for testing (as opposed to training) so you can see how accurately it works for predicting "unknown data"