bedirt / ground-up-machine-learning Goto Github PK

Fast-Track to Machine Learning:
A Curriculum Crafted for Newbies and Busy Bees

Welcome to Ground-Up Machine Learning (GML) - a personal project that sprang from my desire to demystify the world of machine learning (ML) for friends and colleagues who expressed a keen interest but found themselves overwhelmed by the existing resources. This course is my attempt to break down ML into digestible, engaging, and accessible segments, ensuring that anyone, regardless of their background, can grasp the fundamentals of machine learning and see its beauty and utility in the modern world.

1. Introduction & Overview
   • The Genesis of GML
   • Target Audience
   • Prerequisites
   • Tools and Libraries
   • Course Structure
   • Expected Effort
   • Ideal Way to Follow the Curriculum
   • Join the Journey
2. Week 1: Introduction to ML & Supervised Learning - Regression
   • Lecture Table of Contents
   • Homework Assignment: Building a Linear Regression Model from Scratch
3. Week 2: Supervised Learning / Classification - From Basics to Deep Dive into Logistic Regression
   • Lecture Table of Contents
   • Hands-On Lab Session: Multi-variable Logistic Regression Implementation
   • Homework Assignment: Exploring a New Classification Algorithm

Last month, my brother came to me and said that some of his friends and him are interested in learning about Machine Learning, and if I would be willing to hold classes for them and help out in giving some series of introductory lessons. Since I love teaching, and since my favorite crowd is people who are willingly putting themselves out there to learn, I jumped the gun and excitedly agreed.

While discussing the time requirements and how much time I can actually make them work on some extracurricular series (they are all already working hard on different things) we realized that there is little time and so much to learn.

This led to me doing some research on how a busy person can get exposed to Machine Learning as a complete entity without getting lost in the details and without having any background. This forced math out the door for the most part, and we decided a commitment of 8 hours a week (2h lecture + 6h self study).

In my research I couldn't find any resources that satisfies our criteria of having:

• Structured short introduction to ML that explores various types.
• Doesn't expect the students to have prior knowledge other than basics of python and high school math.
• Focuses on providing only the necessary ML components and theory and jumping into a practical example.
• Emphasizes on simple implementations and from scratch homeworks to enhance comprehension
• Expects students to only commit 5-8 hours a week.

So I created one for my class, hoping that I can use it for future as well!

This curriculum is mainly designed for:

• Absolute beginners curious about machine learning.
• Busy people looking for a streamlined yet meaningful overview of ML.
• Educators looking for a structured guide to introduce ML concepts to students (Could also be the starting point).

I think it's important to mention that the topics are not really designed for fully self-study, so a teacher or group study might be necessary. See Course Structure for more information on that.

• A foundational knowledge of Python is expected, as the course will briefly touch upon Python basics but primarily focus on applying it within ML context.
• While not math-heavy, a basic understanding of calculus and linear algebra can help your learning experience.

Throughout the lectures we leverage several Python tools and libraries:

• Python: The primary language of instruction.
• NumPy and Pandas: For numerical computing and data manipulation.
• Matplotlib/Seaborn: For visualizing data.
• Scikit-learn: For easy-to-use ML models.

It's not really necessary to be familiar with any of these libraries for the course though.

GML unfolds over four weeks, each dedicated to a different part of machine learning:

1. Introduction to ML & Supervised Learning - Regression
2. Supervised Learning - Classification
3. Unsupervised Learning & Clustering
4. Introduction to Reinforcement Learning

To fully benefit from this curriculum, students/participants are expected to commit to approximately 2-3 hours of lecture time (really depends on the teacher) and 6-7 hours of self-study each week. This self-study time includes going through lab materials, completing assignments, and additional reading or practice as needed. The weekly structure is flexible and can be adjusted based on individual or group needs, making the course adaptable to different learning environments.

GML is structured to maximize learning through interaction, hands-on practice, and experimentation. To get the most out of this course, we recommend the following approach:

1. Engage with the Lecture Content: Whether you're learning solo with an online community, in a classroom, or part of a study group, start by digesting the lecture content together. This collaborative approach allows for discussion, clarification of concepts, and sharing of insights, making the learning experience richer and more comprehensive.

2. Hands-on Practice with Lab Material: After the lecture, dive into the lab material. This is where you'll get practical experience with the topics and methods introduced during the lecture. The labs are designed to be interactive, allowing you to apply what you've learned in a guided environment. Ideally, this should be done with the support of a teacher or within your study group, providing a collaborative space to explore and learn from each other.

3. Weekly Assignments: Each week, you'll be tasked with an assignment that encourages you to implement and experiment with the week's topics. These assignments are crucial for deepening your understanding, as they require you to build concepts from scratch. Writing and debugging your code, and reflecting on your approach, will ensure you have a solid grasp of the material. These assignments are not only about reinforcing what you've learned but also about fostering creativity and problem-solving skills in real-world scenarios.

Learning machine learning, or any complex subject, is best achieved through a blend of theoretical understanding and practical application. By following this structured approach, you're not just passively absorbing information; you're actively engaging with the material, applying it in practical contexts, and solidifying your understanding through creation and collaboration. This methodology is designed to cater to diverse learning styles and to accommodate varying schedules, making ML accessible to everyone interested in embarking on this journey.

GML is more than just a course; it's an invitation to explore the world of machine learning in a way that's engaging, accessible, and, most importantly, grounded in real-world application. Whether you're teaching a class, learning with peers, or guiding yourself through the fundamentals of ML, GML offers the tools, insights, and inspiration to embark on this exciting journey.

Welcome to Ground-Up Machine Learning. Let's discover the power of machines that learn, together.

In this week, we will start with the basics of Machine Learning, and then move on to the fundamentals of Supervised Learning, focusing on Regression. We will also introduce the tools we will be using throughout the course, and discuss the importance of understanding data in Machine Learning.

I found that the best way to learn is by doing, so I am emphasizing that when teaching/studying this course we need to ask bunch of questions and do a lot of discussions. Also in the notebooks I tried to add bunch of interactive examples to make the learning process more fun and engaging.

Task(s):

• Ground up coding of the linear regression model and all the bits and pieces we discussed in class using only python and numpy.
• Using this model you built to train a regression model for a dataset you select. Use https://www.kaggle.com/datasets to find a dataset and a question you want to answer!
• Then using your model, and some data cleaning answer your question.
• Follow all the concepts you learned in class (and maybe even more) to solidify what we learned!

In the second week, we move towards understanding Classification. We will start by covering the foundational elements of classification problems, including decision boundaries and the difference between binary and multiclass classification. We will then take a deep dive into Logistic Regression, exploring its mathematical foundations and practical applications.

The week will be rich with hands-on exercises to solidify these concepts, culminating in a homework assignment that challenges students to explore a new classification algorithm.

Objective: Apply logistic regression to a multi-variable dataset to deepen understanding of feature handling, model implementation, and evaluation.

Tasks:

1. Feature Preprocessing: Discuss and implement preprocessing steps including feature scaling and handling categorical variables.
2. Logistic Regression Implementation: Complete guided lab notebook code for logistic regression and gradient descent using vectorized implementations with NumPy.
3. Training and Evaluation: Train the model on the training set and evaluate its performance on the test set using accuracy and at least one other metric (e.g., F1 score).
4. Visualization: Visualize the decision boundary and model predictions on the test set to understand the model's performance. Also;
   • Plot the ROC curve and calculate the AUC score.
   • Investigate the learning curve to understand model performance.
5. Compare with Scikit-learn: Implement logistic regression using Scikit-learn and compare the results with your implementation.

Objective: We will implement and evaluate a classification algorithm not covered in detail in class, such as k-Nearest Neighbors (k-NN) or a Decision Tree, on a multi-variable dataset. This assignment aims to deepen your understanding of classification algorithms and their practical applications along with research and reflection on the learning process.

Tasks:

1. Research and Implementation:

   • Select a classification algorithm not covered in the lecture (e.g., SVM) and research its working principles, advantages, and limitations.
   • Implement the selected algorithm to fit it on the same dataset used in the lab, following a similar structure to the logistic regression implementation. You should implement
   • Train the model on the training set and evaluate its performance on the test set using accuracy and at least one other metric.
   • Visualize the decision boundary of the model on the dataset.

2. Comparison and Reflection:

   • Compare the performance of your model to the logistic regression model from the lab assignment. This comparison should include aspects like accuracy, computational efficiency, and ease of interpretation.
   • Write a short reflection of your learning experience, discussing the challenges you faced, how the chosen algorithm differs from logistic regression, and in what scenarios would you prefer one over the other.

3. Advanced Exploration (Optional):

   • Here are some optional tasks to explore advanced features of your chosen algorithm;
     • Implement a hyperparameter tuning strategy (e.g., grid search or random search) to optimize the model's performance.
     • Investigate the impact of different hyperparameters on the model's performance and visualize the results.
     • Research and implement a more complex variant of the algorithm (e.g., kernel SVM) and compare its performance to the basic implementation.
     • Try a different dataset and compare the performance of the two algorithms on this new dataset (you can use a dataset from sklearn.datasets or kaggle.com).

In the third week, we move towards understanding how we approach Machine Learning when we are not exposed to the answers of the data (labels). Like the previous weeks we start with the basics of Unsupervised Learning, focusing on Clustering, and then move on to a deep dive into K-Means Clustering - one of the most popular clustering algorithms.

Later if we got time we will also discuss the concept of Dimensionality Reduction and how it can be used to reduce the complexity of the data.

I wanted to keep this week shorter and try having mostly learn + apply style of lecture. We will have a section and then students will be sent to rooms/groups to complete some tasks (coding for most parts).

TBD

TBD

We welcome contributions to the Ground-Up Machine Learning (GML) curriculum! Whether you're interested in adding new content, suggesting improvements, or fixing bugs, your input is valuable in making this an even better resource for everyone.

To contribute, please follow these steps:

1. Fork the Repository: Start by forking the GML repository to your own GitHub account.
2. Make Your Changes: Whether it's adding new materials, correcting typos, or suggesting enhancements, make your changes in your forked version.
3. Submit a Pull Request: Once you're happy with your updates, submit a pull request back to the main GML repository. Please provide a clear description of your changes and the reasons for them.
4. Review Process: Your pull request will be reviewed by me or the maintainers of the repo. We may engage with you for discussions or request modifications before merging your contributions.

For significant changes or new content, we recommend opening an issue to discuss your ideas with us before proceeding. This collaborative approach ensures that we maintain the integrity and coherence of the curriculum while incorporating the community's valuable insights.

The Ground-Up Machine Learning (GML) course is made available under the GNU General Public License v3.0. You are free to use, share, and modify the course materials for educational purposes, provided you adhere to the terms of the license.

Please review the full license for more details. This open license is part of our commitment to supporting and contributing to the open-source community, making learning accessible to as many people as possible.