idess / how-to-learn-deep-learning Goto Github PK

A practical, top-down approach, starting with high-level frameworks with a focus on Deep Learning.

There are three main goals to get up to speed with deep learning:

1. Get familiar to the tools you will be working with, e.g. Python, the command line and Jupyter notebooks
2. Get used to the workflow, everything from finding the data to deploying a trained model
3. Building a deep learning mindset, an intuition for how deep learning models behave and how to improve them

• Spend a week on codecademy.com and learn the python syntax, command line and git. If you don't have any previous programming experience, it's good to spend a few months learning how to program. Otherwise, it's easy to become overwhelmed.
• Spend one to two weeks using Pandas and Scikit-learn on Kaggle problems using Jupyter Notebook, e.g. Titanic, House prices, and Iris. This gives you an overview of the machine learning mindset and workflow.
• Spend one month implementing models on cloud GPUs. Start with FastAI and PyTorch. The FastAI community is the go-to place for people wanting to apply deep learning and share the state of the art techniques.

Once you have done this, you will know how to add value with ML.

Your portfolio is a gateway to new opportunities, so focus on things you have an interest in. It's also an opportunity to explore different areas in machine learning. If you're not sure where to start, I'd start with a computer vision project and a language-related project (NLP). Here's a Twitter thread on how to come up with portfolio ideas, and here's another thread on how to write about them.

A common mistake is to collect online course certificates instead of building projects. Employers tend to not value course certificates because they don't trust the rigor of mass accreditation. Instead, the internet-educated need tangible evidence of their knowledge:

• Publishing paper at good conferences
• Winning ML competitions
• Pull requests to established open source projects
• 50-100K readers on ML related topics
• ML tools with +100 weekly users
• ML gig/job
• ML art

If you are not sure of a good first project or where to begin, direct message people on Twitter. Find people on Twitter whos work inspire you, tell them what you have done so far and what you hope to achieve. Write a personal message to 50-100 people - you'll be surprised at how helpful people are. Just make sure you have done your homework and read up on best practices for cold outreach.

Learning how to read papers is a great source to improve your projects, and critical if you want to get into research. There are three key areas to feel comfortable reading papers:

1. Understanding the details of the most frequent algorithms, gradient descent, linear regression, and MLPs, etc
2. Learning how to translate the 50 most frequent math notations
3. Learn the basics of algebra, calculus, statistics, and machine learning

• Spend one month recoding the core concepts in python numpy, including least squares, gradient descent, linear regression, and a vanilla neural network. Andrew Trask's book and blog are great for this. This will help you reduce a lot of cognitive load down the line.
• I believe the best deep learning theory curriculum is the Deep Learning Book by Ian Goodfellow and Yoshua Bengio and Aaron Courville. I use it as a curriculum, and the use the internet to learn the details about each concept. Although, it's good to have six months of practical experience or a math/statistic background before you start with the theory.
• Most of the value comes from memorizing the math notations on the first few pages. Learning that notations are compact logic and how to translate it into code will make you feel less anxious about the theory. If you have a week spare, I'd spend it on 3Blue1Brown's Essence of linear algebra, the Essence of Calculus, and StatQuests' the Basics (of statistics) and Machine Learning. Use a spaced repetition app like Anki and memorize all the key concepts. Use images as much as possible, they are easier to memorize.
• If you have 2-3 months, spend it on part 1 of the Deep learning book. The MachineLearningGod shows a great way to study the book. Use lectures and videos to understand the concepts, and then use Anki and Khan academy exercises to learn them.

[WIP]

• FastAI
• Keras Slack
• Distill Slack
• Pytorch
• Twitter

• Implementing Keras models (I started with TFlearn, but I'd highly recommend using Keras instead) (I mistakenly thought TFlearn was the official frontend. They have now made Keras the official front-end to Tensorflow.)
• Recoding the core concepts in python
• My first paper using CNN
• My first paper using LSTM
• My first paper using a GAN
• My first paper using RL/evolution
• On going via my twitter feed

• S. Zayd Enam
• Catherine Olsson
• Greg Brockman V2
• Greg Brockman V1
• Andrew Ng
• Amid Fish
• Spinning Up by OpenAI
• Confession as an AI researcher
• YC Threads: One and Two

If you have suggestions/questions create an issue or ping me on Twitter.