omarsf / deep-learning-for-professionals Goto Github PK

• Logistics
• Course Description
• Class Structure
• Grades
• Course Policies
• Course Schedule

Instructor: Dr. Jared Thompson [email protected]
Office Hours: By Appointment

Class Location: 44 Tehama St, San Francisco, CA
Lecture Times/Days: 6-7:20p, Monday & Wednesday
Lab Times/Days: 7:30-9p, Monday & Tuesday

This course provides the necessary mathematical, conceptual and technological foundations to practically apply modern neural networks (colloquially called "Deep Learning") to everyday machine learning problems. Although topics are presented in an accessible way, the course is angled towards professionals with a reasonable background in programming and some mathematics background as well.

The class will focus on practical development of fundamental skills, and is intended as a point of departure for professionals interested in moving into machine learning applications germane to many modern embedded technologies such as drones, intelligent systems (e.g. smart homes) and self-driving cars. This area is exciting due to the many possibilities it affords and the relatively low barriers to entry.

Dependent on class performance, we may cover advanced topics such as Neural Turing Machines and Deep-Q networks in the last week.

• Apply Deep Learning to solve real world problems
• Explain and implement backpropagation algorithm
• Build and optimize the following architectures:
  • Multilayer Perceptrons (MLP)
  • Convolutional Neural Networks (CNN)
  • Autoencoders
  • Recurrent Neural Network (RNN)
  • Long Short Term Memory (LSTM)
• Have the resources and experience to be able to progress on your own

• All other kinds of machine learning (We are only covering Deep Learning)
• All other Deep Learning frameworks (We are only covering Keras and TensorFlow)
  • Theano, Caffe, CNTK, DSSTNE, PaddlePaddle, …
  • OpenAI Gym/Universe & DeepMind Lab
  • High Performance Computing (HPC)
  • Distributed systems
• ALL other hardware implementations (We are only covering CPU and GPUs)
  • ASIC
  • Mobile

The student should have the equivalent of 2 years programming experience in Python and basic background in mathematics through 3 semesters of calculus. Strength in vector mathematics will help significantly as will ability to think visually.

• Fundamentals of Deep Learning
• Deep Learning

This course is an "active" learning environment. You'll learn through doing. The focus will be on explaining concepts in your words and applying concepts through programming.

Before class you will complete preparation materials. All preparation materials should be covered prior to the start of each class session. They are always required unless explicitly labeled as optional. These materials will be the source of factual knowledge. You are expected to be familiar with the basic concepts and technical jargon before the start of class.

In-class time is precious - We'll reserve it for discussion, presenting complex material, answering questions, and working on exercises.

Typical class structure:

1. Class Review / discussion of OYO activity
2. Lecture
3. Lab

OYO activity is a creative activity to help you to integrate and apply the the preparation materials. These are to be completed before class begins.

Students will separate into two (or threes) for "pair programming". These exercises may involve a series of short questions, single day projects, or multi-day projects.

We will not be assigning grades, however solution sets will be handed out regularly. You are responsible for assessment of your own performance. The instructor can offer feedback as well.

You must also show up prepared. Each person is important to the dynamic of the class, and therefore students are required to participate in class activities. Expect to be "cold called". I call on students at random, not to put you on the spot but, to keep you engaged in the material at all times.

Attendance is mandatory. You cannot get the benefit of the material simply by reading it. It is the responsibility of each student to attend all classes. If you have to miss class, due to any circumstances, please notify Jared by email ASAP.

Below is a tentative course schedule. The instructor reserves the right to change the schedule as necessary for maximizing learning outcomes.

1. Introduction to Neural Networks
   1. Intro to Machine Learning / Neural Network Fundamentals
   2. Training a PLR / Backpropagation
2. Tensorflow is your friend
   1. Computational Graphs/ Backpropagation
   2. Backpropagation / Tensorflow and Keras
3. Convolutional Neural Networks
   1. Tensorflow and Keras / Fundamentals of CNN
   2. Build and train a CNN / CNN Applications
4. CNNs and Filters
   1. Image preprocessing / Intro to Pipelines
   2. Visualizing the learning process / Convolutional Filters for Style Representations
5. Representations and Embedding
   1. Dimensionality Reduction / Introduction to Autoencoders / Build a Visual Autoencoder
   2. Embeddings / Denoising
6. NLP
   1. Intro to NLP / Skip-Gram Model
   2. Applications of NN to NLP / Word2Vec
7. Sequential Analysis
   1. Intro to Recursive NNs / Character-level generative text models
   2. Intro to LSTMs / Sentiment analysis
8. Advanced Topics
   1. Intro to Generative Adversarial Nets / GAN clustering
   2. Intro to Neural Turing Machines / Review