PyTorch style DNN Library

• Grokking Deep Learning: Chapter 13
• Chapter 5.5: PyTorch’s autograd: Backpropagating all things
• Deep Learning With Pytorch Basics Of Autograd In Pytorch
• IN5400: Lecture 4: Introduction to Pytorch
• PyTorch Autograd Explained - In-depth Tutorial

• Thermometer application from pytorch book
• tests for individual operations
• refactor out next function
  • add does not need to save context
• retain_graph=True after first backward
• Unbind - to bring in action the 2nd parameter of the tuple.
• complex example from youtube video
• Enable graph to be GCed
• E2E
  • MNIST
  • RNN ? numerical gradient