Exploring techniques to make models forget information

Read at : https://joe-arul.github.io/machine_unlearning/

The field of Machine Unlearning specifically deals with removing the influence of data points from a model without having to retrain it from scratch – in other words, making a model forget certain information. The concept struck my interest as the process of deep learning is very similar to human learning – can model forgetting be related to how the human brain forgets information as well? – we all have memories we want to remember, memories we want to forget and maybe we can help the model too!

This project served as an exercise for me to learn about machine unlearning concepts, read papers on prior research, and figure out the best way to solve the problem at hand using existing methods. While I did intend to produce a new unlearning methodology initially when I undertook this project for class, the ideas presented in the scientific papers that I read have been formulated over years of research in the field, and I felt that it would be infeasible for me to learn and reproduce such an effort for a class project. Therefore, in this project report, I hope to add value to the reader by explaining the approach I took to solve an unlearning problem. Hope you enjoy the read 😊

Notebooks:

• Sunshine V1: Fine tuning
• Sunshine V2: Negative Gradient
• Sunshine V3: Selective Forgetting