Giter Club home page Giter Club logo

knowledge_extraction's Introduction

Exercise 8: Explainable AI and Knowledge Extraction

Overview

The goal of this exercise is to learn how to probe what a pre-trained classifier has learned about the data it was trained on.

We will be working with a simple example which is a fun derivation on the MNIST dataset that you will have seen in previous exercises in this course. Unlike regular MNIST, our dataset is classified not by number, but by color! The question is... which colors fall within which class?

CMNIST

In this exercise, we will return to conventional, gradient-based attribution methods to see what they can tell us about what the classifier knows. We will see that, even for such a simple problem, there is some information that these methods do not give us.

We will then train a generative adversarial network, or GAN, to try to create counterfactual images. These images are modifications of the originals, which are able to fool the classifier into thinking they come from a different class!. We will evaluate this GAN using our classifier; Is it really able to change an image's class in a meaningful way?

Finally, we will combine the two methods — attribution and counterfactual — to get a full explanation of what exactly it is that the classifier is doing. We will likely learn whether it can teach us anything, and whether we should trust it!

Setup

Before anything else, in the super-repository called DL-MBL-2024:

git pull
git submodule update --init 08_knowledge_extraction

Then, if you have any other exercises still running, please save your progress and shut down those kernels. This is a GPU-hungry exercise so you're going to need all the GPU memory you can get.

Next, run the setup script. It might take a few minutes.

cd 08_knowledge_extraction
bash setup.sh

This will:

  • Create a conda environment for this exercise
  • Download the data and train the classifier we're learning about Feel free to have a look at the setup.sh script to see the details.

Next, open the exercise notebook!

Acknowledgments

This notebook was written by Diane Adjavon, from a previous version written by Jan Funke and modified by Tri Nguyen, using code from Nils Eckstein.

knowledge_extraction's People

Contributors

adjavon avatar neptunes5thmoon avatar trivoldus28 avatar afoix avatar ben-salmon avatar funkey avatar

Stargazers

Alán F. Muñoz avatar Zachary Burns avatar Manan Lalit avatar Jeff Rhoades avatar

Forkers

leanhphuong201

knowledge_extraction's Issues

Ideas for 2025

  • Make a new figure for the StarGAN (to be exactly like what we're doing)
  • Have a pre-trained stable network ready (not sure whether to make it explicit in the notebook or just give it after a checkpoint)
  • Have a "data visualization" at the end of the notebook, that shows that for this task it is possible to figure out what the classes are by just looking at enough images, sorting them in interesting ways, etc.

Alex Comments

  • In cell 2 (create a balanced sampler): I think that you have the solution copied in the exercise (or do you intend for there to be nothing to change for this cell. The line "Below define a sampler" madde me think that there was something I had to do.
  • Could be useful to include an illustration of the VGG network (https://paperswithcode.com/method/vgg), they've seen it before in florians talk
  • in cell 12 (task 1.2): you define a variable as "ground_truths" then call "ground_truth" on the last line, producing an error. Also "prediction" and "predictions"
  • also in cell 12: the solution is still present in exercises

ran out of GPU memory, will try to finish the notebook later tonight

Updates needed for 2023

  • Standardize whether to use “Task” or “Exercise”
  • Remove training of the VGG classifier. Instead load the model and do evaluation (and use the extra space to give more details). Alternatively, give them a link to a script to train their own model if they choose to.
  • For Exercise 1.4, no good reason to have two methods here, just do one
  • Make questions tasks: have them discuss in Element/Matrix
  • Exercise 2: typo in attributions
  • Goes from 2.1 to 2.3
  • Exercise 2.1: remove hints! (too obvious) “read the docs”, hint to number of lines (2)
  • Part 3: add more links to everything in the explanation, e.g. the lecture slides (CycleGAN, counterfactual explanation)
  • remove exercise 3.1: It is more about data wrangling than about actual training. We could make this available to them.
  • Exercise 3.2: Options good! Add the numbers from the top to the choices in the code
  • Checkpoint 3: Add learning objectives
  • Memory errors is a problem. Perhaps we should just have a smaller dataset, or smaller models.
  • Remove exercise 4.2
  • Move the TODOs inside the blue boxes.
  • Part 4 is all called 5 for some reason

Remove VGG training

Assumes that the VGG that we need has been trained in the Failure modes exercise.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.