Giter Club home page Giter Club logo

conceptbottleneck's Introduction

Concept Bottleneck Models

teaser

This repository contains code and scripts for the following paper:

Concept Bottleneck Models

Pang Wei Koh*, Thao Nguyen*, Yew Siang Tang*, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang

ICML 2020

The experiments use the following datasets:

To download the TravelingBirds dataset, which we use to test robustness to background shifts, please download the CUB_fixed folder from this CodaLab bundle by clicking on the download button. If you use this dataset, please also cite the original CUB and Places datasets.

The NIH Osteoarthritis Initiative (OAI) dataset requires an application for data access, so we are unable to provide the raw data here. To access that data, please first obtain data access permission from the Osteoarthritis Initiative, and then refer to this Github repository for data processing code. If you use it, please cite the Pierson et al. paper corresponding to that repository as well.

Here, we focus on scripts replicating our results on CUB, which is public. We provide an executable, Dockerized version of those experiments on CodaLab.

Abstract

We seek to learn models that we can interact with using high-level concepts: would the model predict severe arthritis if it thinks there is a bone spur in the x-ray? State-of-the-art models today do not typically support the manipulation of concepts like "the existence of bone spurs", as they are trained end-to-end to go directly from raw input (e.g., pixels) to output (e.g., arthritis severity). We revisit the classic idea of first predicting concepts that are provided at training time, and then using these concepts to predict the label. By construction, we can intervene on these concept bottleneck models by editing their predicted concept values and propagating these changes to the final prediction. On x-ray grading and bird identification, concept bottleneck models achieve competitive accuracy with standard end-to-end models, while enabling interpretation in terms of high-level clinical concepts ("bone spurs") or bird attributes ("wing color"). These models also allow for richer human-model interaction: accuracy improves significantly if we can correct model mistakes on concepts at test time.

teaser

Prerequisites

We used the same environment as Codalab's default gpu setting, please run pip install -r requirements.txt. Main packages are:

  • matplotlib 3.1.1
  • numpy 1.17.1
  • pandas 0.25.1
  • Pillow 6.2.2
  • scipy 1.3.1
  • scikit-learn 0.21.3
  • torch 1.1.0
  • torchvision 0.4.0

Note that we updated Pillow and removed tensorflow-gpu and tensorboard from requirements.txt.

Docker

You can pull the Docker image directly from Docker Hub.

docker pull codalab/default-gpu

Usage

Standard task training for CUB can be ran using the scripts/experiments.sh and Codalab scripts can be ran using scripts/codalab_experiments.sh. More information about how to perform data processing and other evaluations can be found in the README in CUB/.

conceptbottleneck's People

Contributors

kohpangwei avatar thaonguyen19 avatar yewsiang avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.