A library to assess the effectiveness of XAI methods through ablation.

Explainable artificial intelligence (XAI) methods lack ground truth. In its place, method developers have relied on axioms to determine desirable properties for their explanations behavior. For high stakes uses of machine learning that require explainability, it is not sufficient to rely on axioms, as the implementation, or its usage, can fail to live up to the ideal. A procedure frequently used to assess their utility, and to some extent their fidelity, is an ablation study. By perturbing the input variables in rank order of importance, the goal is to assess the sensitivity of the model's performance.

This implementation can be used to reproduce the experiments in BASED-XAI: Breaking Ablation Studies Down for Explainable Artificial Intelligence.

Pip

conda create -n ablation python=3.9 --yes
source activate ablation
pip install ablation

If you would like to contribute, please download the code from master to ensure any contributions are made towards vulnerability free code.

Install the development dependencies:

pip install -e ."[dev]"

Install pre-commit hooks:

pre-commit install
pre-commit run

Configure experiment yaml file:

dataset_name: synthetic_cat             # dataset name
model_type: nn                          # model type (nn/linear)
path: synthetic_cat                     # experiment path
load: false                             # load experiment from path
rerun_ablation: false                # rerun ablation if loaded

n_trials: 4                             # number of trials
dataset_sample_perc: 1                  # fraction of dataset to subsample
dataset_n_random_features: 4            # random features to add

explanation_methods:                    # explanation methods
  - random
  - deep_shap
  - kernel_shap
perturbation_config:                    # perturbation methods and dict of params
  constant_median: {}

baseline_config:                        # baseline methods and dict of params
  constant_median: {}
  training:
    nsamples: 50


ablation_args:                          # ablation arguments
  scoring_methods: [log_loss, auroc]    # performance metric
  local: true                           # local vs global explanations

Run experiment

from ablation.experiment import Config, Experiment

config = Config.from_yaml_file("config.yml")
exp = Experiment(config)
results = exp.run_exp()

Experiments for the paper used pytorch_lightning==1.5.7. Due to vulnerabilities, we needed to update to a more recent version. In order to accurately reproduce the experiments in the paper, install release v0.1.0 and revert to the previous version of pytorch lightning:

pip install ablation==0.1.0
pip install pytorch_lightning==1.5.7

To replicate KDD experiments for all real datasets run the following

cd kdd_configs
python run_exp.py

To replicate KDD experiments for synthetic datasets run the following

cd kdd_configs
python run_exp_synthetic.py

To run tests:

python -m pytest tests/unittests

If you use ablation in your work, please cite our paper:

@inproceedings{hameed:basedxai:2022,
  author    = {Hameed, Isha and Sharpe, Samuel and Barcklow, Daniel and Au-Yeung, Justin and Verma, Sahil and Huang, Jocelyn and Barr, Brian and Bruss, C. Bayan},
  title     = {{BASED-XAI: Breaking Ablation Studies Down for Explainable Artificial Intelligence}},
  year      = {2022},
  maintitle = {ACM SIGKDD International Conference on Knowledge Discovery and Data Mining},
  booktitle = {Workshop on Machine Learning in Finance},
}

We welcome Your interest in Capital One’s Open Source Projects (the “Project”). Any Contributor to the Project must accept and sign an Agreement indicating agreement to the license terms below. Except for the license granted in this Agreement to Capital One and to recipients of software distributed by Capital One, You reserve all right, title, and interest in and to Your Contributions; this Agreement does not impact Your rights to use Your own Contributions for any other purpose.

Sign the Individual Agreement

Sign the Corporate Agreement

This project adheres to the Open Code of Conduct By participating, you are expected to honor this code.