Any successful prediction model may be driven by a confounding effect that is correlated with the effect of interest. It is important to control that detected associations are not driven by unwanted effects. It is common issue in in neuroscience, epidemiology, economy, agriculture, etc.

We introduce a non-parametric approach, named “confound-isolating cross-validation”, to control for a confounding effect in a predictive model. It is based on crafting a test set on which the effect of interest is independent from the confounding effect.

Developed framework is based on anti mutual information sampling, a novel sampling approach to create a test set in which the effect of interest is independent from the confounding effect. The grafical illustration of classic and confound-isolating cross-validation:

You provide us

Variables:  

• X - data with shape (n_samples, n_features)
• y - target vector with shape (n_samples)
• z - confound vector with shape (n_samples)

Optional parameters

• min_sample_size - minimum sample size to be reached, default is 10% of the data
• n_remove - number of the samples to be removed on each iteration of sampling, default is 4
• prng - control the pseudo random number generator, default is None
• cv_folds - number of folders in the cross validation, default is 10

We return you

Variables:

• x_test, x_train, y_test, y_train, ids_test, ids_train - train and test of X, y and sampled indexes

confound_prediction package requires:

• Python (>= 3.5)
• Scipy (>=1.1.0)
• Scikit-learn (>=0.21.2)
• Numpy (>=1.14.2)

• Matplotlib (>=2.2.2) for example visualization

pip install TBD

python example/Example_get_testset.py

python example/

python example/

python example/Example_compare_mutual_info_correlation.py

We also provide 2 state of the art in neuroscience deconfounding techniques:

  1. Out-of-sample deconfounding

  2. Deconfounding test and train jointly

[1] TBD

[2] D. Chyzhyk, G. Varoquaux, B. Thirion and M. Milham, "Controlling a confound in predictive models with a test set minimizing its effect," 2018 International Workshop on Pattern Recognition in Neuroimaging (PRNI), Singapore, 2018, pp. 1-4. doi: 10.1109/PRNI.2018.8423961 PDF