darya-chyzhyk / confound_prediction Goto Github PK

Make confound_isolating_cv compatible with Scikit Learn cross validation

Hi Darya Chyzhyk,

Is it possible to use your package to control for categorical variables?
(I have 10-20 categories)...

If so, any tips on how to do so?
Thanks!

print doc at the beginning of the example

I would show the samples in a scatter plot to make it clear that you are sampling points from a dataset.

Less iterations and a smaller dataset to reduce computation time...

"Confound_prediction is a Python module to control confound effect in the prediction or classification model." -> "Confound_prediction is a Python module to control confound effect in prediction or classification models."

3. Deconfounding test and train jointly (which should not be used, and is provided only for illustration)
   -> I find it dangerous to introduce a model while forbiding it. At least, Iwould not- mention it in the top of the documentation

"Developed framework is based" -> "The developed framework is based"

"You provide us" -> "You provide"
"We return you" -> "The function returns"

Hello,
Thank you very much for tackling this issue of confounders, which seems very recurrent in clinical ML problems.

I have some questions about the project/paper:

1. I am wondering why only the test set needs to be Deconfounded? Why not build also a train set which is Deconfounded and a Deconfounded test set (with no data leakage of course)?
2. I tried to make a generalization of your methodology with k multiple confounders
   
   I still used most of your codebase and I used a pseudo generalization of the mutual information of multiple variables.
   The probability to be sampled m_i which was
   

is now:

The quantity
can still be estimated with kernel density estimation.

I made some quick toy examples, it seems to approximately work on simple additive toy examples and when the number of example is sufficient:
For instance with 1000 sample and 10 confounding factors i got:

For instance with 100 sample and 3 confounding factors i got:

It would be also interesting to study the required N to be sure at a certain level the deconfounding capability for k factors considering the type of link.

Do you think this is a correct approach and generalization?

Thank you

Best regards

I'd rather put the simulated_data in the library, e.g. in a '_utils.py' module, otherwise, people will not understand what this stands for.

I find it very important that examples are visual.