ykwon0407 / uq_bnn Goto Github PK

Dear Mr. Kwon,

we enjoyed reading your papers on decomposing predictive variance into aleatoric and epistemic uncertainty in classification settings without the need of an extra output layer. Thank you also for sharing your code online.

After reading your derivation to decomposing the predictive variance as given in Appendix A in your paper "Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation", we had difficulties to understand the step from the second to last line to the next. I'm pasting the the two lines over here:

Can you please clarify why the outer product (denoted as $\otimes 2$) can be moved outside of the difference? I think sum and outer products cannot be interchanged as demonstrated by the example below, but I might be missing some trick or assumption here.

Thank you for your help!

Hi,
According to your code, p_hat is the matrix of all test data's probability vector after softmax, e.g.
[[0.3, 0.2, 0.5],
[0.1, 0.8, 0.1],
[0.4, 0.2, 0.4],
[0.7, 0.1, 0.2]] (assumed that there are 3 classes, two test data and the number of stochastic dropout is 2), and aleatoric and epistemic uncertainty are calculated according to p_hat, am I correct?
Thanks!

Why do you only use [0] of prediction results in line 63, /retina/utils.py? Shouldn't p_hat be made up of the probabilities of all classes?

Could you please let me know whether the eq. 4 in the paper is applicable for multi label segmentation or just the binary segmentation?

Originally posted by @redsadaf in #1 (comment)

Hi @ykwon0407 ,
Thank you for your interesting work!

I am working on your paper and Kendall's paper as a reference.
I was wondering how your suggestion and Kendall's suggestion were implemented, so I looked at the code. I have several questions about the implementation of Kendall's work.

In Kendall's paper, they proposed loss function for estimating uncertainty in classification task like below:

I found creating logit vector x in SampleNormal, but I'm curious about the implementation of loss function like below:

Can you explain K.log(1.+K.exp(linear_predictor_f))) ?
And you used log_variance and exponential function when estimating variance. Is there any reason of that?
In addition, Is there any inference code for kendall's work?

Thank you in advance!

Hi,
In your code, you use MC dropout when inference a new input's outputs. But recently I read another paper Bayesian Convolutional Neural Networks with Variational Inference, the author uses local reparameterization trick for convolutional layer to sample in inference (see line 132-146 in https://github.com/felix-laumann/Bayesian_CNN/blob/master/utils/BBBlayers.py), i.e. output = mean + std * (random(mean)), where (I think) w is drawn from N(mean, std). I can't tell which method is better for sampling, if we can have a discussion about this, I'll be very very thankful!! (By the way, he also introduces aleatoric and epistemic uncertainty in his work:) )

Hi, can I rewrite your equations to get aleatoric uncertainty and epistemic uncertainty of each CLASS of the model, other than a single sample? I think it may show that the model is not good at some class or in some class, the model doesn't get enough good data.

Hi, in your appendix,

Why is y* one-hot encoded? Should y* be the probabilities of all classes? (e.g., there are 3 classes, then y* may be [0.1, 0.6, 0.3])

If it is a label, how to get it by average T times samples?

Thanks

Dear author,
thank you for your exciting work.
I cannot understand the meaning and the function of the operator ⊗ in eq (4) of your paper https://openreview.net/pdf?id=Sk_P2Q9sG&source=post_page
Could you please clarify it for me?
Thanks

Francesco

Dear Yongchan,

I've been reading your paper and code with great interest. Seems like a very interesting way to assess predictive uncertainty in DNN models for segmentation, based purely on inference-time Dropout. However, I can't seem to find the code in which you do inference and actually compute the aleatoric and epistemic uncertainties.

Specifically, the implementation of the key eq. 4 in the paper seems to be missing. Could you provide this or point me to it?

Thanks for this interesting work!

Uncertainty Model 관련한 논문 도움 많이 되었습니다 감사합니다 :)

Thank you for your sharing your code,
I have some questions.
In your paper, appendix A, For categorical variable y∗, Varp(y∗|x∗,ω)(y∗) = Ep(y∗|x∗,ω)(y∗⊗2)−Ep(y∗|x∗,ω)(y∗)⊗2 and it is Ep(y∗|x∗,ω){diag(y∗)}−Ep(y∗|x∗,ω)(y∗)⊗2 because y∗ is one-hot encoded.

However,
You known， the output of softmax is not one-hot encoded variable, each position is the probability of the classes , for example,
the output is [0.9,0.1,0.1](not one-hot), the ground trouth is 1,0,0, so do we really can use the "it is Ep(y∗|x∗,ω){diag(y∗)}−Ep(y∗|x∗,ω)(y∗)⊗2 because y∗ is one-hot encoded. "?

Feng