zhkkke / dualstudent Goto Github PK

Hi and thanks for your interesting work!

I went through you code and I am just wondering how to initialize 2 students(set different seed?)?

Thanks in advance,

best regards,

Hi !
I like your idea and I want to use your code as a baseline to study SSL.
However, I wonder that the scale of consistency losses.
You set the weights to 1:10:100.
(Plus, I've also found that 'mean teacher' which is your baseline, set the consistency weight to 100. )
I think these kind of values are not general since people usually set weight equally.

In my opinion, small weight for 'supervised loss' can help prevent from overfitting but it seems that it is too immoderate to choose such 1:100.

I'd like to ask why you set these parameters as such values.

Thanks !

I would like to know that How to understand
" the outputs of these two models may vary widely, and applying the consistency constraint directly will cause them to collapse into each other by exchanging the wrong knowledge"

Here "cause them to collapse into each other" means what?

Does it Mean that imposing consistency on the output of two networks will cause the two networks to converge the same? Then why doesn't EMA have this problem.
I don't understand it very well.

Hello, your paper is indeed exciting and insightful, while after I finished reading, I still have some questions unsolved.
I will appreciate it if you could spend some time answering the following:

1. You proved that the teacher and the student converge to the same point (eventually), but why this convergence is bad anyway, and why can't they converge to a rather good solution?

2. Secondly, I wonder why the idea of "stable samples" works. Sometimes I think we don't care much about them: for example in active learning people tend to know those near the decision boundary, which are "unstable samples", because they carry more information than stable ones. In your paper, with all due respect and let me point it out, you speculated that "applying the consistency constraint directly will cause them to collapse into each other by exchanging the wrong knowledge", which is not supported by any evidence, and I wonder why the proposed "stable points", on which the consistency constraint is applied, could help solve the collapsing problem you mentioned. Do you have any other derivation on that?

3. Finally, you spent rather short term of discourse to do the ablation study and I'm curious whether there is numerical performance when you not "training with stable samples"?

Thanks for your time again! In general I think the overall idea is very insightful!

Hi and thanks for your interesting work!

I went through you code and Iam just wondering why the CNN13 model has two outputs?

Thanks in advance,

best regards,

M

Thx for your marvelous work!
I am trying to use your method as my baseline, but I find that if I set the epoch larger than 300(which is set in your script originally), after the 200 epoch, the loss value will be NAN.
I cannt figure it out what's wrong, do you have any idea about that?
Thx!

Hi, I'm trying to run this code in colab with the pytorch 1.9 and cuda 102, but need your help in fine tuning the Mean-teacher part of the code.

Hi I was looking through the code but I couldn't figure out whether l_input and r_input belong to the same image but augmented differently (different noise applied). Can you clarify? Thanks in advance!