yugeten / fish Goto Github PK

There is a filter of the indices that selects only latents[:, 0] != 3.

In the code above, this condition is applied on val and test, but not on train. This results in an error in training with ERM. Could you provide the correct way to perform ERM on cdsprites?

Thanks.

Hi fish authors,

Thanks for the nice work!

However, when I was trying to reproduce results in civilcomments using the current code, it seems there was a bug regarding sample_domains in train_fish. Specifically, a RuntimeError was encountered in get_batch and it's said stack expects a non-empty TensorList. After looking into the sampled domains, it seems some domains with 1 batch left were sampled while there was no batch_index actually.

Could you help look into this issue? Thank you very much.

Best, Andrew

Hi Fish authors,

Thanks for the nice work! When I was trying to reproduce the results of Fish in FMoW-wilds, I found the function save_best_model (code) always saves the last epoch model. It seems Fish would use the last epoch model selection instead of selecting the models according to the validation performance, which appears to be different from the wilds evaluation protocol (I also failed to find a corresponding description in the paper).

Maybe I missed something, could you help look into this problem? Thank you very much 😄.

Best, Andrew

Hi, I am a reader of your paper~ And I'm so intrigued by your paper, it inspired me a lot. But I have an question about how to comprehend theorem 3.1.

I think this is an excellent proof, but I have an question about how to comprehend this:
if alpha(inner loop learning rate) is 0 instead of approaching 0, this limit will not be 1, it will be 0.
So when alpha approaches 0, this limit is 1. But when alpha is exactly 0, this limit is 0. How to comprehend this kind of "discontinuity"? Which step in this proof cause this "discontinuity?"

Congratulations for this really interesting work.
I was wondering whether you could provide the DomainBed results, but with the best hyper-parameter chosen on a validation dataset from the test domain (i.e the oracle model selection).
That would be of great help to include your paper as a new comparable approach in upcoming papers.
Best regards
Alexandre

Thanks for your implementation of fish, but when I test poverty with algorithm erm, the worst r is just about 0.2, while the performance noted in wilds is about 0.5 (https://wilds.stanford.edu/leaderboard/), is there some problem?

Hi, I tried running your code on domainbed and got only 55% average accuracy on the officehome dataset. You reported 58% in your paper, I was wondering if you tested the performance of the open source code on this dataset. Is there something wrong with this code?