InfoNCE_Loss skipping about greedy_infomax HOT 2 CLOSED

Thank you for submitting an issue. I looked into it, and I believe the current implementation is correct. Let me know if you feel I misunderstood your point.

The idea is that you want to predict k={1,...,K} steps into the future. Now, for skipping the first k, we add skip_step=1 to each k, resulting in k={2,...,K+1} (and thus skipping k=1). As far as I understand, your proposed solution would result in k={2,2,...,K} instead - this will lead to a smaller loss during training (since predicting smaller k's is easier), but will probably harm downstream performance.

from greedy_infomax.

Sorry, everything you have said is correct, accidental miscomprehension on my part.

Although it is curious that for the task I applied it on accidentally doing k={2,2,...,K} increased downstream classification performance. Possibly predictions too far away detract from performance, or increased weighting for closer predictions increases downstream performance. Regardless, there is no issue.

from greedy_infomax.