Did the proxys are random sampled? about proxy-nca HOT 7 CLOSED

In your code:
self.proxies = nn.Parameter(torch.randn(nb_classes, sz_embed))
torch.nn.init.xavier_uniform(self.proxies)

from proxy-nca.

Yes, the weight parameters are sampled from a normal distribution.

from proxy-nca.

@dichotomies Initially, the proxy weights are randomly sampled from normal distribution.
When the network starts to train, are the weights for the proxies updated class wise? In other words do we have a specific proxy embedding corresponding to particular class whose weights are updated during training?

from proxy-nca.

Hi Sidra, each proxy embedding corresponds to a particular class - also during training. It's somewhat like when training a classification model with softmax, but instead of having single units, we have embeddings. Hope that answers your question!

from proxy-nca.

Hi @dichotomies, I wanna make sure if I understand correctly. These proxies are randomly initialized once. And we never change them during the training. Is it right? Thanks for your helps.

from proxy-nca.

The proxies are randomly initialized and change during training.

from proxy-nca.

The proxies are randomly initialized and change during training.

Cool! That's all I need. Thanks for the answer

from proxy-nca.