Do you know this error? about uniter HOT 2 CLOSED

Looks like an index out of range error. I suspect that it happens in input_ids, type_ids or position_ids.

1. input_ids for VCR are augmented with 81 special tokens, directly applying word embeddings in pretrain.py may not work.
   Please check VCR model to see how to add special tokens:
   

   UNITER/model/vcr.py

   Lines 43 to 51 in 8b8e181

   	def init_word_embedding(self, num_special_tokens):
   	orig_word_num = self.uniter.embeddings.word_embeddings.weight.size(0)
   	new_emb = nn.Embedding(
   	orig_word_num + num_special_tokens, self.uniter.config.hidden_size)
   	new_emb.apply(self.init_weights)
   	emb = self.uniter.embeddings.word_embeddings.weight.data
   	new_emb.weight.data[:orig_word_num, :].copy_(emb)
   	self.uniter.embeddings.word_embeddings = new_emb

2. text_type_ids for VCR are set to 0 for question, 2 for answers and 3 for rationales. image_type_ids are set to 1.
   Pretraining model only have 2 type ids, 0 for text and 1 for image. Please check VCR model to see how to add type ids:
   

   UNITER/model/vcr.py

   Lines 31 to 42 in 8b8e181

   	def init_type_embedding(self):
   	new_emb = nn.Embedding(4, self.uniter.config.hidden_size)
   	new_emb.apply(self.init_weights)
   	for i in [0, 1]:
   	emb = self.uniter.embeddings.token_type_embeddings.weight.data[i, :]
   	new_emb.weight.data[i, :].copy_(emb)
   	emb = self.uniter.embeddings.token_type_embeddings.weight.data[0, :]
   	new_emb.weight.data[2, :].copy_(emb)
   	new_emb.weight.data[3, :].copy_(emb)
   	self.uniter.embeddings.token_type_embeddings = new_emb

3. position_ids should be in the range of [0, 511].

You will also need to call the above two functions in main file similar to

from uniter.

oh, i missed "init_type_embedding" and "init_word_embedding" function.

thank you for your kindly response!! :)

from uniter.