RuntimeError: Expected object of scalar type Byte but got scalar type Bool for argument #2 'mask' about transformer HOT 4 CLOSED

Hi, PyTorch is telling you here that the mask (here represented by the variable attention_mask) should be of type Bool, but is Byte. I encourage you to get deeper into this error to understand why the mask is of type Byte, I can't help you with just this traceback. If you have more info, you can always post it here.

from transformer.

thanks.
I have found out the reason for the error, which has something to do with version of python

from transformer.

btw, I wonder what is the reason for changing softmax to sigmoid? looking forward to your reply :)

from transformer.

I'm currently working on a regression problem, softmax is usually applied to classification, so it's not a good fit here. I'm using sigmoid, but it could be replaced with any other activation function, such as ReLU or its variations. Feel free to play around with the ones implemented in pytorch, it should be easy to plug !

from transformer.