Getting error during training about centermask2 HOT 6 CLOSED

I have figured it out. The thing is for the inputs of torch.max, one is float and the other is double.
Solution: In ./centermask/modeling/centermask/mask_head.py, change
value_eps = 1e-10 * torch.ones(gt_masks.shape[0], device=gt_masks.device)
to
value_eps = 1e-10 * torch.ones(gt_masks.shape[0], device=gt_masks.device).double()
and also change
value_1 = torch.ones(pred_masks.shape[0], device=gt_masks.device)
to
value_1 = torch.ones(pred_masks.shape[0], device=gt_masks.device).double().

from centermask2.

Have you solved it? I meet the same problem.

from centermask2.

Now it evokes a new problem.

{'loss_mask': tensor(0.6932, device='cuda:1', grad_fn=),
'loss_maskiou': tensor(0.5323, device='cuda:1', dtype=torch.float64, grad_fn=),
'loss_fcos_cls': tensor(1.0558, device='cuda:1', grad_fn=),
'loss_fcos_loc': tensor(0.8615, device='cuda:1', grad_fn=),
'loss_fcos_ctr': tensor(0.5908, device='cuda:1', grad_fn=)}

loss_maskiou is of type double now instead of float

next change of line 167 helps
return mask_loss, selected_mask, gt_classes, maskiou_targets.float().detach()

from centermask2.

@VladKatsman

Could you send the PR?

from centermask2.

Is this issue related to pytorch version? Since with pytorch==1.40, I have to comment .double() to make training work.

from centermask2.

Heads up, this update will only work with pytorch > 1.5 , and will break build with pytorch versions less then that.

from centermask2.