Question for training time about burstormer HOT 7 OPEN

For synthetic burst SR, we train the proposed Burstormer for 300 epochs with 4 RTX6000 GPUs, and it takes roughly around 5-6 days. To fine-tune this model for real burst SR it takes around 8 hours.

from burstormer.

For synthetic burst SR, we train the proposed Burstormer for 300 epochs with 4 RTX6000 GPUs, and it takes roughly around 5-6 days. To fine-tune this model for real burst SR it takes around 8 hours.

Thanks for reply. Actually, my GPU is 3090 and I am training BIPNet using 4GPUs. Currently, I am training around 5days but still 51 epochs.
Could you let me know which model is faster between Bustomer and BIPNet?

Note that. To fair comparison with my model, I just changed precision from 16 to 32 and deterministic from True to False.

from burstormer.

Burstormer is faster than BIPNet. In our settings, we keep precision 16 and deterministic True.
In released codes, each GPU serves a single burst (so 4 GPUs combinely make batch size=4). So, to improve training time, you can implement the network architecture such that you can increase batch size.

from burstormer.

Burstormer is faster than BIPNet. In our settings, we keep precision 16 and deterministic True.
In released codes, each GPU serves a single burst (so 4 GPUs combinely make batch size=4). So, to improve training time, you can implement the network architecture such that you can increase batch size.

After getting your comment, I started to train Burstomer. I didn't change the setting that you provided. But the training time per epoch was 2h 10m. I can expect whole training time for 300 epochs 27 days, it was fully different that you comments.

I used training set Zurich RAW to RGB Dataset(22 GB)

Could you advise to get same training time that you mentioned? For example, save dataset on harddisk after applying preprocessing and augment, or use the other dataset such as Cannon RGB Images(5.5GB) that was commented in zurich_raw2rgb_dataset.py.

from burstormer.

One more question. I download pre-trained model that you provided for Synthetic(Track 1)

When I load weight and apply it to model, a lot of error were occurred.

I didn't change any code. Could you let me know do I need change code or could you check provided pre-trained model was correct?

Below is error code.

===========================================================================================
Missing key(s) in state_dict: "back_projection1.feat_fusion.0.weight", "back_projection1.feat_fusion.0.bias", "back_projection1.feat_expand.0.weight", "back_projection1.feat_expand.0.bias", "back_projection2.feat_fusion.0.weight", "back_projection2.feat_fusion.0.bias", "back_projection2.feat_expand.0.weight", "back_projection2.feat_expand.0.bias".
Unexpected key(s) in state_dict: "back_projection1.diff_fusion.weight", "back_projection1.feat_fusion.weight", "back_projection1.feat_expand.weight", "back_projection2.diff_fusion.weight", "back_projection2.feat_fusion.weight", "back_projection2.feat_expand.weight".
size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment0.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment1.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.alignment2.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]).
size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).

from burstormer.

One more question. I download pre-trained model that you provided for Synthetic(Track 1)

When I load weight and apply it to model, a lot of error were occurred.

I didn't change any code. Could you let me know do I need change code or could you check provided pre-trained model was correct?

Below is error code.

=========================================================================================== Missing key(s) in state_dict: "back_projection1.feat_fusion.0.weight", "back_projection1.feat_fusion.0.bias", "back_projection1.feat_expand.0.weight", "back_projection1.feat_expand.0.bias", "back_projection2.feat_fusion.0.weight", "back_projection2.feat_fusion.0.bias", "back_projection2.feat_expand.0.weight", "back_projection2.feat_expand.0.bias". Unexpected key(s) in state_dict: "back_projection1.diff_fusion.weight", "back_projection1.feat_fusion.weight", "back_projection1.feat_expand.weight", "back_projection2.diff_fusion.weight", "back_projection2.feat_fusion.weight", "back_projection2.feat_expand.weight". size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment0.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment0.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment1.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment1.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.alignment2.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.alignment2.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.0.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm1.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk.weight: copying a param with shape torch.Size([192, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([96, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.qk_dwconv.weight: copying a param with shape torch.Size([192, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.v_dwconv.weight: copying a param with shape torch.Size([96, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([48, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.attn.project_out.weight: copying a param with shape torch.Size([96, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.weight: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.norm2.body.bias: copying a param with shape torch.Size([96]) from checkpoint, the shape in current model is torch.Size([48]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_in.weight: copying a param with shape torch.Size([510, 96, 1, 1]) from checkpoint, the shape in current model is torch.Size([254, 48, 1, 1]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.dwconv.weight: copying a param with shape torch.Size([510, 1, 3, 3]) from checkpoint, the shape in current model is torch.Size([254, 1, 3, 3]). size mismatch for align.cascade_alignment.back_projection.encoder1.1.ffn.project_out.weight: copying a param with shape torch.Size([96, 255, 1, 1]) from checkpoint, the shape in current model is torch.Size([48, 127, 1, 1]).

Hi Luis, I got the same issue on Track_1_evaluation.py, have you solved it?

from burstormer.

Hi Luis, I got the same issue on Track_1_evaluation.py, have you solved it?

Hi ChongWang.

I can't solve above issue. DId you solve it?

from burstormer.