Hello!
I used Autoformer on CPU before and it worked fine, but now I'm trying to use with GPU: cuda:0, but I got the following error:
Args in experiment: Namespace(activation='gelu', batch_size=32, bucket_size=4, c_out=1, checkpoints='./checkpoints/', d_ff=2048, d_layers=1, d_model=2048, data='custom', data_path='ts_85afd5ce-bd63-445a-9ec5-23941e3c9421.csv', dec_in=1, des='Exp', devices='0,1,2,3', distil=True, do_predict=False, dropout=0.05, e_layers=2, embed='timeF', enc_in=1, factor=3, features='S', freq='t', gpu=0, is_training=1, itr=1, label_len=90, learning_rate=0.0001, loss='mse', lradj='type1', model='Autoformer', model_id='ts_180', moving_avg=25, n_hashes=4, n_heads=8, num_workers=10, output_attention=False, patience=3, pred_len=90, root_path='./dataset/ts/', seq_len=180, target='OT', train_epochs=2, use_amp=False, use_gpu=True, use_multi_gpu=False)
Use GPU: cuda:0
start training : ts_180_Autoformer_custom_ftS_sl180_ll90_pl90_dm2048_nh8_el2_dl1_df2048_fc3_ebtimeF_dtTrue_Exp_0
train 3945
val 514
test 1115
iters: 100, epoch: 1 | loss: 0.3764235
speed: 0.4895s/iter; left time: 71.9599s
Epoch: 1 cost time: 60.09772181510925
Traceback (most recent call last):
File "/work/run.py", line 120, in
exp.train(setting)
File "/work/exp/exp_main.py", line 176, in train
vali_loss = self.vali(vali_data, vali_loader, criterion)
File "/work/exp/exp_main.py", line 77, in vali
outputs = self.model(batch_x, batch_x_mark, dec_inp, batch_y_mark)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/work/models/Autoformer.py", line 88, in forward
enc_out, attns = self.encoder(enc_out, attn_mask=enc_self_mask)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/work/layers/Autoformer_EncDec.py", line 103, in forward
x, attn = attn_layer(x, attn_mask=attn_mask)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/work/layers/Autoformer_EncDec.py", line 69, in forward
new_x, attn = self.attention(
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/work/layers/AutoCorrelation.py", line 156, in forward
out, attn = self.inner_correlation(
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/work/layers/AutoCorrelation.py", line 124, in forward
V = self.time_delay_agg_inference(values.permute(0, 2, 3, 1).contiguous(), corr).permute(0, 3, 1, 2)
File "/work/layers/AutoCorrelation.py", line 72, in time_delay_agg_inference
tmp_delay = init_index + delay[:, i].unsqueeze(1).unsqueeze(1).unsqueeze(1).repeat(1, head, channel, length)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
Thank you very much for any help!