yanwii / seq2seq Goto Github PK

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCTensorCopy.c:18
出现这个问题是我改了代码
#words = jieba.lcut(sentence)
words = [word for word in sent]
改动之后没有使用结巴分词，直接用的字符做的输入，这样训练的时候为什么会有上面的错误，谢谢

另外有点问题想请教下：
１　你的代码中对原始数据也就是问答数据进行jieba分词处理，最后是直接对词进行的编码然后输入，我想问的是这种分词与否是否会对结果有影响，为什么？
２　中文数据中的标点符号需要去除吗？

项目下下来简单填了几个answer和question然后跑起来测试，发现可以运行并且效果还不错，就搞了将近2mb的answer和question，在preprocessing阶段通过，开始训练的时候就提示下面的错误了。

THCudaCheck FAIL file=C:/new-builder_3/win-wheel/pytorch/aten/src/ATen/native/cuda/Embedding.cu line=247 error=77 : an illegal memory access was encountered
Traceback (most recent call last):
File "seq2seq.py", line 436, in
seq.train()
File "seq2seq.py", line 210, in train
loss, logits = self.step(inputs, targets, self.max_length)
File "seq2seq.py", line 265, in step
loss.backward()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\torch\tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35\lib\site-packages\torch\autograd_init_.py", line 90, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: cuda runtime error (77) : an illegal memory access was encountered at C:/new-builder_3/win-wheel/pytorch/aten/src/ATen/native/cuda/Embedding.cu:2
47

一脸懵逼，我该怎么办

我的参数设置：
self.input_size = 100
self.output_size = 1200
self.hidden_size = 1200
self.max_length = 1200
训练时报错：数组越界错误，不知道是什么情况，忘不吝赐教，暂时没深入网络中去，想先跑个例子学习下，问题可能有点无脑
File "/Users/sunwc/Documents/pythonproject/seq2seq-master/seq2seq.py", line 35, in forward
embedded = self.embedding(word_inputs).view(seq_len, 1, -1)
RuntimeError: index out of range at /Users/soumith/code/builder/wheel/pytorch-src/aten/src/TH/generic/THTensorMath.cpp:352

增加了自己的数据，训练时loss越来越大，请问这是为什么？请问更改哪些参数能提高预测结果呢？

, TensorInfo<long, IndexType>, int, int, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2]: block: [0,0,0], thread: [91,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/torch/lib/THC/THCTensorIndex.cu:279: void indexSelectSmallIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2]: block: [0,0,0], thread: [92,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/torch/lib/THC/THCTensorIndex.cu:279: void indexSelectSmallIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2]: block: [0,0,0], thread: [93,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/torch/lib/THC/THCTensorIndex.cu:279: void indexSelectSmallIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2]: block: [0,0,0], thread: [94,0,0] Assertion srcIndex < srcSelectDimSize failed.
/pytorch/torch/lib/THC/THCTensorIndex.cu:279: void indexSelectSmallIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2]: block: [0,0,0], thread: [95,0,0] Assertion srcIndex < srcSelectDimSize failed.
THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/THCTensorCopy.c line=21 error=59 : device-side assert triggered
Traceback (most recent call last):
File "seq2seq.py", line 453, in
seq.train()
File "seq2seq.py", line 227, in train
loss, logits = self.step(inputs, targets, self.max_length)
File "seq2seq.py", line 259, in step
decoder_input = decoder_input.cuda()
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/variable.py", line 298, in cuda
return CudaTransfer.apply(self, device, async)
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/_functions/tensor.py", line 201, in forward
return i.cuda(async=async)
File "/usr/local/lib/python3.5/dist-packages/torch/_utils.py", line 69, in cuda
return new_type(self.size()).copy(self, async)
RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCTensorCopy.c:21

执行：
python3 preprocessing.py
python3 seq2seq.py train
之后，出现：
请问是不是python或pytorch的版本不对？

Traceback (most recent call last):
File "seq2seq.py", line 438, in
seq.train()
File "seq2seq.py", line 213, in train
loss, logits = self.step(inputs, targets, self.max_length)
File "seq2seq.py", line 253, in step
decoder_output, decoder_context, decoder_hidden, decoder_attention = self.decoder(decoder_input, decoder_context, decoder_hidden, encoder_outputs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "seq2seq.py", line 109, in forward
attn_weights = self.attn(rnn_output.squeeze(0), encoder_outputs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "seq2seq.py", line 65, in forward
attn_energies[i] = self.score(hidden, encoder_outputs[i])
File "seq2seq.py", line 76, in score
energy = hidden.dot(energy)
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/variable.py", line 631, in dot
return Dot.apply(self, other)
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/_functions/blas.py", line 211, in forward
return vector1.new((vector1.dot(vector2),))
RuntimeError: Expected argument self to have 1 dimension(s), but has 2 at /pytorch/torch/csrc/generic/TensorMethods.cpp:23020

在问答集中增加问题后，就报了这个错误。
我就改了这几个参数
self.input_size = 1840
self.output_size = 1840
self.hidden_size = 64
self.max_length = 50
/pytorch/torch/lib/THCUNN/ClassNLLCriterion.cu:57: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [0,0,0] Assertion t >= 0 && t < n_classes failed.
THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/THCTensorCopy.c line=18 error=59 : device-side assert triggered
Traceback (most recent call last):
File "seq2seq.py", line 443, in
seq.retrain()
File "seq2seq.py", line 434, in retrain
self.train()
File "seq2seq.py", line 213, in train
loss, logits = self.step(inputs, targets, self.max_length)
File "seq2seq.py", line 253, in step
decoder_output, decoder_context, decoder_hidden, decoder_attention = self.decoder(decoder_input, decoder_context, decoder_hidden, encoder_outputs)
File "/home/liuyong/venv/seq2seq-master/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "seq2seq.py", line 105, in forward
attn_weights = self.attn(rnn_output.squeeze(0), encoder_outputs)
File "/home/liuyong/venv/seq2seq-master/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "seq2seq.py", line 63, in forward
if USE_CUDA: attn_energies = attn_energies.cuda()
File "/home/liuyong/venv/seq2seq-master/lib/python3.5/site-packages/torch/autograd/variable.py", line 279, in cuda
return CudaTransfer.apply(self, device_id, async)
File "/home/liuyong/venv/seq2seq-master/lib/python3.5/site-packages/torch/autograd/_functions/tensor.py", line 151, in forward
return i.cuda(async=async)
File "/home/liuyong/venv/seq2seq-master/lib/python3.5/site-packages/torch/_utils.py", line 66, in cuda
return new_type(self.size()).copy(self, async)

如何训练自己的数据，以及如何训练多个数据对应的答案！

这个是什么错误 好奇怪的错误
While copying the parameter named encoder.embedding.weight, whose dimensions in the model are torch.Size([5798, 100]) and whose dimensions in the checkpoint are torch.Size([600, 100]), ...
inconsistent tensor size, expected tensor [5798 x 100] and src [600 x 100] to have the same number of elements, but got 579800 and 60000 elements respectively

C:\ProgramData\Anaconda3\envs\torch\python.exe E:/hyx/seq2seq-master/seq2seq.py train
C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\torch\nn\modules\rnn.py:54: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.05 and num_layers=1
"num_layers={}".format(dropout, num_layers))
Error(s) in loading state_dict for seq2seq:
size mismatch for encoder.embedding.weight: copying a param with shape torch.Size([600, 100]) from checkpoint, the shape in current model is torch.Size([14, 100]).
size mismatch for decoder.embedding.weight: copying a param with shape torch.Size([1600, 100]) from checkpoint, the shape in current model is torch.Size([15, 100]).
size mismatch for decoder.out.weight: copying a param with shape torch.Size([1600, 200]) from checkpoint, the shape in current model is torch.Size([15, 200]).
size mismatch for decoder.out.bias: copying a param with shape torch.Size([1600]) from checkpoint, the shape in current model is torch.Size([15]).
No model!
E:/hyx/seq2seq-master/seq2seq.py:68: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
return F.softmax(attn_energies).unsqueeze(0).unsqueeze(0)
E:/hyx/seq2seq-master/seq2seq.py:110: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
output = F.log_softmax(self.out(torch.cat((rnn_output, context), 1)))
E:/hyx/seq2seq-master/seq2seq.py:266: UserWarning: torch.nn.utils.clip_grad_norm is now deprecated in favor of torch.nn.utils.clip_grad_norm_.
torch.nn.utils.clip_grad_norm(self.encoder.parameters(), clip)
E:/hyx/seq2seq-master/seq2seq.py:267: UserWarning: torch.nn.utils.clip_grad_norm is now deprecated in favor of torch.nn.utils.clip_grad_norm_.
torch.nn.utils.clip_grad_norm(self.decoder.parameters(), clip)
Traceback (most recent call last):
File "E:/hyx/seq2seq-master/seq2seq.py", line 436, in
seq.train()
File "E:/hyx/seq2seq-master/seq2seq.py", line 210, in train
loss, logits = self.step(inputs, targets, self.max_length)
File "E:/hyx/seq2seq-master/seq2seq.py", line 271, in step
return loss.data[0] / target_length, decoder_outputs
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number
新手一脸蒙蔽。。。是哪里不对呢

the batch size seems not work and can only be set to 1

这个错误是什么意思呢？
Traceback (most recent call last):
File "seq2seq.py", line 441, in
seq.train()
File "seq2seq.py", line 211, in train
inputs, targets = self.next(1, shuffle=False)
File "seq2seq.py", line 193, in next
inputs = Variable(torch.LongTensor(inputs)).transpose(1, 0).contiguous()
RuntimeError: given sequence has an invalid size of dimension 2: 0

RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed. at /b/wheel/pytorch-src/torch/lib/THNN/generic/ClassNLLCriterion.c:45

这个是什么错误呢？训练到 epoch: 16800 之后出现的错误

I followed the README.But I get this problem.

C:\ProgramData\Anaconda3\envs\torch\python.exe E:/hyx/seq2seq-master/seq2seq.py train
C:\ProgramData\Anaconda3\envs\torch\lib\site-packages\torch\nn\modules\rnn.py:54: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.05 and num_layers=1
"num_layers={}".format(dropout, num_layers))
Error(s) in loading state_dict for seq2seq:
size mismatch for encoder.embedding.weight: copying a param with shape torch.Size([600, 100]) from checkpoint, the shape in current model is torch.Size([14, 100]).
size mismatch for decoder.embedding.weight: copying a param with shape torch.Size([1600, 100]) from checkpoint, the shape in current model is torch.Size([15, 100]).
size mismatch for decoder.out.weight: copying a param with shape torch.Size([1600, 200]) from checkpoint, the shape in current model is torch.Size([15, 200]).
size mismatch for decoder.out.bias: copying a param with shape torch.Size([1600]) from checkpoint, the shape in current model is torch.Size([15]).
No model!
还有下面的报错
Traceback (most recent call last):
File "E:/hyx/seq2seq-master/seq2seq.py", line 436, in
seq.train()
File "E:/hyx/seq2seq-master/seq2seq.py", line 210, in train
loss, logits = self.step(inputs, targets, self.max_length)
File "E:/hyx/seq2seq-master/seq2seq.py", line 271, in step
return loss.data[0] / target_length, decoder_outputs
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number
怎么调都不对，求指点~~

Traceback (most recent call last):
File "seq2seq.py", line 439, in
seq.train()
File "seq2seq.py", line 210, in train
loss, logits = self.step(inputs, targets, self.max_length)
File "seq2seq.py", line 236, in step
encoder_outputs, encoder_hidden = self.encoder(input_variable, encoder_hidden)
File "/home/zy/anaconda3/envs/pytchar/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "seq2seq.py", line 35, in forward
embedded = self.embedding(word_inputs).view(seq_len, 1, -1)
File "/home/zy/anaconda3/envs/pytchar/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/zy/anaconda3/envs/pytchar/lib/python3.7/site-packages/torch/nn/modules/sparse.py", line 110, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/home/zy/anaconda3/envs/pytchar/lib/python3.7/site-packages/torch/nn/functional.py", line 1110, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: index out of range at /opt/conda/conda-bld/pytorch_1535493744281/work/aten/src/TH/generic/THTensorMath.cpp:352

我在执行 python seq2seq.py train 的时候遇到了下面的错误，请问应该如何解决？

(py365) ubuntu@i-o9tnhsxb:~/chatbot/seq2seq$ python seq2seq.py train
While copying the parameter named encoder.embedding.weight, whose dimensions in the model are torch.Size([14, 100]) and whose dimensions in the checkpoint are torch.Size([600, 100]), ...
inconsistent tensor size, expected tensor [14 x 100] and src [600 x 100] to have the same number of elements, but got 1400 and 60000 elements respectively at /pytorch/torch/lib/TH/generic/THTensorCopy.c:86
No model!
Illegal instruction (core dumped)

你好，我用自己的数据在preprocessing运行时，在第43行就停止运行了，应该是self.vocab的错误，是需要我自己对他进行产生我需要的编码吗

i`m sorry to trouble you ,but had you noticed that your data is empty.
such as : ./data/supplementvovab.txt

您好，想請問如何加入自己的對話訓練資料?

謝謝。

数据按问题，答案放到两个文件中，训练时报错
Assertion `cur_target >= 0 && cur_target < n_classes' failed.
网上查的，是label 设置不对？

RuntimeError: index out of range at /b/wheel/pytorch-src/torch/lib/TH/generic/THTensorMath.c:273