cuhksz-nlp / aesiner Goto Github PK

Excuse me, could you please release how to preprocess the original Ontonote4.0, because I can't get the information from the https://github.com/yhcc/OntoNotes-5.0-NER..
Thanks!!!

RT

非常感谢开源代码。但是我发现在执行WE数据的时候存在一些问题

Traceback (most recent call last):
File "train_zen_cn.py", line 268, in
trainer.train(load_best_model=False)
File "AESINER-master/fastNLP/core/trainer.py", line 702, in train
raise e
File "AESINER-master/fastNLP/core/trainer.py", line 695, in train
self._train()
File "AESINER-master/fastNLP/core/trainer.py", line 861, in _train
dep_matrix = self.get_dep_mask_matrix(indices, seq_len=torch.max(batch_x.get("seq_len")).item())
File "AESINER-master/fastNLP/core/trainer.py", line 796, in get_dep_mask_matrix
ret[i][j] = 1
IndexError: list assignment index out of range

Traceback (most recent call last):
File "train_zen_cn.py", line 262, in
trainer.train(load_best_model=False)
File "AESINER-master/fastNLP/core/trainer.py", line 702, in train
raise e
File "AESINER-master/fastNLP/core/trainer.py", line 695, in train
self._train()
File "AESINER-master/fastNLP/core/trainer.py", line 854, in _train
features = self.get_features(indices, seq_len=torch.max(batch_x.get("seq_len")).item())
File "AESINER-master/fastNLP/core/trainer.py", line 748, in get_features
ret_list[i] = torch.tensor(ret_list[i])
ValueError: expected sequence of length 29 at dim 1 (got 31)

請問這個該如何解決？

Traceback (most recent call last):
File "data_helper.py", line 401, in
request_features_from_stanford(data_dir, 'train')
File "data_helper.py", line 161, in request_features_from_stanford
results = nlp.annotate(sentence, properties=props)
File "/home/jsj201-6/mount1/lww/AESINER-master1/corenlp.py", line 160, in annotate
r_dict = json.loads(r.text)
File "/home/jsj201-6/anaconda3/envs/lww/lib/python3.7/json/init.py", line 348, in loads
return _default_decoder.decode(s)
File "/home/jsj201-6/anaconda3/envs/lww/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/jsj201-6/anaconda3/envs/lww/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

是否对数据做过预处理，我用的WEIBO的数据集做的处理。只是将训练集、测试集、开发集改了名，我以为是空行符出现的错误，然后将数据中的空行删除，但也不行，不会爆出错，但是json文件为空了。请问是什么问题？

Hi I am trying to run the code on a GPU machine using my dataset. I am getting the following error, Can you please check?

Save cache to caches/tweet_dataset_bert_elmo_bioes_adatrans_True_123_10_10_10.pkl.
training epochs started 2022-03-12-16-43-15
Traceback (most recent call last):
File "/home/akalia/aesiner_project/train_bert_elmo_en.py", line 237, in
trainer.train(load_best_model=False)
File "/home/akalia/aesiner_project/fastNLP/core/trainer.py", line 702, in train
raise e
File "/home/akalia/aesiner_project/fastNLP/core/trainer.py", line 695, in train
self._train()
File "/home/akalia/aesiner_project/fastNLP/core/trainer.py", line 894, in _train
prediction = self._data_forward(self.model, batch_x)
File "/home/akalia/aesiner_project/fastNLP/core/trainer.py", line 1003, in _data_forward
y = network(
File "/home/akalia/anaconda3/envs/akalia/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/akalia/aesiner_project/models/TENER.py", line 444, in forward
return self._forward(chars, target, bigrams, pos_features, dep_features, chunk_features,
File "/home/akalia/aesiner_project/models/TENER.py", line 411, in _forward
hidden = self.embed(chars)
File "/home/akalia/anaconda3/envs/akalia/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/akalia/aesiner_project/fastNLP/embeddings/stack_embedding.py", line 94, in forward
outputs.append(embed(words))
File "/home/akalia/anaconda3/envs/akalia/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/akalia/aesiner_project/fastNLP/embeddings/elmo_embedding.py", line 136, in forward
outputs = self.model(words)
File "/home/akalia/anaconda3/envs/akalia/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/akalia/aesiner_project/fastNLP/embeddings/elmo_embedding.py", line 313, in forward
encoder_output = self.encoder(token_embedding, seq_len)
File "/home/akalia/anaconda3/envs/akalia/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/akalia/aesiner_project/fastNLP/modules/encoder/_elmo.py", line 319, in forward
inputs = nn.utils.rnn.pack_padded_sequence(inputs, sort_lens, batch_first=True)
File "/home/akalia/anaconda3/envs/akalia/lib/python3.9/site-packages/torch/nn/utils/rnn.py", line 249, in pack_padded_sequence
_VF._pack_padded_sequence(input, lengths, batch_first)
RuntimeError: 'lengths' argument should be a 1D CPU int64 tensor, but got 1D cuda:0 Long tensor

在其他数据集上一直出现IndexError:list index out of range问题，不清楚是不是索引有问题？从0开始最大应该是5249，但是会出现5250，应该如何解决？

您好，我想问一下tencent_unigram.txt文件在哪里怎么获取呢？或者您可以发给我一份么？

运行data_helper.py出现错误