mswellhao / chinesener Goto Github PK

rzai@rzai00:~/prj/chineseNER/KG/workDir$ python ../main.py train --train data/traindata --dev data/devdata --test data/testdata --score_dir trainResult
Using gpu device 0: GeForce GTX 1080 (CNMeM is disabled, cuDNN 5105)
choose model type .................. LSTMbaseline
embedding size : 150 embeddings num : 316001
{'I-PRO': 2, 'B-PRO': 1, 'B-ORG': 9, 'I-COM': 8, 'I-PER': 4, 'B-COM': 7, 'O': 0, 'I-LOC': 6, 'I-ORG': 10, 'B-LOC': 5, 'B-PER': 3}
model training args : Namespace(CRF=False, action='train', batch_size=64, corpus='data/rawdata', dev='data/devdata', drop_pro=0.35, emb_lrate=0.01, embeddings='vector/baidu_charvectors_150', epoch=10, eva='data/testdata', fix_emb=True, gradbound=-0.1, highreg_weight=0.1, istoken=False, lowreg_weight=0.1, lrate=0.02, model='', model_type='LSTMbaseline', mweight=0.9, nc=3, net_size=[150, 450, 300, 11], opt='adaGrad', output='result', score_dir='trainResult', second_layer='forward', test='data/testdata', train='data/traindata', variance=0.01, wbound=-0.1, wdecay=0.0)
whether fixing word embedding : True
model size : [150, 450, 300, 11] ; model type : LSTMbaseline ; model second layer type : forward
adopt sequence based training ..............
adopt learning method ............. adaGrad
current learning rate ..............0.02
new epoch start.....................0
batch loss : 143.739120483
batch loss : 32.5928192139
.............................................................................................
batch loss : 4.02528572083
batch loss : 2.54068398476
epoch done ............... sum loss : 728.014112115
trainscore ........ epoch : 3 P: 0.712865819988 R : 0.399690402476 F : 0.512200068248

devscore ........ epoch : 3 P: 0.597014925364 R : 0.30769230769 F : 0.406091480781

Segmentation fault (core dumped)
rzai@rzai00:~/prj/chineseNER/KG/workDir$

Hi,
python main.py extract --model model.NER --corpus data/rawdata --output result
choose model type .................. LSTMbaseline
Traceback (most recent call last):
File "main.py", line 291, in
main(args)
File "main.py", line 14, in main
embeddings = pickle.load(open(args.embeddings))
IOError: [Errno 2] No such file or directory: 'vector/baidu_charvectors_150'
molyswu
Thank you!

I use this command "python ../main.py extract --model model.NER --corpus data/rawdata --output result
", but turns out to be an exception:

Traceback (most recent call last):
File "../main.py", line 288, in
main(args)
File "../main.py", line 116, in main
ens = model.extract(data)
File "/home/t/th/chineseNER/KG/NERmodel.py", line 107, in extract
tags, _ = self.get_best_seq(ins[0])
File "/home/t/th/chineseNER/KG/LSTM.py", line 99, in get_best_seq
tags, logp = self.evafunc(x)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 786, in call
allow_downcast=s.allow_downcast)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/type.py", line 177, in filter
data.shape))
TypeError: ('Bad input argument to theano function with name "/home/t/th/chineseNER/KG/LSTM.py:90" at index 0(0-based)', 'Wrong number of dimensions: expected 1, got 0 with shape ().')

I slightly change your code to debug, so maybe line number is not exactly matched, it seems input should be sequence?

mswellhao，您好！

链接打不开，导致baidu_charvectors_150下载不了。能帮忙放在github上，好吗？
多谢！！！！

hi :
thanks for sharing this project. i follow your instruction and train the model,all the parameters are defualt. after 10 epoch iteration,i got the final model "model_9"。 with this model,i do some extract,but result is so bad,for example
$more rawdata
李彦宏是腾讯公司的好老板。
$more result
[[[0, 1, "PER"]]]
the person name “李彦宏” 都没有识别出来
can you give me some suggestion?

运行程序时报错，没有上述文件，请问楼主这个文件是什么呢