conll2003 results are not reproducible by using params written in paper about ncrfpp HOT 6 CLOSED

The configuration of the paper is almost the same with the demo.train.config, you can just change the iteration=1 to iteration=100 in your experiments.

You need first use the right data and tag scheme, as well as the pretrained word embeddings.

from ncrfpp.

this is the best dev F1 i got with 200 iterations:
I am using BIMS tag scheme and glove.100 word embeddings with 200 iterations. rest all remains the same as demo.train.config. Running code on AWS nvidia K80 instances (p2)

Epoch: 82 training finished. Time: 100.20s, speed: 149.57st/s,  total loss: 879.6669143559411
Dev: time: 5.02s, speed: 700.19st/s; acc: 0.9851, p: 0.9309, r: 0.9251, f: 0.9280
Test: time: 4.79s, speed: 781.61st/s; acc: 0.9779, p: 0.8996, r: 0.8982, f: 0.8989

reproducing same results is important for my paper where I am adding word LM layer on top of your architecture.

from ncrfpp.

Can you give me the front log of your experiment? I need to check if you use the right data and configuration. Generally, the performance is various based on the running environment, but they are almost always larger than 91%. If your results is less than 90, there must exists problems in either the data and the configuration.

BTW, what do you mean the BIMS tag? It should be either BIO or BMES/BIOES.

from ncrfpp.

the more iterations i run , the more f1 keeps improving. I suspect this slowness in converging might be happening due to lr_decay.
but still in 1000 iterations, best I got yet is 90.3. Please find attached log and kindly check if you find some anomaly in parameters.
train.log

from ncrfpp.

@dhwajraj The first problem I find is that you set the ave_batch_loss=True while the default setting is ave_batch_loss=False, you can change this setting as False and run your experiment again.

I am not sure if there is any other problem in your experiment, you can run it first and update with me.

from ncrfpp.

@dhwajraj @jiesutd were you able to reproduce the results with this?

from ncrfpp.