Prediction Acc is 0.533 when running main.py and judger.py about cail2019 HOT 4 CLOSED

It was the bug of OpenCLaP BERT models. I opened an issue at their github repo 4 months ago, but they didn't fix it. You may see the warning when you save the models:

2019-11-05 16:06:53 - train model - INFO - Epoch 2, train Loss: 475.4495771, eval acc: 0.8539215686274509, eval loss: 286.7390137
Saving vocabulary to model/vocab.txt: vocabulary indices are not consecutive. Please check that the vocabulary is not corrupted!

It's caused by some white space lines in vocab.txt, my code will load the vocab.txt with no problem, but when I save it to the final model, the vocabulary indexes are broken which leads to bad performance.

You can fix it by copy the original vocab.txt to your final model:

cp OpenCLaP/vocab.txt model/vocab.txt

from cail2019.

I use already learned 民事文书BERT as my pretrained model downloaded from https://github.com/thunlp/OpenCLaP
And I set fp16=False
I train the model with your code. The training is good.
I copy some information from the train.log

But it turns out the acc of the prediction is 0.53 as I run the main.py and judger.py
Does the saving model has some problems?

I have also met the problem that you mentioned, do you solve this ?

from cail2019.

It was the bug of OpenCLaP BERT models. I opened an issue at their github repo 4 months ago, but they didn't fix it. You may see the warning when you save the models:

2019-11-05 16:06:53 - train model - INFO - Epoch 2, train Loss: 475.4495771, eval acc: 0.8539215686274509, eval loss: 286.7390137
Saving vocabulary to model/vocab.txt: vocabulary indices are not consecutive. Please check that the vocabulary is not corrupted!

It's caused by some white space lines in vocab.txt, my code will load the vocab.txt with no problem, but when I save it to the final model, the vocabulary indexes are broken which leads to bad performance.

You can fix it by copy the original vocab.txt to your final model:

cp OpenCLaP/vocab.txt model/vocab.txt

It was the bug of OpenCLaP BERT models. I opened an issue at their github repo 4 months ago, but they didn't fix it. You may see the warning when you save the models:

2019-11-05 16:06:53 - train model - INFO - Epoch 2, train Loss: 475.4495771, eval acc: 0.8539215686274509, eval loss: 286.7390137
Saving vocabulary to model/vocab.txt: vocabulary indices are not consecutive. Please check that the vocabulary is not corrupted!

It's caused by some white space lines in vocab.txt, my code will load the vocab.txt with no problem, but when I save it to the final model, the vocabulary indexes are broken which leads to bad performance.

You can fix it by copy the original vocab.txt to your final model:

cp OpenCLaP/vocab.txt model/vocab.txt

Can you provide a complete vocab.txt file ? thx!

from cail2019.

Just use the original vocab.txt in the pretrained models. Copy it into output model directory.

from cail2019.