有关评估指标的问题 about bertner HOT 5 CLOSED

2021-09-28 10:39:47,071 - log/train.log - INFO - iteration:26 step:25/163, NER loss: 0.226979
I0928 10:39:47.071518 140003545884416 train.py:164] iteration:26 step:25/163, NER loss: 0.226979
2021-09-28 10:41:23,881 - log/train.log - INFO - iteration:26 step:125/163, NER loss: 0.257177
I0928 10:41:23.881218 140003545884416 train.py:164] iteration:26 step:125/163, NER loss: 0.257177
2021-09-28 10:41:53,836 - log/train.log - INFO - evaluate:dev
I0928 10:41:53.836905 140003545884416 train.py:77] evaluate:dev
2021-09-28 10:42:05,035 - log/train.log - INFO - processed 106932 tokens with 3661 phrases; found: 3673 phrases; correct: 3471.

I0928 10:42:05.035623 140003545884416 train.py:81] processed 106932 tokens with 3661 phrases; found: 3673 phrases; correct: 3471.

2021-09-28 10:42:05,035 - log/train.log - INFO - accuracy: 99.27%; precision: 94.50%; recall: 94.81%; FB1: 94.66

I0928 10:42:05.035877 140003545884416 train.py:81] accuracy: 99.27%; precision: 94.50%; recall: 94.81%; FB1: 94.66

2021-09-28 10:42:05,035 - log/train.log - INFO - : precision: 0.00%; recall: 0.00%; FB1: 0.00 3

I0928 10:42:05.035951 140003545884416 train.py:81] : precision: 0.00%; recall: 0.00%; FB1: 0.00 3

2021-09-28 10:42:05,036 - log/train.log - INFO - LOC: precision: 94.48%; recall: 96.11%; FB1: 95.29 1830

I0928 10:42:05.036015 140003545884416 train.py:81] LOC: precision: 94.48%; recall: 96.11%; FB1: 95.29 1830

2021-09-28 10:42:05,036 - log/train.log - INFO - ORG: precision: 91.92%; recall: 89.75%; FB1: 90.82 953

I0928 10:42:05.036077 140003545884416 train.py:81] ORG: precision: 91.92%; recall: 89.75%; FB1: 90.82 953

2021-09-28 10:42:05,036 - log/train.log - INFO - PER: precision: 97.63%; recall: 98.19%; FB1: 97.91 887

I0928 10:42:05.036137 140003545884416 train.py:81] PER: precision: 97.63%; recall: 98.19%; FB1: 97.91 887

2021-09-28 10:42:05,038 - log/train.log - INFO - evaluate:test
I0928 10:42:05.038823 140003545884416 train.py:77] evaluate:test
2021-09-28 10:42:25,667 - log/train.log - INFO - processed 214621 tokens with 7456 phrases; found: 7474 phrases; correct: 7033.

I0928 10:42:25.667727 140003545884416 train.py:81] processed 214621 tokens with 7456 phrases; found: 7474 phrases; correct: 7033.

2021-09-28 10:42:25,667 - log/train.log - INFO - accuracy: 99.23%; precision: 94.10%; recall: 94.33%; FB1: 94.21

I0928 10:42:25.667995 140003545884416 train.py:81] accuracy: 99.23%; precision: 94.10%; recall: 94.33%; FB1: 94.21

2021-09-28 10:42:25,668 - log/train.log - INFO - : precision: 0.00%; recall: 0.00%; FB1: 0.00 2

I0928 10:42:25.668069 140003545884416 train.py:81] : precision: 0.00%; recall: 0.00%; FB1: 0.00 2

2021-09-28 10:42:25,668 - log/train.log - INFO - LOC: precision: 94.03%; recall: 95.03%; FB1: 94.53 3501

I0928 10:42:25.668134 140003545884416 train.py:81] LOC: precision: 94.03%; recall: 95.03%; FB1: 94.53 3501

2021-09-28 10:42:25,668 - log/train.log - INFO - ORG: precision: 91.49%; recall: 90.81%; FB1: 91.15 2150

I0928 10:42:25.668197 140003545884416 train.py:81] ORG: precision: 91.49%; recall: 90.81%; FB1: 91.15 2150

2021-09-28 10:42:25,668 - log/train.log - INFO - PER: precision: 97.42%; recall: 97.47%; FB1: 97.45 1821

I0928 10:42:25.668256 140003545884416 train.py:81] PER: precision: 97.42%; recall: 97.47%; FB1: 97.45 1821

2021-09-28 10:42:25,960 - log/train.log - INFO - new best test f1 score:94.210
I0928 10:42:25.960424 140003545884416 train.py:94] new best test f1 score:94.210

以一个iteration而言，打印出来的FB!值是训练时的分数嘛，为什么和new best test f1 score是一样的，这个new best test f1 score是测试集上的分数吗，它是怎么定义的，是按预测对一个实体算还是一个标签算？ 希望能回答一下，还有上面 iteration:26 有上下两个部分的预测精度，具体都代表什么？

from bertner.

@zsh123123 建议先学一下基础，验证集选择模型，测试集检验泛化性能

from bertner.

@zsh123123 建议先学一下基础，验证集选择模型，测试集检验泛化性能

所以前面的评估是在验证集上，选出了最佳的模型，后面的评估是在测试集上，测试模型的泛化性能，但是在训练集上的评估是通过loss函数，并没有设置precision/recall/f1值。还有一个疑问是之前没有见过FB1，是不是就是f1

from bertner.

@zsh123123 是一样的

from bertner.

@zsh123123 是一样的

还有一个问题，准确度是按预测对一个实体算还是一个标签算

from bertner.