doubt in metric.py about ncrfpp HOT 4 CLOSED

You can refer this paper https://arxiv.org/pdf/1707.06799.pdf for the difference between BIO/BIOES/IOB.

from ncrfpp.

infact in the above case tag_list will have only one null string

from ncrfpp.

The metric is correct. You use the wrong data format, only BIO/BMES format are supported.

You need do more homework before doubt others code.

from ncrfpp.

@jiesutd Yes, thank you for pointing that out, the above example is not even correct with respect to BIO or BIOES as first I-MISC is followed by another I-MISC, I just checked and the dataset I was using had other such sentences, I corrected it to BIOES format and It seems to be giving f-score (!=-1)

However, I have no reference to check if my converted dataset is correct, do you have such reference? Does CoNLL website provide BIO and BIOES format as well? (I don't have their dataset directly as it requires some manual authentication/NDA process)

comments?

from ncrfpp.