Question about argument-trigger distance experiment about paie HOT 6 CLOSED

Hi, thanks for your interest.
Could you provide a targeted explanation about your problem?

from paie.

For trigger-argument distance experiment, my steps are as follows:
After training the model, the ground truth label and the predicted label are divided into 5 sets according to the distance between the trigger word and the argument, and then F1 is calculated in each set.
I found the overall F1 to be reasonable, but the F1 per set was slightly lower, so wanted to ask you if I was missing some steps, thanks for the reply.

from paie.

Yes, your steps are right.
According to our experiments, we find the F1-scores grouped by distances are highly unstable (especially the +2 and -2 settings), since the argument amount in these settings are too small (tens of, I remember?). Suggest you to train with other random seeds.

from paie.

When dividing the distance sets of ground truth label and predicted label, should we use the ground truth label and predicted label for distance judgment, or use both ground truth label?

from paie.

Use ground truth label only.

from paie.

thanks for your answer.

from paie.