The code in APNet/modeling/baseline.py does not match the description in Figure 2 about apnet HOT 5 CLOSED

Thanks for the good question.

Both of them are in the form of a hieratical pyramid and can achieve great performance. We chose to first split and then use whole attention as one implementation in the experiments. Please follow the code.

from apnet.

Because when use attention, you will lose a lot of information.
If you use whole attention first, and then split attention, the effect may not be as good as first split and then use whole attention , but I haven’t verified it by experiments.This is my idea, what do you think？

from apnet.

Thanks for the good question.

Both of them are in the form of a hieratical pyramid and can achieve great performance. We chose to first split and then use whole attention as one implementation in the experiments. Please follow the code.

from apnet.

I think the attention is not a bad guy which causes the loss of information. Instead, well-trained attention should guide the model to find clues. In my memory, the early experiments may show a similar performance of them in the level 2 APNet (one whole att and one split att).
Besides, I think it is very interesting to explore the order of fine-grained attention and whole attention and corresponding effects, especially with more layers.
Welcome to share your finding with us.

from apnet.

from apnet.