Thanks for open sourcing your wonderful work, "Learning Attributed Graph Representations with Communicative Message Passing Transformer." I found a potential bug in your in_scores cross-attention computation. It seems problematic when the edge feature matrix is not symmetric.
And by einsum definition, the Eq. (8) and your code seem inconsistent. I could understand the Eq. (8), where the broadcast of node/token embedding for M_i is along the column axis. But your code implementation is not the same as such.