tech-srl / how_attentive_are_gats Goto Github PK

Code for the paper "How Attentive are Graph Attention Networks?" (ICLR'2022)

Python 96.97% Shell 3.03%

how attentive are graph attention networks graph-neural-networks pytorch graph-attention-networks gat

how_attentive_are_gats's Issues

Reproduce the GAT v1 attention matrix

Thanks for your great contribution!!
I'm confused about Figure 1 (a) in your paper. Which layer of GAT is this attention matrix in? Is the attention matrix of all layers the same? Is the attention matrix between different heads in one layer like this?

Best regards

(Possible) bug when graph.is_block is True

Sorry for bothering again.

This might be dumb, but I am reading the DGL implementation to understand it, and I really don't get L272.

how_attentive_are_gats/gatv2_conv_DGL.py

Lines 266 to 272 in b5ccb61

 if self.share_weights: 

 feat_dst = feat_src 

 else: 

 feat_dst = self.fc_dst(h_src).view( 

 -1, self._num_heads, self._out_feats) 

 if graph.is_block: 

 feat_dst = feat_src[:graph.number_of_dst_nodes()]

The only thing that makes sense to me is that the rhs was supposed to be feat_dst as well, and this is some bug. Otherwise, it does not make sense to set share_weights=False when graph.is_block is True. Am I missing something?

Publishing code for reproduction

Thank you for your great contribution!
Currently, we are doing research with GAT, GAT-V2 (and Graph Transformers) and try to reproduce your results on the ogbn-datasets.
Do you already know when you will publish the rest of the code?

Best regards

Reproducing results on proteins

Hi!

I was trying to reproduce the results on the proteins dataset on a V100 GPU, and I am running into a few problems. First I had some issues with the BatchSampler (samplers cannot be passed to iterative datasets), so I just removed it (I am using the latest version of dgl, since 0.6 is not available with my CUDA version).

After fixing that, I came across this error message
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 64])
in line 98 of models.py

how_attentive_are_gats/proteins/models.py

Lines 90 to 100 in b5ccb61

 for i in range(self.n_layers): 

 h = self.convs[i](subgraphs[i], h).flatten(1, -1) 

 if h_last is not None: 

 h += h_last[: h.shape[0], :] 

 h_last = h 

 h = self.norms[i](h) 

 h = self.activation(h, inplace=True) 

 h = self.dropout(h)

Is this error related with removing the BatchSampler? And which tensor size is it expected at that line?

Thanks for the help

I guess the attention heads are not being aggregated? I noticed the DGL version does not have an is_concat option like in the annotated code demo

tech-srl / how_attentive_are_gats Goto Github PK

how_attentive_are_gats's Issues

Reproduce the GAT v1 attention matrix

(Possible) bug when graph.is_block is True

Publishing code for reproduction

Reproducing results on proteins

Why the performance of GAT in paper is so low?

How can I draw the image inside the Figure 1?

Number of features does not match the number of nodes when num_heads > 1

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	if self.share_weights:
	feat_dst = feat_src
	else:
	feat_dst = self.fc_dst(h_src).view(
	-1, self._num_heads, self._out_feats)
	if graph.is_block:
	feat_dst = feat_src[:graph.number_of_dst_nodes()]

	for i in range(self.n_layers):
	h = self.convs[i](subgraphs[i], h).flatten(1, -1)

	if h_last is not None:
	h += h_last[: h.shape[0], :]

	h_last = h

	h = self.norms[i](h)
	h = self.activation(h, inplace=True)
	h = self.dropout(h)