Publishing code for reproduction about how_attentive_are_gats HOT 5 CLOSED

Hi @LeadBeetle,
Thank you for your interest in our work!

We are in the process of organizing and refactoring our code and hope to release everything by the end of September.
We will send a comment in this thread once the entire code is public.

Best,
Shaked

from how_attentive_are_gats.

Hi @shakedbr,

alright, thanks for your fast reply.
In the GAT-V2 paper, you are mentioning some parameters of your trained models in the appendix. Could you elaborate on the exact configuration of your models (i.e. dropout or the use of bag of tricks)? With the parameters you mentioned in the paper we run into overfitting and slow convergence as we try to reproduce your accuracy scores.

Best regards

from how_attentive_are_gats.

Hi @LeadBeetle,

Here are the hyperparameters we used for the ogbn datasets.
For full reproducibility of our experiments, please wait for the full code release.

ogbn-arxiv:

num_layers 3                  
batch_size 20000
hidden_channels 256
dropout 0.25
lr 0.01
epochs 50
use_layer_norm
use_residual
use_saint
saint_num_steps 30
saint_walk_length 3

ogbn-mag:

num_layers 2     
batch_size 20000
hidden_channels 256
dropout 0.5
lr 0.01
epochs 100
use_layer_norm
use_residual
use_neighbor_sample

ogbn-products:

num_layers 3     
batch_size 256
hidden_channels 128
dropout 0.5
lr 0.001
epochs 100
use_residual_with_linear
use_neighbor_sample

ogbn-proteins:

num_layers 6     
batch_size (len(train) + 9) // 10
hidden_channels 64
dropout 0.25
lr 0.01
epochs 1200
use_batch_norm
use_residual
use_neighbor_sample

Best,
Shaked

from how_attentive_are_gats.

Many thanks to you!

from how_attentive_are_gats.

Hi @LeadBeetle

The code for reproduction is now available!

from how_attentive_are_gats.