abojchevski / graph_cert Goto Github PK

Hi, I'm trying to run the code through our server. However, I can't find the package gust and not able to install it through pip. Could you provide more detail about the package gust used in your code?

Hi @abojchevski,
I see the model is based on personalized PageRank.
However, is it also applicable to other GNN models? Based on minimizing the logits of the output layer.

Any problems, for example, solving the linear system or the nonlinearity inside the model itself?

Hi,
Really nice work! When I read the paper, the senario of ''Certifiable Robustness'' on graph really confused me. Could we assume that we have the clean (unperturbed) graph data in this senario? If we have access to the clean graph, it seems that there could be some simple tricks to make the model "certifiable robustness". For example, we could force the model to allign its predictions with the unperturbed one.
Please correct me if I made it wrong. Looking forward to your reply. Thanks a lot!

Hi @abojchevski,
When I try the demo of 'adversarial training demo', there is one step to get the init fragile set.
fragile = get_fragile(adj=adj_matrix, threat_model='rem')
However, if I change the threat_model to be add_rem, which represents both add or remove edges operations in the graph, then the output for the ppnp_rce model becomes:

Epoch    0: loss_train: 1.81733, loss_val: 1.81765  acc_train: 0.16667, acc_val: 0.16667 p_robust 0.16667
Epoch    5: loss_train: 1.81652, loss_val: 1.84317  acc_train: 0.70833, acc_val: 0.37500 p_robust 0.04583
Epoch   10: loss_train: 1.79881, loss_val: 1.80250  acc_train: 0.33333, acc_val: 0.22500 p_robust 0.04583
Epoch   15: loss_train: 1.79466, loss_val: 1.79636  acc_train: 0.27500, acc_val: 0.18333 p_robust 0.05833
Epoch   20: loss_train: 1.79322, loss_val: 1.79434  acc_train: 0.16667, acc_val: 0.16667 p_robust 0.16667
Epoch   25: loss_train: 1.79250, loss_val: 1.79307  acc_train: 0.16667, acc_val: 0.16667 p_robust 0.16667
Epoch   30: loss_train: 1.79221, loss_val: 1.79247  acc_train: 0.16667, acc_val: 0.16667 p_robust 0.16667
Epoch   35: loss_train: 1.79200, loss_val: 1.79215  acc_train: 0.16667, acc_val: 0.16667 p_robust 0.16667
...

In contrast, if the thread_model is rem, then the output is

Epoch    0: loss_train: 1.79935, loss_val: 1.80071  acc_train: 0.16667, acc_val: 0.16667 p_robust 0.16667
Epoch    5: loss_train: 0.73769, loss_val: 1.19908  acc_train: 0.87500, acc_val: 0.67500 p_robust 0.74167
Epoch   10: loss_train: 0.23642, loss_val: 1.38974  acc_train: 0.96667, acc_val: 0.65833 p_robust 0.75833
Epoch   15: loss_train: 0.09648, loss_val: 1.97128  acc_train: 0.99167, acc_val: 0.64167 p_robust 0.77917
Epoch   20: loss_train: 0.05001, loss_val: 2.41539  acc_train: 1.00000, acc_val: 0.63333 p_robust 0.77083
Epoch   25: loss_train: 0.02917, loss_val: 2.70049  acc_train: 1.00000, acc_val: 0.63333 p_robust 0.77083
Epoch   30: loss_train: 0.01836, loss_val: 2.83246  acc_train: 1.00000, acc_val: 0.63333 p_robust 0.78333
Epoch   35: loss_train: 0.01293, loss_val: 2.86569  acc_train: 1.00000, acc_val: 0.62500 p_robust 0.7833
...

It seems when both adding and deleting operations are allowed, it is not learning at all.
Only when the initial fragile set is the minimum spanning tree, then it optimizes and improves the robustness?

Correct me if I made it wrong.

Dear author,
I am really struggling with understanding the algorithm 1, why is the policy improvement calculated in that way (line 4 of the Algorithm 1)? could you explain more about it? Thanks very much!