Training with multiple GPU? about graphsage HOT 4 OPEN

Hi,

We did that in another paper (Graph Convolutional Neural Networks for Web-Scale Recommender Systems).
If you need a quick version, you can simply have each GPU run the model for a different minibatch, do an average of the gradients, and make updates.

Of course you can apply EASGD or other more sophisticated distributed optimization algorithm. It should be no different to doing distributed optimization for other models.

from graphsage.

Thank you for the response!

from graphsage.

Hi William, RexYing
Will the code for Graph Convolutional Neural Networks for Web-Scale Recommender Systems be published? When I try a big graph with 16000 features for each node, I still have the memory problem because features passed into SupervisedGraphsage class are still as a whole, not in a mini-batch style. e.g.

# line 163 of supervised_train.py
 model = SupervisedGraphsage(num_classes, placeholders, 
                features,
                adj_info,
                minibatch.deg,
                layer_infos, 
                model_size=FLAGS.model_size,
                sigmoid_loss = FLAGS.sigmoid,
                identity_dim = FLAGS.identity_dim,
                logging=True)

the parameter features has the shape of [num_samples, nm_features], which will cause the memory error when the graph is huge. Any suggestions for this problem?

from graphsage.

We were unable to make a standalone PinSage code repo since the code is deeply integrated into the SNAP platform that Pinterest used, as well as other pipelines such as LSH and Pixie.

You are right. You need to do re-indexing of the adjacency (only the nodes in the neighborhood of the minibatch of nodes), and extract the features used in the node neighborhood in the minibatch.

The details are in the PinSage paper as well. Let me know if you have difficulty implementing it.

from graphsage.