Forming sentence embedding about pytorch-biggraph HOT 4 CLOSED

I believe you are confused about what a graph embedding is. The objective isn't to assign one vector to the whole graph, but rather to learn a vector for each vertex. PBG does not provide any way of aggregating the embedding of different nodes in order to obtain a single embedding for the whole graph. This is not a goal.

You may have been induced into confusion by graph neural networks, which are quite a different thing. To try to highlight the difference:

• PBG (and models like it) can be seen as an algorithm that takes as input a graph structure (just the topology, vertices and edges, no features or labels about them) and trains (in an unsupervised way) a very large model where the embeddings of the vertices are the parameters of the model and the output of the model are scores between pairs of vertices.
• Graph neural networks (GNNs) are algorithms that take as input a set of data points (each with some features and labels) which happen to be connected by a graph structure, and they train (in a supervised way) a rather small model (usually convolutional) whose output is a prediction of the labels of the vertices. The embedding of a node, in this case, is usually the activation values of an intermediate layer of the network when run on that node. (Of course I had to simplify quite a bit here).

If you need more information you should read our documentation (this page and this page) and perhaps take a look at @adamlerer's talk at SysML, which you can find here.

from pytorch-biggraph.

@lerks is correct, PBG takes as input a graph and outputs embeddings of the nodes; it does not construct an embedding of a graph. I will point out though that the terminology is a little overloaded because there are other methods that construct embeddings of graphs (often for things like embeddings of protein molecules) - they often work by doing our kind of graph embedding and then aggregating the node embeddings in some way, but there are other approaches.

Anyway, I agree that PBG is not the best approach for this problem. AFAIK the state of the art for sentence embeddings would be BERT(one example I found by Google search is https://github.com/lonePatient/bert-sentence-similarity-pytorch )

from pytorch-biggraph.

Hi @Arjunsankarlal , i think what are you saying can make sense if you are using a shallow encoding approach such as using SkipGram to encode the structure of a graph (DeepWalk) .
with this approach , nodes can be seen as words and sub-graphs (set of nodes) as sentences (set of words) .
You can even train doc2vec model to encode sub-graphs as sentence embedding .
I might add to what @lerks mentioned , GNNs can be unsupervised too and the features could be just the hot-encoding of the nodes ( GraphSage for example ) .

from pytorch-biggraph.

I'm closing this as the OP has acknowledged the recent posts but there has been no further activity for almost two weeks. Feel free to reopen.

from pytorch-biggraph.