facebookresearch / poincare-embeddings Goto Github PK

View Code? Open in Web Editor NEW

1.7K 1.7K 230.0 404 KB

PyTorch implementation of the NIPS-17 paper "Poincaré Embeddings for Learning Hierarchical Representations"

License: Other

Python 73.90% Shell 2.37% Cython 23.73%

poincare-embeddings's Introduction

Poincaré Embeddings for Learning Hierarchical Representations

PyTorch implementation of Poincaré Embeddings for Learning Hierarchical Representations

Installation

Simply clone this repository via

git clone https://github.com/facebookresearch/poincare-embeddings.git
cd poincare-embeddings
conda env create -f environment.yml
source activate poincare
python setup.py build_ext --inplace

Example: Embedding WordNet Mammals

To embed the transitive closure of the WordNet mammals subtree, first generate the data via

cd wordnet
python transitive_closure.py

This will generate the transitive closure of the full noun hierarchy as well as of the mammals subtree of WordNet.

To embed the mammals subtree in the reconstruction setting (i.e., without missing data), go to the root directory of the project and run

./train-mammals.sh

This shell script includes the appropriate parameter settings for the mammals subtree and saves the trained model as mammals.pth.

An identical script to learn embeddings of the entire noun hierarchy is located at train-nouns.sh. This script contains the hyperparameter setting to reproduce the results for 10-dimensional embeddings of (Nickel & Kiela, 2017). The hyperparameter setting to reproduce the MAP results are provided as comments in the script.

The embeddings are trained via multithreaded async SGD. In the example above, the number of threads is set to a conservative setting (NHTREADS=2) which should run well even on smaller machines. On machines with many cores, increase NTHREADS for faster convergence.

Dependencies

Python 3 with NumPy
PyTorch
Scikit-Learn
NLTK (to generate the WordNet data)

References

If you find this code useful for your research, please cite the following paper in your publication:

@incollection{nickel2017poincare,
  title = {Poincar\'{e} Embeddings for Learning Hierarchical Representations},
  author = {Nickel, Maximilian and Kiela, Douwe},
  booktitle = {Advances in Neural Information Processing Systems 30},
  editor = {I. Guyon and U. V. Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
  pages = {6341--6350},
  year = {2017},
  publisher = {Curran Associates, Inc.},
  url = {http://papers.nips.cc/paper/7213-poincare-embeddings-for-learning-hierarchical-representations.pdf}
}

License

This code is licensed under CC-BY-NC 4.0.

poincare-embeddings's People

Contributors

Stargazers

Watchers

Forkers

alexbw ml-lab kastnerkyle ahoyosid shafiahmed kormilitzin johann-petrak zhouyonglong little1tow awesome-archive btm520 gtesei shubhampachori12110095 saraswat vishalbelsare xc35 yalechang chinmoym2004 jdschmitz hoangcuong2011 tahirahmad2030 tozammel existeundelta gyunt sougata09 mehrdad-shokri sarathknv lanpa gballardin jooooh phucntnii lixiangxu hhy5277 harmanpreet93 mfajcik navidre enggen branidiaz shagunsodhani vikingmew helw150 mindis nd7141 ahug cdmatters yzh119 ottj mittald tanghuil sakhawatsumit simyang0 marcevrard gentlezhu jayparks batermj herdmoon jessielovelife kaushalya rn-v dolphintear zxch3n ai3dvision vickzhang wensiding alexemg wranai afcarl apoorvajha oooo31 kevtimova saxonelf ibulu hyzcn baokunguo namisan ethan-asapp anirband kyoungrok0517 cuptea raldir fducau sungjinlees gxiu chan-y-park arita37 lematt1991 malcolmgreaves sobalgi davidmcdonald1993 masoudmlk gmmarconi kevinking ganow gokul-uf friendshipity skydve yangyang233 wrongwhp skblaz bisratm

poincare-embeddings's Issues

Unable to run example.sh

I'm unable to run example.sh

OS: Ubuntu 16.04, Using Anaconda 4.3.30, Python 3.6.5, Pytorch 0.4.0

Command: NHTHREDS=2 ./example.sh

Error Log:

Using 2 threads
slurp: objects=1181, edges=6541
Indexing data
json_conf: {"distfn": "poincare", "dim": 5, "lr": 0.3, "batchsize": 10, "negs": 50}
Burnin: lr=0.003
'Traceback (most recent call last):\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in <listcomp>\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/harman_workspace/poincare-embeddings/model.py", line 185, in __getitem__\n    if n not in self._weights[t]:\nKeyError: tensor(491)\n'
Traceback (most recent call last):
  File "/home/harman/harman_workspace/poincare-embeddings/train.py", line 19, in train_mp
    train(model, data, optimizer, opt, log, rank, queue)
  File "/home/harman/harman_workspace/poincare-embeddings/train.py", line 46, in train
    for inputs, targets in loader:
  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 286, in __next__
    return self._process_next_batch(batch)
  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in <listcomp>\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/harman_workspace/poincare-embeddings/model.py", line 185, in __getitem__\n    if n not in self._weights[t]:\nKeyError: tensor(491)\n'
'Traceback (most recent call last):\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in <listcomp>\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/harman_workspace/poincare-embeddings/model.py", line 185, in __getitem__\n    if n not in self._weights[t]:\nKeyError: tensor(491)\n'
Traceback (most recent call last):
  File "/home/harman/harman_workspace/poincare-embeddings/train.py", line 19, in train_mp
    train(model, data, optimizer, opt, log, rank, queue)
  File "/home/harman/harman_workspace/poincare-embeddings/train.py", line 46, in train
    for inputs, targets in loader:
  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 286, in __next__
    return self._process_next_batch(batch)
  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/anaconda3/envs/pytorch-venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 57, in <listcomp>\n    samples = collate_fn([dataset[i] for i in batch_indices])\n  File "/home/harman/harman_workspace/poincare-embeddings/model.py", line 185, in __getitem__\n    if n not in self._weights[t]:\nKeyError: tensor(491)\n'

Any help will be appreciated.

Module not found when trying to train model

I am getting the following error after following the installation instructions. Could you please advice how to resolve it?

(poincare) local-poincare-embeddings $./train-mammals.sh Traceback (most recent call last): File "embed.py", line 12, in <module> from hype.adjacency_matrix_dataset import AdjacencyDataset ModuleNotFoundError: No module named 'hype.adjacency_matrix_dataset'

Reconstruction.py gives completely different results from running evaluation

Hello,

I've trained the model using train-nouns.sh, up to 1500 epochs. At the final print-out evaluation during training, performance was as expected: MAP at around .9 and mean rank around 4. However, when evaluating the model using reconstruction.py, on any checkpoint (I removed the recommended line in train.py to save all checkpoints), including the final checkpoint, I get very, very bad results. For example, the results from the final checkpoint are: Mean rank: 44117.63953297465, mAP rank: 0.00015312198997491055.

I investigated the code from train.py and reconstruction.py to see if I could spot what is causing the discrepancy, but I can't see it.

If I run embed.py using the final checkpoint, the first running evaluation matches the final printout from the previous run, so I at least know that the checkpoint file didn't somehow fail to contain the embeddings from the correct epoch or something. But I don't know why reconstruction.py gets different results on the same checkpoint.

Is this behavior expected?

Observing very strong Euclidean baseline results for reconstruction ...

Hi,

I was able to replicate the results for Poincare and Lorentz manifolds as reported in your publications. However, when recreating the Euclidean baselines I am noticing much stronger reconstruction scores. For example, with the following changes to ./train-nouns.sh:

-manifold euclidean
-dims 200
-lr 1.0

After just 200 epochs, we get:

json_stats: {"epoch": 199, ..., "mean_rank": 1.69, "map_rank": 0.90}

And after 1400 epochs, we get:

"mean_rank": 1.19, "map_rank": 0.95

No other changes were made to the code. Are we doing something wrong?
Note that we had to add an entry into the train-nouns.sh script for Euclidean manifold and used the same learning rate as specified for the Poincare manifold (1.0), and not the default of 1000.0 set in the code.

Thanks!

Question on using these embeddings in practice

I wrote this question on stack overflow here... this is not so much an issue with the repo, as it is a question of how to implement these embeddings in practice.

My assumption for how to implement these is to take a sentence, like """The US and UK could agree a “phenomenal' trade deal after Britain leaves the EU.""" , tokenize into a list of synsets [[], Synset('united_states.n.01'), [], Synset('united_kingdom.n.01'), ... ]... but in order to do that, one needs each unique representative synset node of the wordnet, based on the context in which that word lives.

This seems like a pretty difficult aspect of using these embeddings, and I'm wondering what strategies there are to solve this, what literature is available, whether this is totally not how their implemented in practice, or if there are open source projects which can take both the word and the sentence context into account to map a sentence to a list of "optimal" synsets. Seems like this is a critical aspect of using these (and wordnets in general), but not frequently discussed.

BTW many thanks for open-sourcing this, I find this technology really fascinating.

Doubt on normalize() function in Lorentz manifold code

I was re-implementing the Lorentz normalizing function in numpy (I wanted to get rid of the pytorch tensor allocation overhead) and I noticed a weird argument in the function renorm.

Specifically I am referring to renorm_ in this function:

poincare-embeddings/hype/lorentz.py

Lines 49 to 58 in 2e598d2

 def normalize(self, w): 

 """Normalize vector such that it is located on the hyperboloid""" 

 d = w.size(-1) - 1 

 narrowed = w.narrow(-1, 1, d) 

 if self.max_norm: 

 narrowed.view(-1, d).renorm_(p=2, dim=0, maxnorm=self.max_norm) 

 tmp = 1 + th.sum(th.pow(narrowed, 2), dim=-1, keepdim=True) 

 tmp.sqrt_() 

 w.narrow(-1, 0, 1).copy_(tmp) 

 return w

I don't understand the parameter dim=0. This would cause the function to interpret each column as a point, as opposed to the convention in the rest of the code where each point is a tensor row.

Is this function used for a transposed version of the tensor? Or is this possibly a bug?

EDIT: Ok, I found that the convention for dim in renorm_ is different that what I would have expected in numpy. This makes sense. closing the issue.

Lack of hypernymysuite

Hi, I try to use this tool follow the readme file. But the latest version does not include hypernymysuite file. My system return me

"from hypernymysuite.base import HypernymySuiteModel
ModuleNotFoundError: No module named 'hypernymysuite'"

Recreating Link Prediction Results

I'm interested in recreating the results of the link prediction results in the paper.

Could you provide some info on the setup for those experiments as they relate to the code here?

MAP around 0.4 after 300 epochs, far less than results in your paper

Hi, I just train your code on pytorch 0.4.0, on the mammals dataset, with your default hyperparameters, I can just get an embedding of MAP around 0.4 after 300 epochs, which is far more less than your reported results 0.927 in your paper. BTW, I just add 6 lines of code in the form of t= int(t) because the version of pytorch, other codes remain the same. I am wondering how could this happen?

in getitem\n if n not in self._weights[t]:\nKeyError: tensor(143)\n'

mldl@ub1604:/ub16_prj/poincare-embeddings$ NTHREADS=2 ./example.sh
Using 2 threads
slurp: objects=1181, edges=6541
Indexing data
json_conf: {"distfn": "poincare", "dim": 5, "lr": 0.3, "batchsize": 10, "negs": 50}
Burnin: lr=0.003
'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(143)\n'
Traceback (most recent call last):
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 19, in train_mp
train(model, data, optimizer, opt, log, rank, queue)
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 46, in train
for inputs, targets in loader:
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(143)\n'
'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(143)\n'
Traceback (most recent call last):
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 19, in train_mp
train(model, data, optimizer, opt, log, rank, queue)
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 46, in train
for inputs, targets in loader:
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(143)\n'
mldl@ub1604:/ub16_prj/poincare-embeddings$

Using the pth File

Sorry I think this must be a very simple question since it hasn't come up yet in known issues. How is the output in the ".pth" file able to be used? I am interested in being able to get the words' locations in the embedding space and also the distances between words. Thanks so much!

Pretrained embeddings

Did you share results of your embeddings?
If yes, for what model did you share?
If not, can you at least share information on running time for your results?

rgrad return

Why does rgrad return d_p rather than u; what am I missing?

def rgrad(self, p, d_p):
    """Riemannian gradient for hyperboloid"""
    if d_p.is_sparse:
        u = d_p._values()
        x = p.index_select(0, d_p._indices().squeeze())
    else:
        u = d_p
        x = p
    u.narrow(-1, 0, 1).mul_(-1)
    u.addcmul_(self.ldot(x, u, keepdim=True).expand_as(x), x)
    return d_p

Poincare Embedding for Images

I read the paper but I could not understand why Poincare embedding cannot be used for images embedding learning. Images can also have a hierarchical structure eg in the case of fashion images etc. I know this is off topic but I would be really grateful if somebody can help me.

Number of epochs in link prediction results

Any idea or suggested number of epochs for the link prediction experiments, please?
The paper shows good representations after only 20 epochs!

How to work on datasets like wordnet verbs and mesh?

I was looking for ways to run the code on different datasets. The transitive.py code downloads the wordnet dataset, how do I specify it should download the verbs dataset as well?

Thanks in advance!

Unable to find train-mammals.sh as instructed in the README

Please advise as to where I can find the same. The README says it is in the root directory of the project.

Embedding of Tree Structures: root no in the center of the disk

Hi,
I'm dealing with embedding of hierarchical structures (a tree) and I used your algorithm.
It works, but when I visualise the 2D or 3D embedding the root of the tree is not in the center of the disk.
Is there some command or flag to add to the code? Or is the root supposed to be in the center on his own?

Makefile:8: recipe for target 'noun_closure.tsv' failed

Please help me~

Do I miss something? It seems that some files in your Makefile is missing?

mammal_closure.tsv: noun_closure.tsv
	cat noun_closure.tsv | grep -e "\smammal.n.01" | cut -f1 | sed 's/\(.*\)/\^\1/g' > mammals.txt
	cat noun_closure.tsv | grep -f "mammals.txt" | grep -v -f mammals_filter.txt > mammal_closure.tsv

noun_closure.tsv:
$(PYTHON) transitive_closure.py

And I got this error log~

KeyError: 'Traceback

mldl@ub1604:~/ub16_prj/poincare-embeddings$ NTHREADS=2 ./train-nouns.sh
Using 2 threads
slurp: objects=82115, edges=743086
Indexing data
json_conf: {"distfn": "poincare", "dim": 10, "lr": 1, "batchsize": 50, "negs": 50}
Burnin: lr=0.01
'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(23511)\n'
Traceback (most recent call last):
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 19, in train_mp
train(model, data, optimizer, opt, log, rank, queue)
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 46, in train
for inputs, targets in loader:
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(23511)\n'
'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(23511)\n'
Traceback (most recent call last):
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 19, in train_mp
train(model, data, optimizer, opt, log, rank, queue)
File "/home/mldl/ub16_prj/poincare-embeddings/train.py", line 46, in train
for inputs, targets in loader:
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
KeyError: 'Traceback (most recent call last):\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 57, in \n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/mldl/ub16_prj/poincare-embeddings/model.py", line 185, in getitem\n if n not in self._weights[t]:\nKeyError: tensor(23511)\n'

ValueError: Buffer dtype mismatch, expected 'long_t' but got 'long'

Thank you for sharing this great code. However, I encountered one valueError when I am trying to reproduce the train-mammals.sh. The error information can be as follows:

(poincare) C:\Users\DELL\Github\poincare-embeddings-master>sh train-mammals.sh
Specified hogwild training with GPU, defaulting to CPU...
Using edge list dataloader
Traceback (most recent call last):
  File "embed.py", line 246, in <module>
    main()
  File "embed.py", line 147, in main
    manifold, opt, idx, objects, weights, sparse=opt.sparse
  File "C:\Users\DELL\Github\poincare-embeddings-master\hype\sn.py", line 64, in initialize
    opt.ndproc, opt.burnin > 0, opt.dampening)
  File "hype\graph_dataset.pyx", line 75, in hype.graph_dataset.BatchedDataset.__cinit__
    self._mk_weights(idx, weights)
  File "hype\graph_dataset.pyx", line 81, in hype.graph_dataset.BatchedDataset._mk_weights
    def _mk_weights(self, npc.ndarray[npc.long_t, ndim=2] idx, npc.ndarray[npc.double_t, ndim=1] weights):
ValueError: Buffer dtype mismatch, expected 'long_t' but got 'long'

I am using pytorch 1.0 with cuda 10.0 on windows:

(poincare) C:\Users\DELL\Github\poincare-embeddings-master>python -c "import torch; print(torch.version.cuda)"
10.0
(poincare) C:\Users\DELL\Github\poincare-embeddings-master>nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:04_Central_Daylight_Time_2018
Cuda compilation tools, release 10.0, V10.0.130

I found that the problem comes from earlier steps python setup.py build_ext --inplace. When I run python setup.py build_ext --inplace, it shows the following information:

(poincare) C:\Users\DELL\Github\poincare-embeddings-master>python setup.py build_ext --inplace
Compiling hype/graph_dataset.pyx because it depends on C:\Anaconda3\envs\poincare\lib\site-packages\Cython\Includes\numpy\__init__.pxd.
Compiling hype/adjacency_matrix_dataset.pyx because it depends on C:\Anaconda3\envs\poincare\lib\site-packages\Cython\Includes\numpy\__init__.pxd.
[1/2] Cythonizing hype/adjacency_matrix_dataset.pyx
C:\Anaconda3\envs\poincare\lib\site-packages\Cython\Compiler\Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\DELL\Github\poincare-embeddings-master\hype\adjacency_matrix_dataset.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
[2/2] Cythonizing hype/graph_dataset.pyx
C:\Anaconda3\envs\poincare\lib\site-packages\Cython\Compiler\Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\DELL\Github\poincare-embeddings-master\hype\graph_dataset.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'hype.graph_dataset' extension
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Anaconda3\envs\poincare\lib\site-packages\numpy\core\include -IC:\Anaconda3\envs\poincare\include -IC:\Anaconda3\envs\poincare\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" /EHsc /Tphype/graph_dataset.cpp /Fobuild\temp.win-amd64-3.6\Release\hype/graph_dataset.obj -std=c++11
cl: 命令行 warning D9002 :忽略未知选项“-std=c++11”
graph_dataset.cpp
c:\anaconda3\envs\poincare\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
hype/graph_dataset.cpp(3033): warning C4244: “=”: 从“Py_ssize_t”转换到“int”，可能丢失数据
hype/graph_dataset.cpp(3418): warning C4244: “=”: 从“__pyx_t_5numpy_long_t”转换到“long”，可能丢失数据
hype/graph_dataset.cpp(3429): warning C4244: “=”: 从“__pyx_t_5numpy_long_t”转换到“long”，可能丢失数据
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Anaconda3\envs\poincare\libs /LIBPATH:C:\Anaconda3\envs\poincare\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64" /EXPORT:PyInit_graph_dataset build\temp.win-amd64-3.6\Release\hype/graph_dataset.obj /OUT:C:\Users\DELL\Github\poincare-embeddings-master\hype\graph_dataset.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\hype\graph_dataset.cp36-win_amd64.lib
  正在创建库 build\temp.win-amd64-3.6\Release\hype\graph_dataset.cp36-win_amd64.lib 和对象 build\temp.win-amd64-3.6\Release\hype\graph_dataset.cp36-win_amd64.exp
正在生成代码
已完成代码的生成
building 'hype.adjacency_matrix_dataset' extension
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Anaconda3\envs\poincare\lib\site-packages\numpy\core\include -IC:\Anaconda3\envs\poincare\include -IC:\Anaconda3\envs\poincare\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" /EHsc /Tphype/adjacency_matrix_dataset.cpp /Fobuild\temp.win-amd64-3.6\Release\hype/adjacency_matrix_dataset.obj -std=c++11
cl: 命令行 warning D9002 :忽略未知选项“-std=c++11”
adjacency_matrix_dataset.cpp
c:\anaconda3\envs\poincare\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
hype/adjacency_matrix_dataset.cpp(3144): warning C4244: “=”: 从“Py_ssize_t”转换到“int”，可能丢失数据
hype/adjacency_matrix_dataset.cpp(5484): warning C4244: “=”: 从“Py_ssize_t”转换到“long”，可能丢失数据
hype/adjacency_matrix_dataset.cpp(5595): warning C4244: “=”: 从“Py_ssize_t”转换到“long”，可能丢失数据
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Anaconda3\envs\poincare\libs /LIBPATH:C:\Anaconda3\envs\poincare\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64" /EXPORT:PyInit_adjacency_matrix_dataset build\temp.win-amd64-3.6\Release\hype/adjacency_matrix_dataset.obj /OUT:C:\Users\DELL\Github\poincare-embeddings-master\hype\adjacency_matrix_dataset.cp36-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.6\Release\hype\adjacency_matrix_dataset.cp36-win_amd64.lib
  正在创建库 build\temp.win-amd64-3.6\Release\hype\adjacency_matrix_dataset.cp36-win_amd64.lib 和对象 build\temp.win-amd64-3.6\Release\hype\adjacency_matrix_dataset.cp36-win_amd64.exp
正在生成代码
已完成代码的生成

Your suggestion are welcome. Thanks.

How to use the saved model to get the distance between 2 entities?

Settings to reproduce Euclidean and Translational in Table 1

First I'd like to thank you for sharing the code. While running your code, I got some questions on the models you compared against the poincaré model.

May I ask what hyperparameter settings were used to produce Euclidean and Translational models reported in Table 1 of Nickel & Kiela (2017)? Especially, I wonder what was the number of epochs used for the two models.

Problem of Poincare Distance and Norm

Hi, I trained a 100 dimension model with train-nouns.sh in poincare manifold. I tested the model by calculating the tensor.norm of some nodes and poincare.distance between some nodes.

However, for Gauss, Brain, Blue and many other nodes near to the edge, the norm of them become 1.0000, which means infinity in poincare ball. In addition, if I calculate the poincare distance between these nodes near to the edge and other nodes, the distance will be always 12.2061, which is a little confusing.

Thanks for your attention!

When I run "python setup.py build_ext --inplace", some problems occur

Traceback (most recent call last):
File "setup.py", line 21, in
out = check_output([CC, '--version'])
File "C:\ProgramData\Anaconda3\envs\poincare\lib\subprocess.py", line 356, in check_output
**kwargs).stdout
File "C:\ProgramData\Anaconda3\envs\poincare\lib\subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\ProgramData\Anaconda3\envs\poincare\lib\subprocess.py", line 729, in init
restore_signals, start_new_session)
File "C:\ProgramData\Anaconda3\envs\poincare\lib\subprocess.py", line 1017, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] 系统找不到指定的文件。

reconstruction.py does not handle CSV data input

The transitive_closure.py script creates CSV files for both nouns and mammals sets in WordNet. These are used to train the hyperbolic embeddings in embed.py.

If we run the reconstruction.py script on a pretrained model it fails as currently the script only accepts HDF5 input:

poincare-embeddings/reconstruction.py

Line 41 in 61406b1

format = 'hdf5' if dset.endswith('.h5') else 'csv'

poincare-embeddings/reconstruction.py

Line 42 in 61406b1

dset = load_adjacency_matrix(dset, 'hdf5')

format = 'hdf5' if dset.endswith('.h5') else 'csv'
dset = load_adjacency_matrix(dset, 'hdf5')

The format 'hdf5' is hard coded in line 42.

In addition, the function being called: load_adjacency_matrix expects an adjacency matrix to be passed as an argument, whereas [noun,mammal]_closure.csv are both a list of edges.
To fix this we borrowed code from embed.py:

if dset.endswith('.h5'):
   # existing code (mostly) ...
elif dset.endswith('.csv'):
    adj_temp = {}
    idx, _, _ = load_edge_list(dset, sym)
    for row in idx:
        x = row[0].item()
        y = row[1].item()
        if x in adj_temp:
            adj_temp[x].add(y)
        else:
            adj_temp[x] = {y}
    sample_size = args.sample or len(adj_temp)
    sample = np.random.choice(list(adj_temp.keys()), size=sample_size, replace=False)
    adj = {i: adj_temp[i] for i in sample}
else:
    # do something ...

With this change, we get the same mean and MAP rank as calculated and printed in the training loop.

Sample Code for Link Prediction?

Hi!
Excellent work, I was wondering if there are plans to share a sample code for setting up link prediction experiments? Also for visualising the embeddings.

Thanks!

There is something wrong with transitive_closure.py

I have trouble when running as the readme suggests transitive_closure.py. Please help me figure it out. Thanks:
(poincare) jiajunhuang@jiajunhuang-Ubantu01:~/poincare-embeddings/wordnet$ python transitive_closure.py
Traceback (most recent call last):
File "transitive_closure.py", line 14, in
wn.all_synsets
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/corpus/util.py", line 123, in getattr
self.__load()
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/corpus/util.py", line 83, in __load
root = nltk.data.find('{}/{}'.format(self.subdir, self.__name))
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/data.py", line 673, in find
return find(modified_name, paths)
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/data.py", line 660, in find
return ZipFilePathPointer(p, zipentry)
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/compat.py", line 228, in _decorator
return init_func(*args, **kwargs)
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/data.py", line 506, in init
zipfile = OpenOnDemandZipFile(os.path.abspath(zipfile))
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/compat.py", line 228, in _decorator
return init_func(*args, **kwargs)
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/site-packages/nltk/data.py", line 1055, in init
zipfile.ZipFile.init(self, filename)
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/zipfile.py", line 1131, in init
self._RealGetContents()
File "/home/jiajunhuang/anaconda3/envs/poincare/lib/python3.6/zipfile.py", line 1198, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

Is the network embedding loss included?

Hi,

I am trying to figure out what the exact loss used in for the network embeddings (Section 4.2 in the paper). However, after reading the code, I find only one loss is used (defined in the SNEmbedding class) and it is a CrossEntropyLoss corresponding to the Eq. (5) in the paper. I wonder whether I miss anything or this repo just doesn't contain the code for network embedding part? If it's the latter case, would you mind revealing the exact loss used for network embedding? My current guess for that loss will be the one that replaces e^{-d(u,v)} with P((u,v)=1|\Theta) in Eq. (5). Am I right?

P.S.: I am referring to the NIPS version of paper.

Thanks.

Using Adjacency Matrix Data Set

I would like to use the code for my own datasets. However, my data sets are graph networks and I am not sure how to the format the hdf5 file to work with your code.

I imagine 'ids' is an array of the node 'ids' and 'weights' is a matrix for the edge weights.

Is 'neighbors' the adjacency matrix? I don't know what 'offsets' and 'objects' should be.

Thank you for help.

Minor typo (distfn) in embed.py

The line

poincare-embeddings/embed.py

Line 136 in 35c2322

elif opt.distf == 'euclidean':

should be
elif opt.distfn == 'euclidean':

Unable to use my own data

I'm trying to use my own data in tsv format but I have a problem:

The console returns me this message:

"./poincare-embeddings/model.py", line 156, in init assert len(objects) == nents, 'Number of objects do no match'
AssertionError: Number of objects do no match

I'd like to know what is the meaning of this assertion . Even because if I put a comment on the line, everything works.

Thanks

NaNs when training with the provided hyperparameters

Hi,

I am having trouble training the WordNet noun's model with the provided hyperparameters (in train-nouns.sh). Here is the state of the repo:

ubuntu@hyperbolic-words-2:~/poincare-embeddings$ git diff
diff --git a/train-nouns.sh b/train-nouns.sh
index c106f4a..df4118f 100755
--- a/train-nouns.sh
+++ b/train-nouns.sh
@@ -2,7 +2,7 @@

 # Get number of threads from environment or set to default
 if [ -z "$NTHREADS" ]; then
-   NTHREADS=5
+   NTHREADS=64
 fi

 echo "Using $NTHREADS threads"

And I am running the script using nohup ./train-nouns.sh &. As can be seen from the output below, there are NaNs from epoch 84 onwards.

ubuntu@hyperbolic-words-2:~/poincare-embeddings$ cat nohup.out
Using 64 threads
slurp: objects=82115, edges=743086
Indexing data
json_conf: {"distfn": "poincare", "dim": 10, "lr": 1, "batchsize": 50, "negs": 50}
Burnin: lr=0.01
json_log: {"epoch": 0, "loss": 1.7510217754905981, "elapsed": 116.03712247902877}
Burnin: lr=0.01
json_log: {"epoch": 1, "loss": 1.654937060126994, "elapsed": 118.86135302501498}
Burnin: lr=0.01
json_log: {"epoch": 2, "loss": 1.5647581015296042, "elapsed": 118.4134448269906}
Burnin: lr=0.01
json_log: {"epoch": 3, "loss": 1.4822902187269897, "elapsed": 117.75066023998079}
Burnin: lr=0.01
json_log: {"epoch": 4, "loss": 1.4054055172298818, "elapsed": 119.41065871602041}
Burnin: lr=0.01
json_log: {"epoch": 5, "loss": 1.3346420868443154, "elapsed": 119.11423735501012}
Burnin: lr=0.01
json_log: {"epoch": 6, "loss": 1.2689210829544222, "elapsed": 118.58688614101266}
Burnin: lr=0.01
json_log: {"epoch": 7, "loss": 1.2099971524190973, "elapsed": 118.22312185697956}
Burnin: lr=0.01
json_log: {"epoch": 8, "loss": 1.1548116808904323, "elapsed": 118.3734846469888}
Burnin: lr=0.01
json_log: {"epoch": 9, "loss": 1.1028053120125545, "elapsed": 119.84133693398326}
Burnin: lr=0.01
json_log: {"epoch": 10, "loss": 1.0538397419021133, "elapsed": 118.92894380999496}
Burnin: lr=0.01
json_log: {"epoch": 11, "loss": 1.0086800653686467, "elapsed": 118.51857428599033}
Burnin: lr=0.01
json_log: {"epoch": 12, "loss": 0.9661437821506956, "elapsed": 118.79905583200161}
Burnin: lr=0.01
json_log: {"epoch": 13, "loss": 0.9271851564789986, "elapsed": 119.18357229899266}
Burnin: lr=0.01
json_log: {"epoch": 14, "loss": 0.8898532564651916, "elapsed": 118.5679669189849}
Burnin: lr=0.01
json_log: {"epoch": 15, "loss": 0.8561349531560194, "elapsed": 118.50844964102725}
Burnin: lr=0.01
json_log: {"epoch": 16, "loss": 0.8238338795120215, "elapsed": 119.0086042000039}
Burnin: lr=0.01
json_log: {"epoch": 17, "loss": 0.7933640109146769, "elapsed": 118.66447035799501}
Burnin: lr=0.01
json_log: {"epoch": 18, "loss": 0.7658562077606146, "elapsed": 118.73805539900786}
Burnin: lr=0.01
json_log: {"epoch": 19, "loss": 0.7403765493110794, "elapsed": 118.37916503899032}
json_log: {"epoch": 20, "loss": 2.370282818574938, "elapsed": 377.01101255300455}
json_log: {"epoch": 21, "loss": 1.332919087618401, "elapsed": 373.0545672760054}
json_log: {"epoch": 22, "loss": 1.1209379775260888, "elapsed": 373.76439789202414}
json_log: {"epoch": 23, "loss": 0.9983822026230719, "elapsed": 372.82547582898405}
json_log: {"epoch": 24, "loss": 0.9136303973428944, "elapsed": 374.4872254100046}
json_log: {"epoch": 25, "loss": 0.853509014305552, "elapsed": 372.2331004269945}
json_log: {"epoch": 26, "loss": 0.8096088365475078, "elapsed": 373.190142884996}
json_log: {"epoch": 27, "loss": 0.7833678124650717, "elapsed": 364.4548651619989}
json_log: {"epoch": 28, "loss": 0.7738845424189945, "elapsed": 362.44732139699045}
json_log: {"epoch": 29, "loss": 0.7732465087031701, "elapsed": 363.7966738270188}
json_log: {"epoch": 30, "loss": 0.7745720795530572, "elapsed": 365.75540012700367}
json_log: {"epoch": 31, "loss": 0.7754874658808333, "elapsed": 364.4030886440014}
json_log: {"epoch": 32, "loss": 0.7791964033661933, "elapsed": 365.59433861100115}
json_log: {"epoch": 33, "loss": 0.7783978579808019, "elapsed": 363.1505782940076}
json_log: {"epoch": 34, "loss": 0.7788809554177886, "elapsed": 366.6854563309753}
json_log: {"epoch": 35, "loss": 0.7807947874874831, "elapsed": 362.40433710900834}
json_log: {"epoch": 36, "loss": 0.7802273132815123, "elapsed": 363.4012530000182}
json_log: {"epoch": 37, "loss": 0.7821110229706861, "elapsed": 363.1637753259856}
json_log: {"epoch": 38, "loss": 0.7834545301763879, "elapsed": 362.778968936007}
json_log: {"epoch": 39, "loss": 0.7821672492099232, "elapsed": 365.47163956501754}
json_log: {"epoch": 40, "loss": 0.7812606035833877, "elapsed": 372.4652839000046}
json_log: {"epoch": 41, "loss": 0.7831733638117153, "elapsed": 371.5480394010083}
json_log: {"epoch": 42, "loss": 0.7805352591396997, "elapsed": 365.6498189200065}
json_log: {"epoch": 43, "loss": 0.7841648086514452, "elapsed": 366.00326509500155}
json_log: {"epoch": 44, "loss": 0.7817570621436887, "elapsed": 364.9565300550021}
json_log: {"epoch": 45, "loss": 0.7792914231417969, "elapsed": 364.8312062040204}
json_log: {"epoch": 46, "loss": 0.7813605962990672, "elapsed": 364.26285549401655}
json_log: {"epoch": 47, "loss": 0.7832389085209808, "elapsed": 364.54356124799233}
json_log: {"epoch": 48, "loss": 0.7828953792766546, "elapsed": 362.8009843980253}
json_log: {"epoch": 49, "loss": 0.7830643432524169, "elapsed": 364.27069370500976}
json_log: {"epoch": 50, "loss": 0.7832194386006741, "elapsed": 362.9516918490117}
json_log: {"epoch": 51, "loss": 0.7803435126326296, "elapsed": 365.32360184399295}
json_log: {"epoch": 52, "loss": 0.7814182463081865, "elapsed": 364.3582421159954}
json_log: {"epoch": 53, "loss": 0.7808202591523791, "elapsed": 364.49702233899734}
json_log: {"epoch": 54, "loss": 0.7814219246435845, "elapsed": 366.30092610701104}
json_log: {"epoch": 55, "loss": 0.7812639302362918, "elapsed": 364.321928276011}
json_log: {"epoch": 56, "loss": 0.7805546298960221, "elapsed": 365.4223571420007}
json_log: {"epoch": 57, "loss": 0.7827168783831445, "elapsed": 369.57942718299455}
json_log: {"epoch": 58, "loss": 0.7826227238666216, "elapsed": 370.0409598569968}
json_log: {"epoch": 59, "loss": 0.7840471950221404, "elapsed": 369.9050895939872}
json_log: {"epoch": 60, "loss": 0.7821681343282576, "elapsed": 372.4341457049886}
json_log: {"epoch": 61, "loss": 0.7791859689612924, "elapsed": 370.6871205380012}
json_log: {"epoch": 62, "loss": 0.7812333104954267, "elapsed": 367.2430337770202}
json_log: {"epoch": 63, "loss": 0.7819590478411862, "elapsed": 371.23542637602077}
json_log: {"epoch": 64, "loss": 0.7797660082275362, "elapsed": 369.08145095501095}
json_log: {"epoch": 65, "loss": 0.7808701695771313, "elapsed": 371.47186528297607}
json_log: {"epoch": 66, "loss": 0.7825124721070259, "elapsed": 370.3029446750006}
json_log: {"epoch": 67, "loss": 0.7822453611017892, "elapsed": 370.0708697150112}
json_log: {"epoch": 68, "loss": 0.781655567370507, "elapsed": 372.9709478849836}
json_log: {"epoch": 69, "loss": 0.7807679479705355, "elapsed": 369.12473262500134}
json_log: {"epoch": 70, "loss": 0.7785301718368571, "elapsed": 367.71985385299195}
json_log: {"epoch": 71, "loss": 0.7821944001183523, "elapsed": 368.5803866839851}
json_log: {"epoch": 72, "loss": 0.7792014827696869, "elapsed": 369.225650538021}
json_log: {"epoch": 73, "loss": 0.7821118975729462, "elapsed": 370.6435987050063}
json_log: {"epoch": 74, "loss": 0.7816132332723827, "elapsed": 371.09823266702006}
json_log: {"epoch": 75, "loss": 0.780557655646636, "elapsed": 370.9333890530106}
json_log: {"epoch": 76, "loss": 0.7819239331816404, "elapsed": 368.4207367969793}
json_log: {"epoch": 77, "loss": 0.7809382741582769, "elapsed": 369.501590519998}
json_log: {"epoch": 78, "loss": 0.7819559281778887, "elapsed": 369.5081132330233}
json_log: {"epoch": 79, "loss": 0.7785417012889864, "elapsed": 369.03904611899634}
json_log: {"epoch": 80, "loss": 0.7799045569593641, "elapsed": 370.648248100013}
json_log: {"epoch": 81, "loss": 0.7813012339836487, "elapsed": 368.1522722489899}
json_log: {"epoch": 82, "loss": 0.7809676399123354, "elapsed": 368.78767832298763}
json_log: {"epoch": 83, "loss": 0.7801517064626365, "elapsed": 372.1423245120095}
json_log: {"epoch": 84, "loss": nan, "elapsed": 365.5702549460111}
json_log: {"epoch": 85, "loss": nan, "elapsed": 363.09633759901044}
json_log: {"epoch": 86, "loss": nan, "elapsed": 363.81353292398853}
json_log: {"epoch": 87, "loss": nan, "elapsed": 362.3099420409999}
json_log: {"epoch": 88, "loss": nan, "elapsed": 364.5262840819778}
json_log: {"epoch": 89, "loss": nan, "elapsed": 361.63322074498865}
json_log: {"epoch": 90, "loss": nan, "elapsed": 364.6625888540002}
json_log: {"epoch": 91, "loss": nan, "elapsed": 364.0965977109736}
json_log: {"epoch": 92, "loss": nan, "elapsed": 364.00835764498333}
json_log: {"epoch": 93, "loss": nan, "elapsed": 362.5855255270144}
json_log: {"epoch": 94, "loss": nan, "elapsed": 363.579538193997}
json_log: {"epoch": 95, "loss": nan, "elapsed": 362.10383523098426}
json_log: {"epoch": 96, "loss": nan, "elapsed": 365.4729586149915}
json_log: {"epoch": 97, "loss": nan, "elapsed": 364.0538249380188}
json_log: {"epoch": 98, "loss": nan, "elapsed": 362.8323765830137}
Process Process-66:
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/ubuntu/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "embed.py", line 68, in control
    mrank, mAP = ranking(types, model, distfn)
  File "embed.py", line 38, in ranking
    ap_scores.append(average_precision_score(_labels, -_dists))
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 199, in average_precision_score
    sample_weight=sample_weight)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/metrics/base.py", line 75, in _average_binary_score
    return binary_metric(y_true, y_score, sample_weight=sample_weight)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 191, in _binary_uninterpolated_average_precision
    y_true, y_score, sample_weight=sample_weight)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 441, in precision_recall_curve
    sample_weight=sample_weight)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/metrics/ranking.py", line 324, in _binary_clf_curve
    assert_all_finite(y_score)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py", line 54, in assert_all_finite
    _assert_all_finite(X.data if sp.issparse(X) else X)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/sklearn/utils/validation.py", line 44, in _assert_all_finite
    " or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float32').
...

Accelerate by using GPU?

Could the code easily be adapted to run on GPU to accelerate training?

How do you create the visualization of embeddings in hyperbolic space: wn-nouns.jpg?

“train-nouns.py” stops

Hi, I am wondering is it normal for “train-nouns.py” to stop for a relatively long time after many epochs. I have run it for nearly 1 day and it does not have any outputs now.

How to choose the best model

Hi. When I finished running train-nouns.sh, nouns.bin.99 ~ nouns.bin.1499 were generated. I am wondering if nouns.bin.1499 is the best model? If not, how can we decide which is the best?

Euclidean embedding

In the code, options for only poincare and lorentzian embedding are set out. However it looks like EuclideanManifold has been implemented. Is there any reason why I should not do "-manifold euclidean" (asking since it's not provided as one of the options). Also any recommendation for learning rate to use in this case? thanks

Evaluation benchmark and WordNet Reconstruction code

Hello,
"A Poincaré embedding with d = 5 achieves mean rank 1.26 and MAP 0.927 on this." After how many epochs do you arrive at these evaluation metrics? Also there is no information on how to run reconstruction.py in the readme.

How to get vector for a node?

After training the model, how can we get the vector for the concepts we have embedded?
Like the Gensim word2vec library where we could easily call a function like model.getvector("Man").

Direct in readme the required packages

Install would be easier if the required packages were included in the installation - nltk, tqdm, Cython, torch. Just a suggestion!

Please provide example code for network embedding (experiment 2)

I cannot find it so I implement it according to your paper. I find MAP metric is always around 0.6, and I cannot figure out what is wrong.

Please help me.

giving 2-d poincare embeddings, which tool can we use to get a good looking visualization like this one.

The nice visualization given in the repository is,

Does anybody know which tool can we use to create such a figure if we already have the 2D Poincare embeddings?

Poincare problem

Hi, when I change the manifold by using ./train-nouns.py -m poincare, it cannot run anymore. Does anyone have met the same problem?

Unexplained DataLoader output

I tried running the example provided.
In the file "train.py" and specifically in the function train each instance of the data matrix has a shape of (1,52).
This is good since each instance has
in the first index the node we focus on,
in the second index a true neighboring node,
and the next 50 are random nodes.

On the other hand in this line :
for inputs, targets in loader:
we get batches of size (batch_size,7) and i understand that each item in these vectors corresponds to an index in the embedding matrix (and a node as well).

If these 7 numbers are the nodes why don't we get 52 as in the data matrix ?
If these numbers are not the candidate neighbors then could you explain what are they ?

Thank you in advance!

Question about evaluation: mean rank and mAP

Hi, I am new to this task and want to know the relation between mean rank and mAP.
I am reproducing the result of link prediction task and trained TransE model as well as Poincare model.
When I evaluation those two models, I found that TransE may get higher mean rank and higher MAP ,but Poincare may get lower mean rank and lower MAP.
Should it always be lower mean rank and higher MAP?
Are there some differences between those two ways of evaluation?
Or maybe there is something wrong with my evaluation code :(

Settings to reproduce results from the paper

Hi,

Thanks for releasing the code.

Can you also provide the hyper-parameter settings for reproducing results in Table 1 in the paper (specifically -lr, -negs, -batchsize, -burnin)? The example script is only for the mammals subset. Are these the same across all the experiments described in the paper?

Thanks,
Bhuwan

Reproducing embedding for network data

I was trying to reproduce MAP for reconstruction on network data and was unable to get anything better than 0.55. Will it be possible to share hyper parameters used to generate the reported data.

Thanks,
Mayank

How can I get result for each epoch?

Hi.

I know little about multiprocessing.
I wonder why there is only one output checkpoint while the 205th line in embed.py save the result for each epoch.

I want to visualize how the embedding changes during training.
How can I get the embeddings after each epoch?

Thank you.

Cannot import hypernimysuite

Hi,

I cloned the repository and try to replicate the experiment but the embed.py returns me an error that say that cannot import hypernymysuite

I checked your other repositories and I found: https://github.com/facebookresearch/hypernymysuite, I cloned and copied the folder hypernymysuite into poincare-embeddings folder and I solved the problem (I think)

This issue is opened only to know if I solved correctly the problem.