other metrics about multicore-tsne HOT 10 OPEN

I have investigated the implementation more carefully.

VPTree works with any metric distance. And it can work well with angular distance (arccosine distance) as it is pseudo metric.
No need to change a gradient calculation as the new distance function is only appplied in the original space. The distance in the target embedding space is euclidean.

I have implemented several different distance metrics + added lots of comments about implementation in my fork https://github.com/asanakoy/Multicore-TSNE.

from multicore-tsne.

Hi, as far as I know quadtree is only suited for euclidean. You can get cosine for free by normalizing your data first. Other metrics are not supported.

from multicore-tsne.

Now it is possible to easily add new metrics (yet in C++ with recompilation). If you have any preferences I could implement some.

from multicore-tsne.

@DmitryUlyanov why quadtree is only suited for euclidean?
If I just change a distance function when you instantiate VpTree<> from euclidean to cosine will it be sufficient?

from multicore-tsne.

Hi @asanakoy, my statement from 25 June was incorrect.

If I just change a distance function when you instantiate VpTree<> from euclidean to cosine will it be sufficient?

Now I realize that it is not that easy. Distance should also be changed in gradient computation and for now it is hardcoded to be euclidean.

from multicore-tsne.

Well, you still can get cosine similarity by normalizing your data to have unit norm.

from multicore-tsne.

Now I realize that it is not that easy. Distance should also be changed in gradient computation and for now it is hardcoded to be euclidean.

Could you please point out which line of code it is?

from multicore-tsne.

Well, you still can get cosine similarity by normalizing your data to have unit norm.

But this is not exactly the same as the cosine distance.

from multicore-tsne.

I have added not normalized cosine distance (basically it is dot product) in my fork asanakoy/Multicore-TSNE@eebf736

But as you mentioned above it's not fully correct because I forgot to change the gradient computation.

from multicore-tsne.

Could you please point out which line of code it is?

https://github.com/DmitryUlyanov/Multicore-TSNE/blob/master/multicore_tsne/tsne.cpp#L242
https://github.com/DmitryUlyanov/Multicore-TSNE/blob/master/multicore_tsne/tsne.cpp#L313

Maybe there are more.

But this is not exactly the same as the cosine distance.

I see, but if we used squared euclidean distance than it is. Looks easy to change in the code :)

By the way, both cosine distance and squared euclidean are not proper distances. This repo actually uses squared euclidean for VP tree by default and it seem to work well, but maybe there are cases where it breaks the method.

from multicore-tsne.