Will there be a version for Windows? about multicore-tsne HOT 10 CLOSED

Hopefully I can save someone some time trying to get this to work on windows.

1. Make sure all libraries and links you're going to be using are the same architecture. If you have a 64bit python, you have to have mingw64, llvm 64bit, cmake 64bit, etc. I'm using 64bit but even with careful installations, the linking process still managed to find 32bit libraries to link to; more on that in a minute.

2. Change the cmake line in the setup.py file to:
   os.system('cmake -DCMAKE_C_COMPILER="C:/Program Files/mingw-w64/mingw64/bin/gcc.exe" -DCMAKE_CXX_COMPILER="C:/Program Files/mingw-w64/mingw64/bin/g++.exe" -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=RELEASE ..')
   Then the make line underneath that should be changed to:
   os.system('mingw32-make VERBOSE=1')
   Everything around it will probabily need to be modified a bit for windows to run it without error; I just split it into separate lines and os.chdir'd into the multicore_tsne/release/ directory.

3. Add these flags to CMAKE_CXX_FLAGS in the multicore_tsne/CMakeLists.txt file, after the -funroll-loops: -static -static-libgcc -static-libstdc++. This ensures the dll can still use all the dynamically linked windows APIs.

4. After that, you can run python setup.py install. Hopefully that runs and creates the file, but if not you'll have to debug a bit. The major issue I ran into in this part was unlinking the VS10 cl.exe from cmake - setting the generator (-G "MinGW ...") above should take care of issues with that, but you may have to play around with the generator.

5. With the file in /release created, you'll notice it's a .dll rather than the .so which is expected. Copy that file, and go into your python site-packages directory (e.g. C:/Python27/Lib/site-packages). Paste it into the MulticoreTSNE directory, then open up init.py and change the library opener to
   self.ffi.dlopen(path + "/libtsne_multicore.dll")

6. Now, hopefully at this point you're done. For me however it turned out g++ was finding other 32bit libraries to link to (e.g. one it linked to was a file in my Julia installation, of all things). To debug this, download DependencyWalker and open up the DLL. From there, it will show you what dependencies the DLL has, and whether they're 32bit / 64bit. Right click, show full path, and start removing those files from your path (or just temporarily rename their parent folder).

7. Once all dependency issues are resolved (which may take a few times of recompiling and reopening in DependencyWalker) it should finally result in a valid DLL file which is recognized by python.

from multicore-tsne.

Can you try go into multicore_tsne folder and build it how you usually do it on Windows?

from multicore-tsne.

what files do you have in \lib\site-packages\MulticoreTSNE dir?

from multicore-tsne.

It's just __init__.py and __init__.pyc

from multicore-tsne.

If it is any help - here's the dump of the installation process - now I see that there have been two warnings during install

pip install -r requirements.txt
Requirement already satisfied (use --upgrade to upgrade): numpy in c:\users\mari
o\anaconda2\lib\site-packages (from -r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): psutil in c:\users\mar
io\anaconda2\lib\site-packages (from -r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): cffi in c:\users\mario
\anaconda2\lib\site-packages (from -r requirements.txt (line 3))
Requirement already satisfied (use --upgrade to upgrade): pycparser in c:\users
mario\anaconda2\lib\site-packages (from cffi->-r requirements.txt (line 3))

and

python setup.py install
running install
The syntax of the command is incorrect.
'cp' is not recognized as an internal or external command,
operable program or batch file.
running build
running build_py
creating build
creating build\lib
creating build\lib\MulticoreTSNE
copying python__init__.py -> build\lib\MulticoreTSNE
running egg_info
creating MulticoreTSNE.egg-info
writing requirements to MulticoreTSNE.egg-info\requires.txt
writing MulticoreTSNE.egg-info\PKG-INFO
writing top-level names to MulticoreTSNE.egg-info\top_level.txt
writing dependency_links to MulticoreTSNE.egg-info\dependency_links.txt
writing manifest file 'MulticoreTSNE.egg-info\SOURCES.txt'
reading manifest file 'MulticoreTSNE.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'python\libtsne_multicore.so'
writing manifest file 'MulticoreTSNE.egg-info\SOURCES.txt'
running install_lib
creating C:\Users\Mario\Anaconda2\Lib\site-packages\MulticoreTSNE
copying build\lib\MulticoreTSNE__init__.py -> C:\Users\Mario\Anaconda2\Lib\site
-packages\MulticoreTSNE
byte-compiling C:\Users\Mario\Anaconda2\Lib\site-packages\MulticoreTSNE__init__
.py to init.pyc
running install_egg_info
Copying MulticoreTSNE.egg-info to C:\Users\Mario\Anaconda2\Lib\site-packages\Mul
ticoreTSNE-0.1-py2.7.egg-info

from multicore-tsne.

Well, there is no cp command for windows, can you manually copy .so file from multicore_tsne/build to \lib\site-packages\MulticoreTSNE folder to see if it works ?

from multicore-tsne.

Yeah - that came to my mind, too. Unfortunately there is no .so file in my build folder, so the problem seems to happen during the compilation. And I doubt that .so's work inside of windows.

from multicore-tsne.

That's the problem - I am not experienced in building things on windows that do not come with all the bells and whistles attached so all I have to do is to run one or two command lines. Since there is neither a .sln file or some other mechanism that would tell Visual Studio or CMAKE what to actually compile and how to do it, I am lost. Which is why I am hoping that someone with a bit more knowledge can jump in.

from multicore-tsne.

@BrandonSmithJ Thanks! (closed by accident)

from multicore-tsne.

It should be easy now to install on Windows because of #30.

from multicore-tsne.