Comments (17)
informationcake
commented on May 24, 2024
I get the same issue with my macbook pro running El Capitan. With verbose=True it states that only 1 core is being used (top confirms this in terminal) regardless of the n_jobs parameter.
from multicore-tsne.
PaulRivaud
commented on May 24, 2024
Experiencing the same issue. Is the value hard coded in line 604 there:
https://github.com/DmitryUlyanov/Multicore-TSNE/blob/master/multicore_tsne/tsne.cpp ?
from multicore-tsne.
DmitryUlyanov
commented on May 24, 2024
It usually means that OpenMP was not found in the system. Although @kernc did a great job by improving compatibility with MacOS, some problems still emerge.
Take a look at this thread #1 and try the recipes from there. It should help.
from multicore-tsne.
kernc
commented on May 24, 2024
Indeed, if you are using a pre-built wheel installed from PyPI with pip install MulticoreTSNE, it likely was built without OpenMP support.
The workaround on OS X is possibly:
brew install libomp
# or alternatively, install Xcode
# Force building the package from source
pip install -U --no-binary MulticoreTSNE
# or pip install -U git+https://github.com/DmitryUlyanov/Multicore-TSNE.git@DmitryUlyanov Since we are using Homebrew already to install Python in OS X builds, might as well put in there libomp package for releases.
from multicore-tsne.
PaulRivaud
commented on May 24, 2024
Thank you the suggestions.
I uninstalled MulticoreTSNE using pip, installed libomp and tried building the package from source, but that didn't solve the issue.
from multicore-tsne.
kernc
commented on May 24, 2024
When building the source, was this string present?
CMake Warning at CMakeLists.txt:22 (MESSAGE):
Not using OpenMP. Performance will suffer.
from multicore-tsne.
PaulRivaud
commented on May 24, 2024
Here is the output:
paul@dhcp-24-184:~/Documents$pip install -U git+https://github.com/DmitryUlyanov/Multicore-TSNE.git
Collecting git+https://github.com/DmitryUlyanov/Multicore-TSNE.git
Cloning https://github.com/DmitryUlyanov/Multicore-TSNE.git to /private/tmp/pip-fbXO5V-build
Collecting numpy (from MulticoreTSNE==0.1)
Downloading numpy-1.14.1-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (4.7MB)
100% |████████████████████████████████| 4.7MB 256kB/s
Requirement already up-to-date: cffi in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from MulticoreTSNE==0.1)
Requirement already up-to-date: pycparser in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from cffi->MulticoreTSNE==0.1)
Installing collected packages: numpy, MulticoreTSNE
Found existing installation: numpy 1.13.3
Uninstalling numpy-1.13.3:
Successfully uninstalled numpy-1.13.3
Running setup.py install for MulticoreTSNE ... done
Successfully installed MulticoreTSNE-0.1 numpy-1.14.1Package works fine otherwise but it would be helpful for bigger datasets.
from multicore-tsne.
kernc
commented on May 24, 2024
This doesn't print build stage output. Try:
pip install --verbose -U --no-deps git+https://github.com/DmitryUlyanov/Multicore-TSNE.git
from multicore-tsne.
sg-s
commented on May 24, 2024
I've gotten this working on my MacBook (though I'm using an older version of this repo)
see here: https://github.com/sg-s/Multicore-TSNE
from multicore-tsne.
kernc
commented on May 24, 2024
@PaulRivaud Have you managed to build with --verbose flag to determine whether OpenMP is not available and why?
from multicore-tsne.
PaulRivaud
commented on May 24, 2024
@kernc Sorry for the late response. I tried using the command but got the following:
SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:661)
I updated openssl with brew but I still get that error.
from multicore-tsne.
kernc
commented on May 24, 2024
SSLError ...
Unrelated to Multicore-TSNE depending on OpenMP. More like something to do with another dependency re-downloaded from PyPI and this recent policy update: https://status.python.org/incidents/hdx7w97m5hr8. The proposed workaround for OS X is to update your pip (pip install -U pip) and, indeed, try again later.
from multicore-tsne.
PaulRivaud
commented on May 24, 2024
Thank you, I managed to get rid of the error.
Command output with verbose flag:
Collecting https://github.com/DmitryUlyanov/Multicore-TSNE.git
Looking up "https://github.com/DmitryUlyanov/Multicore-TSNE.git" in the cache
No cache entry available
Starting new HTTPS connection (1): github.com
https://github.com:443 "GET /DmitryUlyanov/Multicore-TSNE.git HTTP/1.1" 301 178
Updating cache with response from "https://github.com/DmitryUlyanov/Multicore-TSNE.git"
Caching permanant redirect
Looking up "https://github.com/DmitryUlyanov/Multicore-TSNE" in the cache
No cache entry available
https://github.com:443 "GET /DmitryUlyanov/Multicore-TSNE HTTP/1.1" 200 None
Downloading https://github.com/DmitryUlyanov/Multicore-TSNE.git
Downloading from URL https://github.com/DmitryUlyanov/Multicore-TSNE.git
/ 122kB 3.1MB/s Updating cache with response from "https://github.com/DmitryUlyanov/Multicore-TSNE"
Cannot unpack file /private/var/folders/sr/dxzg_7ms43z84dfqf9rmnksc0000gn/T/pip-pkFrFJ-unpack/Multicore-TSNE.git (downloaded from /private/var/folders/sr/dxzg_7ms43z84dfqf9rmnksc0000gn/T/pip-5ctuLP-build, content-type: text/html; charset=utf-8); cannot detect archive format
Cleaning up...
Cannot determine archive format of /private/var/folders/sr/dxzg_7ms43z84dfqf9rmnksc0000gn/T/pip-5ctuLP-build
Exception information:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/req/req_set.py", line 620, in _prepare_file
session=self.session, hashes=hashes)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/download.py", line 821, in unpack_url
hashes=hashes
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/download.py", line 663, in unpack_http_url
unpack_file(from_path, location, content_type, link)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip/utils/__init__.py", line 620, in unpack_file
'Cannot determine archive format of %s' % location
InstallationError: Cannot determine archive format of /private/var/folders/sr/dxzg_7ms43z84dfqf9rmnksc0000gn/T/pip-5ctuLP-build```
from multicore-tsne.
kernc
commented on May 24, 2024
Gash, sorry, the command is:
pip install --verbose -U --no-deps git+https://github.com/DmitryUlyanov/Multicore-TSNE.git
(git URL needs git+ prefix).
from multicore-tsne.
PaulRivaud
commented on May 24, 2024
Oh okay. So here it is:
CMake suite maintained and supported by Kitware (kitware.com/cmake).
-- The CXX compiler identification is AppleClang 7.0.2.7000181
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) (Required is at least version "2")
-- Could NOT find OpenMP (missing: OpenMP_CXX_FOUND) (Required is at least version "2")
CMake Warning at CMakeLists.txt:22 (MESSAGE):
Not using OpenMP. Performance will suffer.
I will try to look at it.
from multicore-tsne.
kernc
commented on May 24, 2024
AppleClang (7.0.2) doesn't seem to ship with OpenMP configured. Two things you can try:
- Add
-fopenmp=libompto CMakeLists here. - Use another compiler, such as GCC.
from multicore-tsne.
vishalmhjn
commented on May 24, 2024
Facing the same issue on OSx 10.15
from multicore-tsne.
Related Issues (20)
- Memory Allocation Fail (Big Data)
- TSNE results in dense centred ball HOT 2
- Ruby Library
- License Clarification HOT 1
- Failed building wheel for MulticoreTSNE HOT 1
- Non-verbose crash on input containing NaN
- Error in builds using cmake >= 3.20? HOT 8
- Cannot generate Makefile. HOT 1
- Can't install on Apple M1 chip HOT 13
- Enabling verbose causes kernel crash (likely a divided by zero)
- `pip install cmake==3.18.4` worked for me.
- cmake version requirement.txt HOT 1
- signature of sklearn.datasets.make_blobs has changed
- Fix compiling HOT 5
- why can't Multicore-TSNE speed up my experiment with mnist
- linux compile
- Error in generate MING64 Makefile
- Type error with sklearn 1.2.3 HOT 1
- Static library is built but runtime is trying to load dynamic one HOT 1
- Graphics problem with tsne algorithm
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from multicore-tsne.