Complains about _clfftBakePlan not found when importing about gpyfft HOT 8 OPEN

geggo commented on May 30, 2024

Complains about _clfftBakePlan not found when importing

from gpyfft.

Comments (8)

geggo commented on May 30, 2024

How did you install the clfft libs?

…

Am 21.04.2021 um 11:26 schrieb yves-surrel ***@***.***>: Trying to install gpyfft on a new Mac Mini M1. Successfully built and compiled sources python setup.py build python setup.py install with no errors. When importing in python, I get: In [1]: import gpyfft --------------------------------------------------------------------------- ImportError Traceback (most recent call last) <ipython-input-1-3687f3cada1c> in <module> ----> 1 import gpyfft ~/Downloads/gpyfft-master/gpyfft/__init__.py in <module> 4 5 from .version import __version__ ----> 6 from .gpyfftlib import GpyFFT, GpyFFT_Error, Plan 7 from .fft import * ImportError: dlopen(/Users/wyselight/Downloads/gpyfft-master/gpyfft/gpyfftlib.cpython-38-darwin.so, 2): Symbol not found: _clfftBakePlan Referenced from: /Users/wyselight/Downloads/gpyfft-master/gpyfft/gpyfftlib.cpython-38-darwin.so Expected in: flat namespace in /Users/wyselight/Downloads/gpyfft-master/gpyfft/gpyfftlib.cpython-38-darwin.so Any idea ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#52>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHMNM77XKPJYAGDJ5R5XP3TJ2K6FANCNFSM43J63AFA>.

from gpyfft.

yves-surrel commented on May 30, 2024

Hi Gregor I followed the method you indicated in a previous e-mail to me: " I typically build and install clFFT by cloning latest version from github, and then cd src cmake -G "Unix Makefiles" make sudo make install " and everything has run fine (in terms of how it compiles)! Yves

…

Le 21 avr. 2021 à 16:34, Gregor Thalhammer ***@***.***> a écrit : How did you install the clfft libs? > Am 21.04.2021 um 11:26 schrieb yves-surrel ***@***.***>: > > > Trying to install gpyfft on a new Mac Mini M1. > > Successfully built and compiled sources > > python setup.py build > python setup.py install > with no errors. > > When importing in python, I get: > > In [1]: import gpyfft > --------------------------------------------------------------------------- > ImportError Traceback (most recent call last) > <ipython-input-1-3687f3cada1c> in <module> > ----> 1 import gpyfft > > ~/Downloads/gpyfft-master/gpyfft/__init__.py in <module> > 4 > 5 from .version import __version__ > ----> 6 from .gpyfftlib import GpyFFT, GpyFFT_Error, Plan > 7 from .fft import * > > ImportError: dlopen(/Users/wyselight/Downloads/gpyfft-master/gpyfft/gpyfftlib.cpython-38-darwin.so, 2): Symbol not found: _clfftBakePlan > Referenced from: /Users/wyselight/Downloads/gpyfft-master/gpyfft/gpyfftlib.cpython-38-darwin.so > Expected in: flat namespace > in /Users/wyselight/Downloads/gpyfft-master/gpyfft/gpyfftlib.cpython-38-darwin.so > Any idea ? > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub <#52>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHMNM77XKPJYAGDJ5R5XP3TJ2K6FANCNFSM43J63AFA>. > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQM3HGEESO6MMFQLPCBWDTTJ3O6ZANCNFSM43J63AFA>.

from gpyfft.

geggo commented on May 30, 2024

The error message "Symbol not found: _clfftBakePlan" indicates that the the clFFT library cannot be linked to when importing the wrapper.

You could try 'otool -L gpyfftlib.cpython-38-darwin.so' to check to which libraries the wrapper is linked, and check if they are properly installed.

I don't know how the switch to a new architecture is handled, try checking, e.g. via 'otool -hv yourlib.so' for which architecture the gpyfft and clFFT libs have been built.

hope that helps
Gregor

from gpyfft.

yves-surrel commented on May 30, 2024

You were right, the clFFT library was not linked.

I retried everything from the very beginning, and after more careful examination, I noticed the following when building gpyfft:

g++ -bundle -undefined dynamic_lookup -L/Users/wyselight/miniconda3/lib -arch x86_64 -L/Users/wyselight/miniconda3/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.8/gpyfft/gpyfftlib.o -L/Users/wyselight/clFFT-master/src/library -lclFFT -o build/lib.macosx-10.9-x86_64-3.8/gpyfft/gpyfftlib.cpython-38-darwin.so -stdlib=libc++
ld: warning: ignoring file /Users/wyselight/clFFT-master/src/library/libclFFT.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

So the problem may come from the new hardware the Mac Mini M1 is running on, and it seems that some architecture flags are not the same for clFFT and gpyFFT. What do you think?

from gpyfft.

geggo commented on May 30, 2024

Indeed, that explains the failure. It seems the python your are using for building the gpyfft wrapper uses the x86_64 architecture emulation. I read that native python (with numpy!) is available (anaconda with conda-forge channel)

In case you succeed I am interested in the performance, could you please post the result of running

python -m gpyfft.benchmark

from gpyfft.

yves-surrel commented on May 30, 2024

Good news ! I succeeded in installing gpyfft, but I went the other way round, i.e. forcing clFFT to be compiled to the x86_64 arch instead of installing a native python (I fear to have other problems with the numerous libraries I am using). So, after some googling around, I did in the clFFT-master directory:

cd src
CMAKE_OSX_ARCHITECTURES=x86_64 cmake -G "Unix Makefiles"
make
sudo make install

Here is the benchmark for the OpenCL device 1 (GPU) (using device 0 raises an INVALID_WORKGROUP_SIZE error, well known for Macs, I think):

python -m gpyfft.benchmark
Choose platform:
[0] <pyopencl.Platform 'Apple' at 0x7fff0000>
Choice [0]:0
Choose device(s):
[0] <pyopencl.Device 'Apple M1' on 'Apple' at 0xffffffff>
[1] <pyopencl.Device 'Apple M1' on 'Apple' at 0x1027f00>
Choice, comma-separated [0]:1
Set the environment variable PYOPENCL_CTX='0:1' to avoid being asked again.
out of place transforms (1024, 1024) complex64
axes         in out
(-2, -1)     C   C  7.5e-04  0.93ms 112.29 Gflops
(-2, -1)     C   F  7.5e-04  0.77ms 135.68 Gflops
(-2, -1)     F   C  7.5e-04  0.63ms 167.29 Gflops
(-2, -1)     F   F  7.5e-04  0.87ms 120.15 Gflops
(-1, -2)     C   C  7.6e-04  0.86ms 121.51 Gflops
(-1, -2)     C   F  7.6e-04  0.63ms 165.40 Gflops
(-1, -2)     F   C  7.6e-04  0.74ms 142.16 Gflops
(-1, -2)     F   F  7.6e-04  0.72ms 144.66 Gflops
None         C   C  7.6e-04  0.87ms 119.87 Gflops
None         C   F  7.6e-04  0.66ms 159.64 Gflops
None         F   C  7.5e-04  0.61ms 170.71 Gflops
None         F   F  7.5e-04  0.87ms 120.64 Gflops
in place transforms (1024, 1024) complex64
(-2, -1)     C  0.73ms 144.39 Gflops
(-2, -1)     F  0.63ms 167.35 Gflops
(-1, -2)     C  0.60ms 175.42 Gflops
(-1, -2)     F  0.73ms 142.97 Gflops
None         C  0.72ms 144.75 Gflops
None         F  0.74ms 141.04 Gflops

For reference, here is the benchmark on my MBP 15" 2017, on the 'AMD Radeon Pro 555 Compute Engine' openCL device

(base) MacBook-Pro-de-Yves:~ yves$ python -m gpyfft.benchmark
Choose platform:
[0] <pyopencl.Platform 'Apple' at 0x7fff0000>
Choice [0]:
Choose device(s):
[0] <pyopencl.Device 'Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz' on 'Apple' at 0xffffffff>
[1] <pyopencl.Device 'Intel(R) HD Graphics 630' on 'Apple' at 0x1024500>
[2] <pyopencl.Device 'AMD Radeon Pro 555 Compute Engine' on 'Apple' at 0x1021c00>
Choice, comma-separated [0]:2
Set the environment variable PYOPENCL_CTX=':2' to avoid being asked again.
out of place transforms (1024, 1024) complex64
axes         in out
(-2, -1)     C   C  7.4e-04 18.87ms   5.56 Gflops
(-2, -1)     C   F  7.4e-04 28.72ms   3.65 Gflops
(-2, -1)     F   C  7.4e-04 12.65ms   8.29 Gflops
(-2, -1)     F   F  7.4e-04  1.53ms  68.68 Gflops
(-1, -2)     C   C  7.8e-04  1.52ms  69.20 Gflops
(-1, -2)     C   F  7.8e-04 12.63ms   8.30 Gflops
(-1, -2)     F   C  7.8e-04 28.76ms   3.65 Gflops
(-1, -2)     F   F  7.8e-04 18.98ms   5.52 Gflops
None         C   C  7.8e-04  1.52ms  69.10 Gflops
None         C   F  7.8e-04 12.56ms   8.35 Gflops
None         F   C  7.4e-04 12.66ms   8.28 Gflops
None         F   F  7.4e-04  1.52ms  68.79 Gflops
in place transforms (1024, 1024) complex64
(-2, -1)     C 18.82ms   5.57 Gflops
(-2, -1)     F  1.52ms  69.15 Gflops
(-1, -2)     C  1.32ms  79.39 Gflops
(-1, -2)     F  8.85ms  11.85 Gflops
None         C  1.40ms  74.77 Gflops
None         F  1.32ms  79.35 Gflops

So it seems not too bad with this new Mac Mini M1 ;-)

from gpyfft.

geggo commented on May 30, 2024

Excellent!
Seems you get a decent performance, and your fix to switch architecture for building clFFT is easy to apply. Thanks!

from gpyfft.

psobolewskiPhD commented on May 30, 2024

FYI: you can get arm64 native clFFT from homebrew.
https://formulae.brew.sh/formula/clfft#default
Many native python3 packages are available via miniforge3 conda env and also pip (including numpy, scipy, pyopencl, etc).

I've been using the python sub variant of CLIJ2-clFFT by @bnorthan
https://github.com/clij/clij2-fft
with success for deconvolution.

from gpyfft.

Complains about _clfftBakePlan not found when importing about gpyfft HOT 8 OPEN

Comments (8)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent