conda-forge / zeromq-feedstock Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 21.0 181 KB

A conda-smithy repository for zeromq.

License: BSD 3-Clause "New" or "Revised" License

Shell 42.95% Batchfile 57.05%

zeromq-feedstock's People

Contributors

Watchers

Forkers

jakirkham-feedstocks conda-forge-admin bollwyvl sylvaincorlay chapmanb minrk lexual johanmabille anacondarecipes regro-cf-autotick-bot isuruf ryanvolz egpbos eyeware xhochy davidbrochart h-vetinari anaconda-community

zeromq-feedstock's Issues

Build does not contain `zmq.hpp`

I'm trying to build a package that uses libzmq, and expects a zmq.hpp in the include-folder.

Ubuntu does provide such a zmq.hpp as well as a zmq.h, but the build here only provides the latter.

The static build of zeromq links dynamically with libsodium

Should it be linked statically instead?

Another thing we may want to do is add libsodium to the cmake exports in the generated config files of the static build, if we keep the dynamically linked sodium.

ClobberError with defaults::pyzmq

If you have both conda-forge and defaults in your channel list and have channel_priority: false conda might choose to mix-and-match channels between zeromq and pyzmq.

Unfortunately there's a difference between the conda-forge and defaults packaging which means doing so will result in a ClobberError:

ClobberError: This transaction has incompatible packages due to a shared path.
  packages: conda-forge::zeromq-4.2.5-vc14_1, defaults::pyzmq-17.0.0-py36hfa6e2cd_1
  path: 'library/lib/libzmq.lib'

i.e. on conda-forge zeromq provides the libzmq.lib file whereas on defaults that file is provided by pyzmq resulting in the ClobberError

It would be good if the packaging of the libraries were aligned such that you could reasonably mix and match between channels.

Edit: It seems to me that defaults is probably doing the wrong thing here so I'll also open an issue against AnacondaRecipes

`zeromq`
conda-forge	defaults
zeromq-4.2.5-vc14_1 │ ├───info │ └───recipe └───Library ├───bin │ libzmq-mt-4_2_5.dll │ libzmq.dll │ ├───include ├───lib │ │ libzmq-mt-4_2_5.lib │ │ libzmq-mt-s-4_2_5.lib + │ │ libzmq.lib │ │ │ └───pkgconfig └───share ├───cmake │ └───ZeroMQ └───zmq	zeromq-4.2.5-hc6251cf_0 │ ├───info │ ├───recipe │ └───test └───Library ├───bin │ libzmq-mt-4_2_5.dll │ libzmq.dll + │ libzmq.lib │ ├───include ├───lib │ │ libzmq-mt-4_2_5.lib │ │ libzmq-mt-s-4_2_5.lib │ │ │ └───pkgconfig └───share ├───cmake │ └───ZeroMQ └───zmq

`zeromq`

conda-forge defaults

  zeromq-4.2.5-vc14_1
  │
  ├───info
  │   └───recipe
  └───Library
      ├───bin
      │       libzmq-mt-4_2_5.dll
      │       libzmq.dll
      │
      ├───include
      ├───lib
      │   │   libzmq-mt-4_2_5.lib
      │   │   libzmq-mt-s-4_2_5.lib
+     │   │   libzmq.lib
      │   │
      │   └───pkgconfig
      └───share
          ├───cmake
          │   └───ZeroMQ
          └───zmq

  zeromq-4.2.5-hc6251cf_0
  │
  ├───info
  │   ├───recipe
  │   └───test
  └───Library
      ├───bin
      │       libzmq-mt-4_2_5.dll
      │       libzmq.dll
+     │       libzmq.lib
      │
      ├───include
      ├───lib
      │   │   libzmq-mt-4_2_5.lib
      │   │   libzmq-mt-s-4_2_5.lib
      │   │
      │   └───pkgconfig
      └───share
          ├───cmake
          │   └───ZeroMQ
          └───zmq

`pyzmq`
conda-forge	defaults
pyzmq-17.0.0-py36_4 │ ├───info │ └───recipe └───Lib └───site-packages ├───pyzmq-17.0.0-py3.6.egg-info └───zmq ├───asyncio ├───auth │ └───asyncio ├───backend │ ├───cffi │ └───cython ├───devices ├───eventloop │ └───minitornado │ └───platform ├───green │ └───eventloop ├───log ├───ssh ├───sugar ├───tests │ └───asyncio └───utils	pyzmq-17.0.0-py36hfa6e2cd_1 │ ├───info │ ├───recipe │ └───test ├───Lib │ └───site-packages │ ├───pyzmq-17.0.0-py3.6.egg-info │ └───zmq │ ├───asyncio │ ├───auth │ │ └───asyncio │ ├───backend │ │ ├───cffi │ │ └───cython │ ├───devices │ ├───eventloop │ │ └───minitornado │ │ └───platform │ ├───green │ │ └───eventloop │ ├───log │ ├───ssh │ ├───sugar │ ├───tests │ │ └───asyncio │ └───utils + └───Library + └───lib + libzmq.lib

`pyzmq`

conda-forge defaults

  pyzmq-17.0.0-py36_4
  │ 
  ├───info
  │   └───recipe
  └───Lib
      └───site-packages
          ├───pyzmq-17.0.0-py3.6.egg-info
          └───zmq
              ├───asyncio
              ├───auth
              │   └───asyncio
              ├───backend
              │   ├───cffi
              │   └───cython
              ├───devices
              ├───eventloop
              │   └───minitornado
              │       └───platform
              ├───green
              │   └───eventloop
              ├───log
              ├───ssh
              ├───sugar
              ├───tests
              │   └───asyncio
              └───utils

  pyzmq-17.0.0-py36hfa6e2cd_1
  │ 
  ├───info
  │   ├───recipe
  │   └───test
  ├───Lib
  │   └───site-packages
  │       ├───pyzmq-17.0.0-py3.6.egg-info
  │       └───zmq
  │           ├───asyncio
  │           ├───auth
  │           │   └───asyncio
  │           ├───backend
  │           │   ├───cffi
  │           │   └───cython
  │           ├───devices
  │           ├───eventloop
  │           │   └───minitornado
  │           │       └───platform
  │           ├───green
  │           │   └───eventloop
  │           ├───log
  │           ├───ssh
  │           ├───sugar
  │           ├───tests
  │           │   └───asyncio
  │           └───utils
+ └───Library
+     └───lib
+             libzmq.lib

Need to constrain libsodium version

The current version of this recipe does not specify a version of libsodium. On my machine (Mac), I encountered this error at runtime:

ImportError: dlopen(/miniforge/envs/flyem-forge/lib/python3.6/site-packages/libdvid/_dvid_python.so, 2): Library not loaded: @rpath/libsodium.23.dylib
  Referenced from: /miniforge/envs/flyem-forge/lib/libzmq.5.dylib
  Reason: image not found

Manually upgrading libsodium solved the issue:

conda install -c conda-forge libsodium
...

$ conda list libsodium
# packages in environment at /miniforge/envs/flyem-forge:
#
# Name                    Version                   Build  Channel
libsodium                 1.0.16                        0    conda-forge

zeromq 4.2.3 issue (cannot import libzmq.so.5.1.1)

The latest version of ZeroMQ pushed today (4.2.3) is causing trouble with a missing libzmq.so.5.1.1 shared library symlink.

The build was passing fine yesterday and today we have this error:

./../../miniconda3/lib/python3.5/site-packages/zmq/backend/select.py:26: in select_backend
    mod = __import__(name, fromlist=public_api)
../../../miniconda3/lib/python3.5/site-packages/zmq/backend/cython/__init__.py:6: in <module>
    from . import (constants, error, message, context,
E   ImportError: libzmq.so.5.1.1: cannot open shared object file: No such file or directory

See line 1520 of this log https://travis-ci.org/sequana/sequana

Currently skipping on windows

We are currently skipping the zeromq build on Windows.

zeromq 4.2.1 is missing libzmq.so.5 shared library symlink

The latest version of ZeroMQ pushed today (4.2.1) is missing the libzmq.so.5 shared library symlink present in the previous version:

$ ls -lh ~/install/bcbio-vm/anaconda/pkgs/zeromq-4.1.5-0/lib/
total 21M
-rw-r--r-- 2 chapmanb chapmanb  15M Jun 25  2016 libzmq.a
-rwxr-xr-x 1 chapmanb chapmanb  946 Jun 25  2016 libzmq.la
lrwxrwxrwx 1 chapmanb chapmanb   15 Jan 23 11:49 libzmq.so -> libzmq.so.5.0.1
lrwxrwxrwx 1 chapmanb chapmanb   15 Jan 23 11:49 libzmq.so.5 -> libzmq.so.5.0.1
-rwxr-xr-x 2 chapmanb chapmanb 5.4M Jun 25  2016 libzmq.so.5.0.1
drwxrwxr-x 2 chapmanb chapmanb 4.0K Jan 23 11:49 pkgconfig
$ ls -lh anaconda/pkgs/zeromq-4.2.1-0/lib/
total 2.7M
-rw-rw-r-- 1 chapmanb chapmanb 1.8M Jan 23 07:37 libzmq.a
lrwxrwxrwx 1 chapmanb chapmanb   15 Jan 23 11:42 libzmq.so -> libzmq.so.4.2.1
-rwxrwxr-x 1 chapmanb chapmanb 878K Jan 23 07:39 libzmq.so.4.2.1
lrwxrwxrwx 1 chapmanb chapmanb   15 Jan 23 11:42 libzmq.so.5.1.1 -> libzmq.so.4.2.1
drwxrwxr-x 2 chapmanb chapmanb 4.0K Jan 23 11:42 pkgconfig

This causes pyzmq compiled against the previous version to fail with

ImportError: libzmq.so.5: cannot open shared object file: No such file or directory

Thanks so much for looking into this.

rebuild 4.3.2 for aarch, ppc

#46 was merged while aarch64 and ppc builds are not working due to conda-build problems. We'll need to rebuild 4.3.2 on those arches once conda-build is working again.

CVE-2019-13132: denial of service via stack overflow

A remote, unauthenticated client connecting to a
libzmq application, running with a socket listening with CURVE
encryption/authentication enabled, may cause a stack overflow and
overwrite the stack with arbitrary data, due to a buffer overflow in
the library. Users running public servers with the above configuration
are highly encouraged to upgrade as soon as possible, as there are no
known mitigations. All versions from 4.0.0 and upwards are affected.

See: zeromq/libzmq#3558

New release (4.3.2 ) appears to break pyzmq on windows/python2

An example of failure on Windows + Python 2
https://ci.appveyor.com/project/jupyter/jupyter-server/builds/25869866/job/3md0gatibrmxie7s

This appears to work as expected on other platform and with Python 3.

For reference, here is the error message on appveyor:

======================================================================
ERROR: Failure: ImportError (DLL load failed: The specified module could not be found.)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Miniconda-x64\lib\site-packages\nose\loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "C:\Miniconda-x64\lib\site-packages\nose\importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "C:\Miniconda-x64\lib\site-packages\nose\importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "C:\projects\jupyter-server\jupyter_server\extension\tests\test_extension.py", line 5, in <module>
    from ...extension.application import ExtensionApp
  File "C:\projects\jupyter-server\jupyter_server\extension\application.py", line 14, in <module>
    from jupyter_server.serverapp import ServerApp, aliases, flags
  File "C:\projects\jupyter-server\jupyter_server\serverapp.py", line 46, in <module>
    from zmq.eventloop import ioloop
  File "C:\Miniconda-x64\lib\site-packages\zmq\__init__.py", line 47, in <module>
    from zmq import backend
  File "C:\Miniconda-x64\lib\site-packages\zmq\backend\__init__.py", line 40, in <module>
    reraise(*exc_info)
  File "C:\Miniconda-x64\lib\site-packages\zmq\backend\__init__.py", line 27, in <module>
    _ns = select_backend(first)
  File "C:\Miniconda-x64\lib\site-packages\zmq\backend\select.py", line 28, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "C:\Miniconda-x64\lib\site-packages\zmq\backend\cython\__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: DLL load failed: The specified module could not be found.

Cmake config files are missing for the latest packages

ZeroMQConfig.cmake and ZeroMQConfigVersion.cmake are missing for the latest version for linux on conda-forge.
(https://anaconda.org/conda-forge/zeromq/4.3.3/download/linux-64/zeromq-4.3.3-he1b5a44_1.tar.bz2)
These files were included in previous versions and now cause CMake to fail and not find zeromq

cmake vs autotools

About the pyzmq breakage

So when we released zeromq 4.2.3 , it broke the pyzmq conda package (on default and conda-forge), which depended on zeromq 4.2*. The reason is that with the way pyzmq is built, it tries to explicitly load libzmq.so.4.2.1, 4.2.1 being the version it was built against.

Looking at the build results for zeromq with and without cmake in 4.2.1 and 4.2.3, our understanding is that the same breakage for pyzmq would have occurred with pkg-config as with cmake.

With cmake

zeromq 4.2.1

zeromq 4.2.3

Without cmake

zeromq 4.2.1

zeromq 4.2.3

So, with the way pyzmq is currently built, my understanding is that the runtime version should be pinned to the version it was linked with at build time. Since conda has no means of doing so, we simply pinned it at both runtime and build time.

`cmake` versus `configure+ make`

We also often get frustrated with cmake, but for the C++ community, CMake has clearly won and everyone is moving to it now. It is not perfect, documentation is not always great, but we don't think new projects should be using anything else if they have dependencies or intend to be used by other projects... Even boost libraries, which had their own build system for years announced that they were moving to CMake last summer http://boost.2283326.n4.nabble.com/CMake-Announcement-from-Boost-Steering-Committee-tt4696934.html
Even within the zeromq GitHub organization, other projects are relying on zeromq being presumably built with cmake, such as cppzmq, azmq, the lua bindings and others.
The entire xeus stack now relies on the same.
Generating the cmake build artefacts (projectConfig.cmake, projectConfigVersion.cmake and projectTargets.cmake) from a autotools build would be very cumbersome, especially for the Targets file which is platform-specific and lengthy.

For these reasons, we are -1 on the proposed use of autotools+make for Unix platforms by @minrk #20. If we find issues with the cmake build, we think that it is preferable to fix them than to roll back to the previous build method.

cc @isuruf @ocefpaf

-- Sylvain & Johan

Building with libsodium on windows

I realize we have never enabled the --with-libsodium option on windows.

v4.3.2 win64 is broken

Issue:

When I install zeromq 4.3.2 from conda-forge, I can't start Jupyter because it says that the DLL is missing.

When I install zeromq 4.3.1 from conda-forge, it works fine.

Error message:

(pydataldn2019-notes) C:\Data\OneDrive\My Documents\Training\pydataLDN_2019-maintainable-code-for-data-science>jupyter notebook
Traceback (most recent call last):
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\Scripts\jupyter-notebook-script.py", line 6, in <module>
    from notebook.notebookapp import main
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\lib\site-packages\notebook\notebookapp.py", line 47, in <module>
    from zmq.eventloop import ioloop
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\lib\site-packages\zmq\__init__.py", line 47, in <module>
    from zmq import backend
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\lib\site-packages\zmq\backend\__init__.py", line 40, in <module>
    reraise(*exc_info)
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\lib\site-packages\zmq\utils\sixcerpt.py", line 34, in reraise
    raise value
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\lib\site-packages\zmq\backend\__init__.py", line 27, in <module>
    _ns = select_backend(first)
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\lib\site-packages\zmq\backend\select.py", line 28, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "C:\Tools\miniconda3\envs\pydataldn2019-notes\lib\site-packages\zmq\backend\cython\__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: DLL load failed: The specified module could not be found.

Environment (conda list):

$ conda list

# packages in environment at C:\Tools\miniconda3\envs\pydataldn2019-notes:
#
# Name                    Version                   Build  Channel
attrs                     19.1.0                     py_0    conda-forge
backcall                  0.1.0                      py_0    conda-forge
bleach                    3.1.0                      py_0    conda-forge
ca-certificates           2019.6.16            hecc5488_0    conda-forge
certifi                   2019.6.16                py37_0    conda-forge
colorama                  0.4.1                      py_0    conda-forge
decorator                 4.4.0                      py_0    conda-forge
defusedxml                0.5.0                      py_1    conda-forge
entrypoints               0.3                   py37_1000    conda-forge
icu                       58.2                 ha66f8fd_1
ipykernel                 5.1.1            py37h39e3cac_0    conda-forge
ipython                   7.6.1            py37h5ca1d4c_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.5.0                      py_0    conda-forge
jedi                      0.14.0                   py37_0    conda-forge
jinja2                    2.10.1                     py_0    conda-forge
jpeg                      9c                hfa6e2cd_1001    conda-forge
json5                     0.8.5                      py_0    conda-forge
jsonschema                3.0.1                    py37_0    conda-forge
jupyter                   1.0.0                      py_2    conda-forge
jupyter_client            5.3.1                      py_0    conda-forge
jupyter_console           6.0.0                      py_0    conda-forge
jupyter_core              4.4.0                      py_0    conda-forge
jupyterlab                1.0.2                    py37_0    conda-forge
jupyterlab_server         1.0.0                      py_1    conda-forge
libpng                    1.6.37               h7602738_0    conda-forge
libsodium                 1.0.17               h2fa13f4_0    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6
m2w64-gcc-libs            5.3.0                         7
m2w64-gcc-libs-core       5.3.0                         7
m2w64-gmp                 6.1.0                         2
m2w64-libwinpthread-git   5.0.0.4634.697f757               2
markupsafe                1.1.1            py37hfa6e2cd_0    conda-forge
mistune                   0.8.4           py37hfa6e2cd_1000    conda-forge
msys2-conda-epoch         20160418                      1
nbconvert                 5.5.0                      py_0    conda-forge
nbformat                  4.4.0                      py_1    conda-forge
notebook                  5.7.8                    py37_1    conda-forge
openssl                   1.1.1c               hfa6e2cd_0    conda-forge
pandoc                    2.7.3                         0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
parso                     0.5.0                      py_0    conda-forge
pickleshare               0.7.5                 py37_1000    conda-forge
pip                       19.1.1                   py37_0    conda-forge
prometheus_client         0.7.1                      py_0    conda-forge
prompt_toolkit            2.0.9                      py_0    conda-forge
pygments                  2.4.2                      py_0    conda-forge
pyqt                      5.9.2            py37h6538335_0    conda-forge
pyrsistent                0.15.3           py37hfa6e2cd_0    conda-forge
python                    3.7.3                h510b542_1    conda-forge
python-dateutil           2.8.0                      py_0    conda-forge
pywinpty                  0.5.5                 py37_1000    conda-forge
pyzmq                     18.0.0           py37ha925a31_0
qt                        5.9.7                hc6833c9_1    conda-forge
qtconsole                 4.5.1                      py_0    conda-forge
send2trash                1.5.0                      py_0    conda-forge
setuptools                41.0.1                   py37_0    conda-forge
sip                       4.19.8          py37h6538335_1000    conda-forge
six                       1.12.0                py37_1000    conda-forge
sqlite                    3.29.0               hfa6e2cd_0    conda-forge
terminado                 0.8.2                    py37_0    conda-forge
testpath                  0.4.2                   py_1001    conda-forge
tornado                   6.0.3            py37hfa6e2cd_0    conda-forge
traitlets                 4.3.2                 py37_1000    conda-forge
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.15.26706          h3a45250_4
wcwidth                   0.1.7                      py_1    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.33.4                   py37_0    conda-forge
widgetsnbextension        3.5.0                    py37_0    conda-forge
wincertstore              0.2                   py37_1002    conda-forge
winpty                    0.4.3                         4    conda-forge
zeromq                    4.3.2                h6538335_2    conda-forge
zlib                      1.2.11            h2fa13f4_1004    conda-forge

Working environment has this instead:

zeromq                    4.3.1             he025d50_1000    conda-forge

Details about conda and system ( conda info ):

$ conda info

     active environment : pydataldn2019-notes
    active env location : C:\Tools\miniconda3\envs\pydataldn2019-notes
            shell level : 1
       user config file : C:\Users\james\.condarc
 populated config files :
          conda version : 4.7.5
    conda-build version : not installed
         python version : 3.7.3.final.0
       virtual packages :
       base environment : C:\Tools\miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Tools\miniconda3\pkgs
                          C:\Users\james\.conda\pkgs
                          C:\Users\james\AppData\Local\conda\conda\pkgs
       envs directories : C:\Tools\miniconda3\envs
                          C:\Users\james\.conda\envs
                          C:\Users\james\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.7.5 requests/2.21.0 CPython/3.7.3 Windows/10 Windows/10.0.18362
          administrator : False
             netrc file : None
           offline mode : False

Building with GSSAPI / krb5

Would it be possible to ship libzmq linked to GSSAPI/krb5 ?

Lib sodium pinning

Just help someone upgrading notebook, which pull pyzmq, but apparently did not upgrade (install ?) libsodium.

I don't have the whole session just the traceback when notebook was not starting:

Traceback (most recent call last):
  File "/home/mstobb/anaconda3/bin/jupyter-notebook", line 7, in <module>
    from notebook.notebookapp import main
  File "/home/mstobb/anaconda3/lib/python3.6/site-packages/notebook/notebookapp.py", line 46, in <module>
    from zmq.eventloop import ioloop
  File "/home/mstobb/anaconda3/lib/python3.6/site-packages/zmq/__init__.py", line 47, in <module>
    from zmq import backend
  File "/home/mstobb/anaconda3/lib/python3.6/site-packages/zmq/backend/__init__.py", line 40, in <module>
    reraise(*exc_info)
  File "/home/mstobb/anaconda3/lib/python3.6/site-packages/zmq/utils/sixcerpt.py", line 34, in reraise
    raise value
  File "/home/mstobb/anaconda3/lib/python3.6/site-packages/zmq/backend/__init__.py", line 27, in <module>
    _ns = select_backend(first)
  File "/home/mstobb/anaconda3/lib/python3.6/site-packages/zmq/backend/select.py", line 27, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "/home/mstobb/anaconda3/lib/python3.6/site-packages/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: libsodium.so.23: cannot open shared object file: No such file or directory

From the stacktrace I can see now that PYZMQ_BACKEND seem to be in environ but I'm not quite sure this was set by the user. SHould we depends on libsodium in conda ? Or have a better error message in PyZMQ that point to PYZMQ_BACKEND ?

conda-forge / zeromq-feedstock Goto Github PK

zeromq-feedstock's People

Contributors

Watchers

Forkers

zeromq-feedstock's Issues

zeromq

pyzmq

About the pyzmq breakage

With cmake

Without cmake

cmake versus configure+ make

Recommend Projects

Recommend Topics

Recommend Org

`zeromq`

`pyzmq`

`cmake` versus `configure+ make`