lgro / pyapsi Goto Github PK

We find the same problem on a special cpu (i.e., Intel Xenon E5-2620 v4), while it works fine on other CPUs (e.g., Intel 8336).

Can you provide more details for this issue (i.e., illegal instruction), for example, the explaination from Intel or gcc or Cmake ...

Thanks!

https://github.com/LGro/PyAPSI/blob/main/docker/base.Dockerfile#L13-L17

On CI/CD the vcpkg built dependencies might need to be generated as *.so files whereas they are generated as *.a files, which seems to work locally but fails in the actions workflow. Unify that to make building locally as well as in CI/CD work.

Hello, how is PyApsi intended to be installed. I would like to implement a project that includes PSI functionality. Installing PyPSI through PIP results in an invalid instruction exception when executed. Installing manually using the instructions in /docker is very bad for integration into complex poetry projects. Is there any way to easily install this, or fix the pip installation?

Goal state
apsi is installed as is via vcpkg and can be used with AVX2 on supported platforms.

Current state
apsi built from source after AVX2 support is patched out, building only with AVX even on AVX2 enabled platforms (like the GitHub Actions runners.

Some of the docstrings, e.g. of the Python Server classes still erroneously contain asynchronous for the A in APSI, while it needs to be asymmetric.

• generating oprf request
• handling oprf request
• building query from oprf response
• handling query
• extracting results from query response

@LGro, great work with this wrapper. It looks well designed and possibly more user friendly than the base APSI library. I had some questions regarding your work

1. Does PyAPSI have any improvements over APSI or is just a Python wrapper?
2. Can you in short detail the process of creating this wrapper? (APSI does not seem to have a user friendly documentation and I believe you have already gone in the weeds to make sense of almost everything about APSI)
3. In the building and testing from source can you list the docker commands that need to be run to build and commands for the container to be tested.

Thank you very much in advance.

While task base still works, running task build PYTHON_VERSION=3.10.4 fails with the following error:

#0 3.894 CMake Error at /tmp/vcpkg/scripts/buildsystems/vcpkg.cmake:852 (_find_package):
#0 3.894 Could not find a configuration file for package "SEAL" that is compatible
#0 3.894 with requested version "4".
#0 3.894
#0 3.894 The following configuration files were considered but not accepted:
#0 3.894
#0 3.894 /tmp/vcpkg/installed/x64-linux/share/seal/SEALConfig.cmake, version: 4.1.1

Add build and release setup for Windows compatible wheels.

Provide (non-)labeled APSI wrappers as individual classes.

Add build and release setup for MacOS compatible wheels (beware: Intel and ARM).

This issue seems to occur when the wheel was built on another system.

Steps to reproduce:

• Arch Linux
• CPU: i7 2600
• Conda env with Python 3.8 or 3.10
• pip install apsi==0.1.2
• run example from README

Also reproducible with the following Dockerfile:

FROM python:3.9-slim-bullseye
RUN pip install apsi
COPY ./example.py /example.py
RUN python example.py

Since this seems to happen in the same Docker image that it was built in, I suspect the available CPU instructions as the origin.

• build manylinux1 compatible wheels https://peps.python.org/pep-0513/#rationale
  • specialized old base images like quay.io/pypa/manylinux_2_24_x86_64 are proposed
  • cibuildwheel is used in pybind11 example
    • wants to run on host or at least with docker; potentially build _pyapsi and then move outside of docker?
    • also uses quay.io manylinux base images; but seems to use manylinux2014 not most recent one manylinux_2_24_x86_64, can be overwritten with env CIBW_MANYLINUX_X86_64_IMAGE=quay.io/pypa/manylinux_2_24_x86_64
    • fails with CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool. bceause vcpkg is located in /tmp/ (note, environment variable used in setup.py for vcpkg root dir doesn't seem to be propagated to cibuildwheel dockers)
    • flatc seems to require GLIBCXX_3.4.26 which is not present in the manylinux docker
    • following the pybind11 example workflow seems to work for build_sdist, but also just seems to package sources (likely missing important bits)
  • setup.py build is deprecated, but python -m build conflicts with current cmake setup
• alternative to pre-built: add apsi as submodule, figure out how to patch AVX during/before build, and then have folks do VCPKG_ROOT_DIR=/path/to/vcpkg pip install git+https://github.com/LGro/PyAPSI.git
• upload to pypi on tag

Hello:

Thank you in advance for any help.

I've been working with PyAPSI for a little while now, and things were working well.

Within the last week, however, the build has broken within the execution of the docker script py.Dockerfile.

At the bottom of this message, I am including a trace of the final bits before the error. This is being executed on a fresh install of Ubuntu 22, using the command specified in the README.md, specifically: task wheel PYTHON_VERSION=3.10.4

Prior to running task, on top of the fresh install, I am executing the following precursor steps, in order to have a working environment for the build:

sudo apt install python3-pip
sudo snap install task --classic
sudo snap install docker
pip install poetry

I am also setting some permissions for docker, as below:

sudo groupadd docker
sudo usermod -aG docker $USER
sudo chown root.docker /var/run/docker.sock

[then I re-login to get docker group permissions].

Prior to last week, this was working. I have tried a few things I've found regarding making sure that the wheel Python package is installed [it is, always has been]. However, it seems likely that some repository's packages have updated, and we end up with the following error trace, per below, during the py.Dockerfile portion of the build:

Step 12/16 : RUN poetry install
---> Running in 2b812b584760
Creating virtualenv pyapsi-s1QCMSlm-py3.10 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 22 installs, 0 updates, 0 removals

• Installing pyparsing (3.0.9)
• Installing tomli (2.0.1)
• Installing attrs (21.4.0)
• Installing click (8.1.3)
• Installing iniconfig (1.1.1)
• Installing mypy-extensions (0.4.3)
• Installing packaging (21.3)
• Installing pathspec (0.9.0)
• Installing pep517 (0.12.0)
• Installing platformdirs (2.5.2)
• Installing pluggy (1.0.0)
• Installing py (1.11.0)
• Installing pyelftools (0.28)
• Installing snowballstemmer (2.2.0)
• Installing auditwheel (5.1.2)
• Installing black (22.3.0)
• Installing isort (5.10.1)
• Installing build (0.8.0)
• Installing pybind11 (2.9.2)
• Installing pydocstyle (6.1.1)
• Installing pytest (7.1.2)
• Installing toml (0.10.2)

Installing the current project: pyapsi (0.1.2)

The current project could not be installed: No file/folder found for package pyapsi
If you do not want to install the current project use --no-root
Removing intermediate container 2b812b584760
---> c5d4e85b1e45
Step 13/16 : RUN poetry run python setup.py bdist_wheel
---> Running in ca24f11fffa5
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: invalid command 'bdist_wheel'
The command '/bin/sh -c poetry run python setup.py bdist_wheel' returned a non-zero code: 1
exit status 1

Since PyAPSI is such a thin wrapper, including the version of the wrapped apsi library in the apsi Python package version seems to make sense to increase transparency to users.