python-jaxlib installation error about arch4edu HOT 12 CLOSED

I installed your prebuilt binary package but still get the same error.

Does the same error mean RuntimeError: This version of jaxlib was built using AVX instructions? If so, then maybe there is another jax installation in your machine. It looks like you are using venv/conda so it's very likely that there is another jaxlib installed in the virtual environment. You can check it with pip list and pip show [package name].

If you are using conda, then you have to compile the package yourself with the toolchain in conda.

If you are using venv, then you can add the --system-site-packages option when creating the environment to also use the packages installed by pacman like:

$ virtualenv --system-site-packages .env

I assume that you will then install the dependencies via requirements.txt. They always fixed the versions in requirements.txt so it's very likely that there are some conflicts. You have to resolve the conflicts yourself then. If lucky, you can just remove the versions in requirements.txt to use the latest ones. You may have to update the code to meet the api in the latest versions.
You can also try ignoring requirements.txt and manually install the dependencies when the code says it's needed.

from arch4edu.

Hi, the problem is in the PKGBUILD --target_cpu_features is set to release. And now AVX is a target architecture by default in release configuration. If you are compiling the package yourself, then try setting --target_cpu_features to native. It will detect and fit your CPU instruction set locally.

The patch you mentioned in our cactus.yaml is used to change --target_cpu_features native to --target_cpu_features=release. So it is useless now since they already set --target_cpu_features=release in the PKGBUILD.

from arch4edu.

Does the same error mean RuntimeError: This version of jaxlib was built using AVX instructions?

Yes. And I was using venv and had another jaxlib installed via pip.

I did:

python -m venv --system-site-packages .env
source .env/bin/activate --system-site-packages

and success.

Thank you very much for your support and your patience.
It allows me to use speecht5_tts now and play with other super interesting projects on Hugging Face.

from arch4edu.

Thank you for this tip. It compiles but at 4767 out of 5900... my system hanged. Do you know it if it possible to compile it with less features, so that it may work only for TTS or resume compiling after system restart?

from arch4edu.

It compiles but at 4767 out of 5900.

Could you share your build log so we can see the actual error?

Do you know it if it possible to compile it with less features

I haven't tested it but I think this may work:

1. Set --target_cpu_features to release.
2. In build/build.py, change avx_posix to posix. You can do it by adding a line like sed 's/avx_posix/posix/' -i build/build.py before the build command in PKGBUILD.
3. Build the package.

from arch4edu.

And you can also try just setting --target_cpu_features to default according to the help messages in build/build.py.

from arch4edu.

And you can also try just setting --target_cpu_features to default according to the help messages in build/build.py.

@carlosal1015 Should we also apply this change in our repository to disable the AVX support?

from arch4edu.

tip

Could you share your build log so we can see the actual error?

I don't know where they are. I looked at /var/log/pacman.log but there is nothing about it there. System hanged and I restarted so maybe nothing was written.

I will follow instructions and try again later.

Edition:
Unfortunately it hanged again, even sooner. So, I think my pc is too weak for compiling this package and I give up.

from arch4edu.

Yes, if we disable AVX support, then more users will have jax working.

I am using python-jaxlib-cuda from arch4edu and its working fine.

from arch4edu.

Hi @omega3, could you list of packages that you need to have installed from AUR? Did you enable the arch4edu repository?

I see, Manjaro does not have full support, maybe can try run under docker.

from arch4edu.

Unfortunately it hanged again, even sooner. So, I think my pc is too weak for compiling this package and I give up.

You can try the prebuilt binary from arch4edu. You can just download it from https://github.com/arch4edu/cactus/actions/runs/7288311166.

It's built on Arch Linux so it's possible that it won't work on Manjaro. But it still worths a try.

from arch4edu.

I installed your prebuilt binary package but still get the same error. I guess it is because I installed system wide, while this TTS module I installed via pip along with torch and transformers and I run it as they advise in virtual environment, like this:
source .env/bin/activate

from arch4edu.