Update documentation for indexed-zstd requirement on Apple Silcone about ratarmount HOT 4 CLOSED

Should be fixed with indexed_zstd 1.6.1

from ratarmount.

Maybe it is too late in the evening already, but I am confused by these things:

• Your solution has the same pipx call as the non-working pipx call above it. Why is it suddenly working? What is the solution you speak of in the "Solution" subheading?
• Normally, you would have to install zstd development files via some method, which is mentioned in the readme. Therefore, I am confused what should be be done regarding the issue title: "update documentation". It is already documented.
• In your alternative you are speaking about existing wheels. What do you mean by that? Wheels for indexed_zstd are definitely not existing (on PyPI), or else you wouldn't get the error with the missing zstd.h.
• Why are you using pipx install ratarmount indexed-zstd instead of pipx install ratarmount?

As far as I know, the current state is:

• The zstd.h missing error does occur when installing on non-x86-64 architectures. That's also partly why I changed macos-latest to macos-12 in the CI for now.
• indexed_zstd only has wheels for Darwin x86-64, not for ARM.

What should be done:

• Now that Github Actions seems to have a new macos-latest target since a few days ago, indexed_zstd should create wheels for ARM on MacOS.
• The indexed-zstd tarball could bundle and statically link zstd. The BSD-3 license is sufficiently permissive enough. This makes it more future-proof and less cumbersome from the user side.

What could be done:

• Alternatively, indexed_zstd as a requirement could be excluded from ratarmount specifically for Darwin and instruction could be added to facilitate manual installation if necessary.
• I guess, the installation instructions could be written down more generically, or simply zstd could be added to the brew install macfuse call mentioned in the ReadMe. Currently, it simply shows the apt install command line because that's the system I use.

from ratarmount.

• Your solution has the same pipx call as the non-working pipx call above it. Why is it suddenly working? What is the solution you speak of in the "Solution" subheading?

Sorry! I meant to copy the one that just was ratarmount. It had indexed_zstd with an underscore in that version, which doesnt exist, so the output should basically be the same as just ratarmount by itself. Either way, I updated it.

Normally, you would have to install zstd development files via some method, which is mentioned in the readme. Therefore, I am confused what should be be done regarding the issue title: "update documentation". It is already documented.

I have brew install zstd, which was already mentioned as not a solution martinellimarco/indexed_zstd#17 (comment). Maybe I misinterpreted that you would be aware that this is not a solution, at least for me, in being able to install ratarmount. I did have it, but still experienced the error until I had pip also install indexed-zstd.

In your alternative you are speaking about existing wheels. What do you mean by that? Wheels for indexed_zstd are definitely not existing (on PyPI), or else you wouldn't get the error with the missing zstd.h.

I can install indexed-zstd on M2.

❯ pip wheel --no-deps indexed-zstd
Collecting indexed-zstd
  Using cached indexed_zstd-1.6.0.tar.gz (66 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: indexed-zstd
  Building wheel for indexed-zstd (pyproject.toml) ... done
  Created wheel for indexed-zstd: filename=indexed_zstd-1.6.0-cp312-cp312-macosx_11_0_arm64.whl size=48534 sha256=92177ea2221af0900e8a06699ecb333b3c661635aaf1bd4048a1d48cdb500061
  Stored in directory: /Users/drewbitt/Library/Caches/pip/wheels/7b/d6/7a/795af962ac143dab3666923527a04f7b0b644efb1ce164e63b
Successfully built indexed-zstd

These will not be installed when I run pipx install ratarmount by itself; see output in first message.

Why are you using pipx install ratarmount indexed-zstd instead of pipx install ratarmount?

Because ratarmount errors with a lack of zstd.h and the one with both doesn't :)

indexed_zstd only has wheels for Darwin x86-64, not for ARM.

I see that now, so that is why I have to build them for Mac M1 and (via pipx indexed-zstd or an alternative), which is what this is all about, I guess - and noting that they are not being built with just an install of ratarmount. I am noting that I cannot install this package on a silicone mac without additional work which is not included in the README, and I advised updating the documentation. There seems to be some pushback here, so no worries, just close this issue out if you are not agreeable and it will at least exist as documentation for others.

from ratarmount.

I have brew install zstd, which was already mentioned as not a solution martinellimarco/indexed_zstd#17 (comment). Maybe I misinterpreted that you would be aware that this is not a solution, at least for me, in being able to install ratarmount. I did have it, but still experienced the error until I had pip also install indexed-zstd.

I'm sorry, I forgot that brew install zstd didn't help as the issue is some months old, and I didn't reread the issue as far as that comment yesterday.

I still find it weird... Why would there be any difference between explicitly listing the indexed-zstd dependency and installing it as a dependency of ratarmount. It just makes no sense to me. Unfortunately, I don't have a Mac and testing via the CI is cumbersome. If you have a fix for this behavior, I'd definitely merge it, but from a user point, this almost looks like a bug in pipx... What I don't seem to see in your pipx output is which version is installed. I'd assume indexed-zstd 1.6.0 in both cases, but maybe not? The ratarmount requirement is:

    indexed_zstd >= 1.3.1, < 2.0; sys_platform=="darwin"
    indexed_zstd >= 1.2.2, < 2.0; platform_system!="Windows"

Similarly, I don't understand why your pip wheel --no-deps indexed-zstd call works but not the pip install call. Why does it find zstd.h in one case but not the other? Is something modifying the path variables?

I see that now, so that is why I have to build them for Mac M1 and (via pipx indexed-zstd or an alternative), which is what this is all about, I guess - and noting that they are not being built with just an install of ratarmount. I am noting that I cannot install this package on a silicone mac without additional work which is not included in the README, and I advised updating the documentation. There seems to be some pushback here, so no worries, just close this issue out if you are not agreeable and it will at least exist as documentation for others.

I definitely agree that users should be able to install without any errors. There is no pushback against that. I was just confused about the problem itself because I have no Mac. Simply having a discoverable issue is nice to have. In the meantime, I'll ask upstream about releasing the indexed_zstd-1.6.0-cp312-cp312-macosx_11_0_arm64.whl to PyPI. That should fix this issue, although I am still very much confused about why it installs when explicitly listed vs. simply as a dependency.

from ratarmount.