afsctool breaks binaries, sometimes (HFS+) about afsctool HOT 9 OPEN

from afsctool.

from afsctool.

If you get a reproducible example, a copy of the binary would be great.

from afsctool.

Okay, I'm fairly confused right now.

I've been unable to reproduce this issue. I started with just copying a couple of binaries outside of /opt/local/bin and compressing them. I remembered having seen errors about zip, so I tried compressing that. Worked fine. zsh for sure didn't work — I had to use Terminal.app because iTerm was configured to use zsh from MacPorts, and it'd crash on start. That also worked after compression.

Then I made a backup of /opt/local and started compressing folders inside, one by one, and try to launch zsh afterwards. Kept working. Did the whole folder again, in case I had missed something. Still worked. Dumbfounded.

I then started thinking about the messages about decmpfs xattr in the report afsctool generates at the end. And then I remembered having installed the GNU version of xattr with MacPorts when I was writing a script to store file checksums in extended attributes. I wondered if somehow afsctool is simply calling the xattr binary and if perhaps there was some sort of difference between the GNU version and the macOS version that could've caused this.

I checked which xattr I was using, and it was the macOS one. Tried installing the MacPorts one to test, but it failed building. I remembered having seen this error before, when I was reinstalling MacPorts and the packages I had after I broke it a week or two ago.

Thing is, this Mac was running Yosemite - where it already had MacPorts xattr installed, and I upgraded to High Sierra, and then upgraded MacPorts and a bunch of other software where applicable. So when my I afsctool'ed my /opt/local folder and it broke everything, I definitely had a GNU xattr on my system, and it was favored in my $PATH. But now I can't build one to test.

Could this be it? Does afsctool use the xattr binary, and does it find that by relying on PATH to resolve to it rather than using /usr/bin/xattr? If so, and if there is a difference between the two (like how GNU mv doesn't keep xattrs, but macOS's mv does), that might explain it.

Other than that, maybe it's just a super weird coincidence that this would happen twice (Homebrew earlier, MacPorts now) for some other reason, while leading me to believe it's related to afsctool :) Stranger things have happened.

from afsctool.

from afsctool.

I've had an app/binary break recently, repeatedly.

Previously, I've recreated the app but in future I'll be sure to investigate more fully.

I automatically run afsctool on new additions to my /Applications folder, so I imagine it will happen again soon.

from afsctool.

Oh my, so many replies :) Thank you!

First, sorry for your mishap.

Thank you so much! And thanks for responding, I hadn't actually expected that :)

I don't see why "any" port install command would be doing a git cherry-pick; which ports are concerned by that? This does make me wonder if you shouldn't be verifying your drive. Try smartctl -a /dev/disk0 (or whatever the device, from port:smartmontools) first, then do at least a check with Disk Utility (for repairing your boot drive you will have to boot off a different drive).

That's a good question. I just copied that line randomly from something like 30-50 lines of the same error. I think I may have just assumed that MacPorts used git somewhere, so running port would do something-something to check for updates (my brain is damaged from too many years on Homebrew). I'm guessing now it's either some of my shell extensions that attempted to do some git checking, or some checked-out code doing that... But either way, the most interesting part was the Error mapping file into memory message, because that's the only one I'll se when I do this (when I afcstool the /usr/local dir). And when MacPorts does its (sorry, this is from memory) "checking binaries for linking errors something are you healthy hello?" stuff at the end of doing other stuff, this was the error message I'd get from a slew of binaries, including randoms like zip and zstd.

Note that it's not impossible that compression could interfere with signing, notarisation and other of Apple's new-fangled attempts to lock us out of our system. I'm not certain how that could be the case with a simple binary though (i.e. not an app bundle).

Also interesting. I'd like to point out, though, that my first encounter with this (my Homebrew story using bkirch's original afsctool) was with either Mavericks or Yosemite (that story happened on another computer, btw, an Air), and that I'm currently on High Sierra, so this isn't something super new in terms of Apple sucking. I also don't see any warnings, and the behavior when trying to run these binaries is as described, it simply just resembles doing an echo that returns exit code 0.

One good way to make certain afsctool is the culprit in damaging an executable installed through MacPorts would be to extract the port image (the tar file in $prefix/var/macports/software/$name/) with bsdtar --hfsCompression. Actually, systematic HFS compression can probably be activated via the hfscompression setting in $prefix/etc/macports/macports.conf and installing port:libarchive .

libarchive has its own HFS/APFS compression implementation, probably using the standard zip compression level.

All of this is news to me, and frankly sounds pretty awesome. I'm sorry to be so lame, but I'm gonna just say, "I will definitely do this!" I just can't do it right now - Easter. But I will dig in to this, and report back.

Thanks again :) And happy Easter (also you, Ginger Beard Man 🤣 )

from afsctool.

from afsctool.

I just finished reinstalling all my relevant MacPorts, and afterwards I ran afsctool over /opt/local (sudo afsctool -cvkl -9 -m 0 -s 1). Immediately after, I could no longer open an iTerm tab (I use zsh from MacPorts).

Isn't that just you maybe forgetting the -f option to detect hard links? I just had that too (albeit still using the original afsctool from a rusty H…**argh**…brew installation). Everything with hardlinks got mangled to 0 byte files (albeit Finder reports funny filesizes) and exposed "com.apple.decmpfs" extended attributes.

$ mkdir newtest
$ cd newtest/
$ echo "fooooooobaaaaar" >test1.txt
$ ln test1.txt test2hardlink.txt

$ ls -l@
total 16
-rw-r--r--  2 turtlewilly  staff  16 Apr  2 17:51 test1.txt
-rw-r--r--  2 turtlewilly  staff  16 Apr  2 17:51 test2hardlink.txt

$ afsctool -vc .
/Users/turtlewilly/Desktop/newtest/.:
Number of HFS+ compressed files: 1
Total number of files: 2
Total number of folders: 0
Total number of items (number of files + number of folders): 2
Folder size (uncompressed; reported size by Mac OS 10.6+ Finder): 16 bytes / 4 KB (kilobytes) / 4 KiB (kibibytes)
Folder size (compressed - decmpfs xattr; reported size by Mac OS 10.0-10.5 Finder): 0 bytes / 0 KB (kilobytes) / 0 KiB (kibibytes)
Folder size (compressed): 33 bytes / 0 KB (kilobytes) / 0 KiB (kibibytes)
Compression savings: -106.2%
Approximate total folder size (files + file overhead + folder overhead): 1186 bytes / 1 KB (kilobytes) / 1 KiB (kibibytes)

$ ls -l
total 0
-rw-r--r--@ 2 turtlewilly  staff  0 Apr  2 17:51 test1.txt
-rw-r--r--@ 2 turtlewilly  staff  0 Apr  2 17:51 test2hardlink.txt

$ cat test1.txt   # note: no output

$ ls -l@
total 0
-rw-r--r--@ 2 turtlewilly  staff  0 Apr  2 17:51 test1.txt
	com.apple.decmpfs	33 
-rw-r--r--@ 2 turtlewilly  staff  0 Apr  2 17:51 test2hardlink.txt
	com.apple.decmpfs	33

I guess the (compressed) data is still there in those decmpfs attributes, just something with the file flags went wrong. So remember: always use the -f option. Or check with find . -xdev -type f -not -links 1 first to see what's up in the directory regarding hardlinks, else you have to find . -type f -size 0 afterwards and try to clean up the messup from backups.

F.ex. I ended up with 0 byte unzip/ zipinfo executables (zipinfo is a hardlink to unzip). Executing them just did nothing.

from afsctool.