Operation not permitted when mounting as non-root about mergerfs HOT 5 OPEN

46212 02:07:12.744338 mount("/tmp/mergerfs/branch", ".", "fuse", MS_NOSUID|MS_NODEV, "default_permissions,fd=3,rootmode=40000,user_id=1000,group_id=1000") = -1 EPERM (Operation not permitted) <0.000009>

From the strace. The kernel, not mergerfs, is giving you perm denied. It is clearly something about your system. Are you trying to mount this in a container? On a system without fusermount having setuid bits?

from mergerfs.

Without digging in yet...

FUSE filesystems need auxiliary tooling to be mounted as non-root. If you were mixing installed versions then there could be some incompatibility or not setup correctly.

from mergerfs.

It works on my systems fine.

from mergerfs.

I've found the reason (not the cause), but still don't understand it.
Anyway, you are right: it works (after I uninstalled the Distribution's version).

If you are interested in the root cause why it fails on my system, I am happy to dig deeper. Just let me know.

Some more details (in case you are interested):
Although (initially?) the expected, self built mergerfs binary get executed, it somehow "collides" with the version installed by the distribution (Debian testing). $PATH is set correctly afaik. But there is also a mergerfs-fusermount and mount.mergerfs binary installed by the package manager:

doak@flatbox:~$ which mergerfs
/home/doak/daten/software/linux/mergerfs/build/mergerfs
doak@flatbox:~$ which mergerfs-fusermount 
/usr/bin/mergerfs-fusermount

It works when mergerfs is not installed by Debian:

doak@flatbox:/tmp$ ~/daten/software/linux/mergerfs/build/mergerfs branch merged
warning: mergerfs is not running as root and may not work correctly
fusermount: mount failed: Operation not permitted
doak@flatbox:/tmp$ sudo apt-get remove -y mergerfs >/dev/null
doak@flatbox:/tmp$ ~/daten/software/linux/mergerfs/build/mergerfs branch merged
warning: mergerfs is not running as root and may not work correctly
doak@flatbox:/tmp$ mount | grep merged
/tmp/branch on /tmp/merged type fuse.mergerfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,default_permissions)

46212 02:07:12.744338 mount("/tmp/mergerfs/branch", ".", "fuse", MS_NOSUID|MS_NODEV, "default_permissions,fd=3,rootmode=40000,user_id=1000,group_id=1000") = -1 EPERM (Operation not permitted) <0.000009>

strace revleals that the mount() system call even fails in case it succeeds (on my machine™). It also differs in the third argument:

doak@flatbox:/tmp$ mount | grep merged
doak@flatbox:/tmp$ strace ~/daten/software/linux/mergerfs/build/mergerfs branch merged 2>&1 | grep mount
mount("/tmp/branch", "/tmp/merged", "fuse.mergerfs", MS_NOSUID|MS_NODEV, "default_permissions,fd=5,rootmod"...) = -1 EPERM (Operation not permitted)
doak@flatbox:/tmp$ mount | grep merged
/tmp/branch on /tmp/merged type fuse.mergerfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,default_permissions)

Honestly, I don't understand why it succeeds eventually. Here is the full strace.log for the successful mount (i.e. mount shows the merged directory):

doak@flatbox:/tmp$ mount | grep merged
doak@flatbox:/tmp$ strace ~/daten/software/linux/mergerfs/build/mergerfs branch merged &>strace.log
doak@flatbox:/tmp$ mount | grep merged
/tmp/branch on /tmp/merged type fuse.mergerfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,default_permissions)

from mergerfs.

Just had a look into mergerfs' readme:

If mergerfs doesn't work as a type it could be due to how the mount.mergerfs tool was installed. Must be in /sbin/ with proper permissions.

Although it makes no sense, since there is no difference regarding the setuid bit in case merferfs have or have not been installed by the distribution, I just want to mention it: the setuid bit is not set for these two binaries:

doak@flatbox:~$ ls -l /sbin/mount.{fuse3,mergerfs}
-rwxr-xr-x 1 root root 18664 Jan 12 16:46 /sbin/mount.fuse3
-rwxr-xr-x 1 root root 14568 Sep  2  2022 /sbin/mount.mergerfs

from mergerfs.