Giter Club home page Giter Club logo

Comments (13)

robertswiecki avatar robertswiecki commented on August 16, 2024 2

Also, Aas far as I remember it will not mount if there's something mounted on top of your /proc. E.g. some people mount

/dev/null

on top of /proc/kcore

or similar, to make contents of some files in /proc inaccessible. Maybe it's the same problem.

from nsjail.

robertswiecki avatar robertswiecki commented on August 16, 2024 1

Nice! I guess we can remove relro.

Can you do,

mount | grep /proc

on your system. Proc is special, it can be only mounted with the same flags it's mounted on your global namespace. Maybe it's missing some flag, like MS_NOSUID|MS_NODEV|MS_NOEXEC or so

from nsjail.

ebadi avatar ebadi commented on August 16, 2024
mount
rootfs on / type rootfs (rw,size=100404k,nr_inodes=25101)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
cgroup on /sys/fs/cgroup type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset,cpu,cpuacct,blkio,memory,devices,freezer,net_cls,perf_event,pids,debug)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
tmpfs on /dev type tmpfs (rw,nosuid,relatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,mode=600,ptmxmode=000)
debugfs on /sys/kernel/debug type debugfs (rw,noatime)

https://www.gnu.org/software/libc/manual/html_node/Mount_002dUnmount_002dRemount.html

EACCES

        The filesystem is inherently read-only (possibly due to a switch on the device) and the process attempted to mount it read/write (by setting the MS_RDONLY bit off).
        special_file or dir is not accessible due to file permissions.
        special_file is not accessible because it is in a filesystem that is mounted with the MS_NODEV option. 
 cat .config | grep PROC
CONFIG_KERNEL_PROC_PAGE_MONITOR=y
CONFIG_KERNEL_PROC_PID_CPUSET=y
# CONFIG_BUSYBOX_DEFAULT_NPROC is not set
# CONFIG_BUSYBOX_DEFAULT_FEATURE_TOP_SMP_PROCESS is not set
# CONFIG_PROCD_SHOW_BOOT is not set
# CONFIG_PROCD_ZRAM_TMPFS is not set

I also set the following flags but it didn't change anything.

+CONFIG_KERNEL_DEVTMPFS=y
+CONFIG_KERNEL_DEVTMPFS_MOUNT=y

Additionally I tried --disable_proc

root@LEDE:/# nsjail -Me --chroot / --disable_proc -- /bin/echo "ABC"
[2017-10-27T11:12:14+0000] Mode: STANDALONE_EXECVE
[2017-10-27T11:12:14+0000] Jail parameters: hostname:'NSJAIL', chroot:'/', process:'/bin/echo', bind:[::]:0, max_conns_per_ip:0, time_limit:0, personality:0, daemonize:false, clone_newnet:true, clone_newuser:true, clone_newns:true, clone_newpid:true, clone_newipc:true, clonew_newuts:true, clone_newcgroup:false, keep_caps:false, tmpfs_size:4194304, disable_no_new_privs:false, max_cpus:0
[2017-10-27T11:12:14+0000] Mount point: src:'/' dst:'/' type:'' flags:MS_RDONLY|MS_BIND|MS_REC|MS_PRIVATE|0 options:'' isDir:true
[2017-10-27T11:12:14+0000] Uid map: inside_uid:0 outside_uid:0 count:1 newuidmap:false
[2017-10-27T11:12:14+0000] [W][2375] cmdlineLogParams():247 Process will be UID/EUID=0 in the global user namespace, and will have user root-level access to files
[2017-10-27T11:12:14+0000] Gid map: inside_gid:0 outside_gid:0 count:1 newgidmap:false
[2017-10-27T11:12:14+0000] [W][2375] cmdlineLogParams():258 Process will be GID/EGID=0 in the global user namespace, and will have group root-level access to files
[2017-10-27T11:12:14+0000] [E][2] mountInitNsInternal():373 pivot_root('/tmp/nsjail.root', '/tmp/nsjail.root'): Invalid argument
root@LEDE:/# 

from nsjail.

robertswiecki avatar robertswiecki commented on August 16, 2024

Honestly, no idea why even pivot_root doesn't work here :(. I guess reading kernel sources and trying to figure something out is the only idea here.

from nsjail.

ebadi avatar ebadi commented on August 16, 2024

I compiled lede on ARM few times today with different flags but I think the problem is not architecture nor kernel flags. I'm convinced that the problem lies, somehow, in the file system layout and how file systems are mounted and the way nsjail handles different situations.

Trying nsjail on Linux LEDE 4.9.58 #0 SMP Thu Oct 26 17:15:27 2017 armv7l GNU/Linux

boot with initramfs embedded in:


qemu-system-arm -nographic -M virt -cpu cortex-a15 -m 64 -kernel bin/targets/armvirt/32/lede-armvirt-32-zImage-initramfs 

results in: mountInitNsInternal():373 pivot_root('/tmp/nsjail.root', '/tmp/nsjail.root'): Invalid argument

rootfs on / type rootfs (rw,size=20652k,nr_inodes=5163)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)

Booting with a separate rootfs:

qemu-system-arm -nographic -M virt -cpu cortex-a15 -m 64 -kernel bin/targets/armvirt/32/lede-armvirt-32-zImage -drive file=bin/targets/armvirt/32/lede-armvirt-32-root.ext4,format=raw,if=virtio -append 'root=/dev/vda rootwait'

results in: mountRemountRO():248 mount('/', flags:MS_RDONLY|MS_REMOUNT|MS_NOATIME|0): Operation not permitted

/dev/root on / type ext4 (rw,noatime,block_validity,delalloc,barrier,user_xattr,acl)
devtmpfs on /dev type devtmpfs (rw,relatime,size=24848k,nr_inodes=6212,mode=755)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)

Finally nsjail on Raspbian Linux raspberrypi 4.9.41-v7+ #1023 SMP Tue Aug 8 16:00:15 BST 2017 armv7l GNU/Linux works perfectly fine.

pi@raspberrypi:~/nsjail $ mount 
/dev/mmcblk0p2 on / type ext4 (rw,noatime,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=470180k,nr_inodes=117545,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
mqueue on /dev/mqueue type mqueue (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=94956k,mode=700,uid=1000,gid=1000)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)

from nsjail.

robertswiecki avatar robertswiecki commented on August 16, 2024

Thanks for checking, it's still mysterious, though I'll try to download this image/kernel and test it. In the meantime, would it be possible for you to do the following tests.

1). in mountRemountRO change unsigned long new_flags = MS_REMOUNT | MS_RDONLY | MS_BIND -> unsigned long new_flags = MS_REMOUNT | MS_REC | MS_RDONLY | MS_BIND

2). I think block_validity,delalloc,barrier,user_xattr,acl are the options, as in the 5th argument of the mount syscall. Maybe these need to be repeated for remounting? Like maybe, inside mountRemountRO changing 5th argument of mount() to this string block_validity,delalloc,barrier,user_xattr,acl instead of current 0/NULL if the mount point is /

from nsjail.

ebadi avatar ebadi commented on August 16, 2024

unsigned long new_flags = MS_REMOUNT | MS_RDONLY | vfs.f_flag;

I tried all these:

unsigned long new_flags = MS_REMOUNT | MS_RDONLY | MS_REC ;
unsigned long new_flags = MS_REMOUNT | MS_RDONLY | MS_REC | vfs.f_flag;
unsigned long new_flags = MS_REMOUNT | MS_REC | MS_RDONLY | MS_BIND ;

result for all:

 mountMount():207 mount('src:'[NULL]' dst:'/proc' type:'proc' flags:MS_RDONLY|0 options:'' isDir:true') src:'none' dst:'/tmp/nsjail.root//proc' failed: Operation not permitted

I also changed the fifth argument of the mount syscall as you suggested, both for mountMount and `mountRemountRO' but didn't change anything!

from nsjail.

robertswiecki avatar robertswiecki commented on August 16, 2024

Are you using the newest nsjail. I've change the logic of RemountRO recently. The previous one appending vfs.f_flag was incorrect, as vfs.f_flag contain some flags which are not present as flags for mount().

Specifically, this: https://github.com/google/nsjail/blob/master/mount.c#L244

from nsjail.

ebadi avatar ebadi commented on August 16, 2024

Yes, I used an older version as the new versions break my build.
This pull request can solve some of my build problems.
I also hard-coded ST_RELATIME in mount.c : #define ST_RELATIME 4096 as a temporarily solution for: mount.c:253:20: error: 'ST_RELATIME' undeclared (first use in this function)

With the recent nsjail changes it seems that it goes further but ...

[2017-11-08T15:38:11+0000] [W][1] mountMount():207 mount('src:'[NULL]' dst:'/proc' type:'proc' flags:MS_RDONLY|0 options:'' isDir:true') src:'none' dst:'/tmp/nsjail.root//proc' failed: Operation not permitted
[2017-11-08T15:38:11+0000] [W][1] mountMount():212 procfs can only be mounted if the original /proc doesn't have any other file-systems mounted on top of it (e.g. /dev/null on top of /proc/kcore): Invalid argument

from nsjail.

robertswiecki avatar robertswiecki commented on August 16, 2024

Thanks for the patches. I merges, and tweaked it a bit.

Feel free to try with --disable_proc, if that works, then proc would be the last thing that needs to be fixed.

from nsjail.

robertswiecki avatar robertswiecki commented on August 16, 2024

If course, you might want to try to add some flags to "proc" (e.g. in cmdline.c where proc is added to the mount list)
e.g.

MS_BIND, MS_REC

from nsjail.

robertswiecki avatar robertswiecki commented on August 16, 2024

Hm.. MS_BIND won't work, but maybe MS_REC will.

Also, in outputs of your mount command there's

/proc/sys/fs/binfmt_misc

it might be the culprit for mount/proc failing. But, from what I can tell you're testing with LEDE, and not with Raspberry Pi now?

from nsjail.

ebadi avatar ebadi commented on August 16, 2024

nsjail works without any problem on Rasbian(arm) on raspberrypi. So I don't think /proc/sys/fs/binfmt_misc was the problem.

Yes; --disable_proc makes nsjail works both ARM and MIPS if it boots with a separate rootfs:(without initramfs).
It still has the problem with initramfs but feel free to close the issue.

Thanks again.

from nsjail.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.