Giter Club home page Giter Club logo

junest's Introduction

JuNest

The lightweight Arch Linux based distro that runs, without root privileges, on top of any other Linux distro.

JuNest

Project Status Donation Communication
Build status OpenHub Github Sponsors PayPal Buy me a coffee Join the Discord server at https://discord.gg/ttfBT7MKve

Table of Contents

Description

JuNest (Jailed User Nest) is a lightweight Arch Linux based distribution that allows the creation of disposable and partially isolated GNU/Linux environments within any generic GNU/Linux host OS and without requiring root privileges to install packages.

JuNest is built around pacman, the Arch Linux package manager, which allows access to a wide range of packages from the Arch Linux repositories.

The main advantages of using JuNest include:

  • Install packages without root privileges.
  • Create partially isolated environments in which you can install packages without risking mishaps on production systems.
  • Access a wider range of packages, particularly on GNU/Linux distros with comparatively limited repositories (such as CentOS and Red Hat).
  • Run on a different architecture from the host OS via QEMU.
  • Available for x86_64 and arm architectures but you can build your own image from scratch too!
  • All Arch Linux lovers can enjoy their favourite distro everywhere!

JuNest follows the Arch Linux philosophy.

How different is JuNest from Docker and Vagrant?

Although JuNest sounds similar to a virtualisation/Linux container-like system, JuNest is quite different from solutions like Docker or Vagrant. In fact, the purpose of JuNest is not to build a completely isolated environment but, conversely, to provide the ability to run programs as if they were running natively from the host OS. Almost everything is shared between the host OS and the JuNest sandbox (kernel, process subtree, network, mounting, etc) and only the root filesystem gets isolated (since the programs installed in JuNest need to reside elsewhere).

This allows interaction between processes belonging to both host OS and JuNest. For example, you can install the top command in JuNest and use it to monitor processes belonging to the host OS.

Installation

Dependencies

JuNest comes with a very short list of dependencies in order to be installed in most of GNU/Linux distributions. Before installing JuNest be sure that all dependencies are properly installed in your system:

Installation from git repository

Just clone the JuNest repo somewhere (for example in ~/.local/share/junest):

git clone https://github.com/fsquillace/junest.git ~/.local/share/junest
export PATH=~/.local/share/junest/bin:$PATH

Optionally you want to use the wrappers to run commands installed in JuNest directly from host:

export PATH="$PATH:~/.junest/usr/bin_wrappers"

Update your ~/.bashrc or ~/.zshrc to get always the wrappers available.

Installation using AUR (Arch Linux only)

If you are using an Arch Linux system you can, alternatively, install JuNest from the AUR repository. JuNest will be located in /opt/junest/

Quickstart

Setup environment

The first operation required is to install the JuNest environment in the location of your choice via JUNEST_HOME environment variable (it must contain an absolute path) which by default is ~/.junest:

junest setup

The script will download the image from the repository and will place it to the default directory ~/.junest.

Access to environment

JuNest uses the Linux namespaces (aka ns) as the default backend program. To access via ns just type:

junest

You can use the command sudo to acquire fakeroot privileges and install/remove packages.

Alternatively, you can access fakeroot privileges without using sudo all the time with the -f (or --fakeroot) option:

junest -f

Another execution mode is via Proot:

junest proot [-f]

There are multiple backend programs, each with its own pros/cons. To know more about the JuNest execution modes depending on the backend program used, see the Usage section below.

Run JuNest installed programs directly from host OS

Programs installed within JuNest can be accessible directly from host machine without entering into a JuNest session (namely, no need to call junest command first). For instance, supposing the host OS is an Ubuntu distro you can directly run pacman by simply updating the PATH variable:

export PATH="$PATH:~/.junest/usr/bin_wrappers"
sudoj pacman -S htop
htop

By default the wrappers use ns mode. To use the ns --fakeroot you can use the convenient command helper sudoj. For more control on backend modes you can use the JUNEST_ARGS environment variable too. For instance, if you want to run iftop with real root privileges:

sudoj pacman -S iftop
sudo JUNEST_ARGS="groot" iftop

Bin wrappers can be always recreated (e.g. in case for some reasons they get corrupted) with:

junest create-bin-wrappers -f

Bin wrappers are automatically generated each time they get installed inside JuNest. This only works for executables located in /usr/bin path. For executables in other locations (say /usr/mybinpath) you can only create wrappers manually by executing the command:

junest create-bin-wrappers --bin-path /usr/mybinpath

Obviously, to get access to the corresponding bin wrappers you will need to update your PATH variable accordingly:

export PATH="$PATH:~/.junest/usr/mybinpath_wrappers"

Install packages from AUR

In ns mode, you can easily install package from AUR repository using the already available yay command. In proot mode, JuNest does no longer support the building of AUR packages.

Remember that in order to build packages from AUR, base-devel package group is required first:

pacman -S base-devel

JuNest uses a modified version of sudo provided by junest/sudo-fake. And the original core/sudo package will be ignored (and must not be installed) during the installation of base-devel.

Have fun!

If you are new on Arch Linux and you are not familiar with pacman package manager visit the pacman rosetta page.

Usage

There are three different ways you can run JuNest depending on the backend program you decide to use.

Linux namespaces based

The Linux namespaces represents the default backend program for JuNest. The requirements for having Linux namespaces working are:

  1. Kernel starting from Linux 3.8 allows unprivileged processes to create user and mount namespaces.
  2. The Linux kernel distro must have the user namespace enabled.

In the last years, the majority of GNU/Linux distros have the user namespace enabled by default. This means that you do not need to have root privileges to access to the JuNest environment via this method. This wiki provides the state of the user namespace on several GNU/Linux distros.

In order to run JuNest via Linux namespaces:

  • As normal user - Allow to make basic operations or install/remove packages with sudo command: junest ns or junest
  • As fakeroot - Allow to install/remove packages: junest ns -f or junest -f

This mode is based on the fantastic bubblewrap command.

PRoot based

Proot represents a portable solution which allows unprivileged users to execute programs inside a sandbox and works well in most of GNU/Linux distros available.

In order to run JuNest via Proot:

  • As normal user - Allow to make basic operations: junest proot

  • As fakeroot - Allow to install/remove packages: junest proot -f

In proot mode, the minimum recommended Linux kernel for the host OS is 2.6.32 on x86 (64 bit) and ARM architectures. It is still possible to run JuNest on lower 2.6.x host OS kernels but errors may appear, and some applications may crash. For further information, read the Troubleshooting section below.

Chroot based

This solution suits only for privileged users. JuNest provides the possibility to run the environment via chroot program. In particular, it uses a special program called GRoot, a small and portable version of arch-chroot wrapper, that allows to bind mount directories specified by the user, such as /proc, /sys, /dev, /tmp, /run/user/<id> and $HOME, before executing any programs inside the JuNest sandbox. In case the mounting will not work, JuNest is even providing the possibility to run the environment directly via the pure chroot command.

In order to run JuNest via chroot solutions:

  • As root via GRoot - Allow to have fully root privileges inside JuNest environment (you need to be root for executing this): junest groot

  • As root via chroot - Allow to have fully root privileges inside JuNest environment (you need to be root for executing this): junest root

Execution modes comparison table

The following table shows the capabilities that each backend program is able to perform:

QEMU Root privileges required Manage Official Packages Manage AUR Packages Portability Support User modes
Linux Namespaces NO NO YES YES Poor YES Normal user and fakeroot
Proot YES NO YES NO YES YES Normal user and fakeroot
Chroot NO YES YES YES YES YES root only

Advanced usage

Build image

You can build a new JuNest image from scratch by running the following command:

junest build [-n]

The script will create a directory containing all the essentials files in order to make JuNest working properly (such as pacman and proot). The option -n will skip the final validation tests if they are not needed. Remember that the script to build the image must run in an Arch Linux OS with arch-install-scripts and the base-devel packages installed. To change the build directory just use the JUNEST_TEMPDIR (by default /tmp).

After creating the image junest-x86_64.tar.gz you can install it by running:

junest setup -i junest-x86_64.tar.gz

For more details, you can also take a look at junest-builder that contains the script and systemd service used for the automatic building of the JuNest image.

Related wiki page:

Run JuNest using a different architecture via QEMU

The following command will download the ARM JuNest image and will run QEMU in case the host OS runs on x86_64 architecture:

$> export JUNEST_HOME=~/.junest-arm
$> junest setup -a arm
$> junest proot -- uname -m
armv7l

Bind directories

To bind a host directory to a guest location:

junest -b "--bind /home/user/mydata /mnt/mydata"

Or using proot arguments:

junest proot -b "-b /mnt/mydata:/home/user/mydata"

The option -b to provide options to the backend program will work with PRoot, Namespace and GRoot backend programs. Check out the backend program options by passing --help option:

junest [u|g|p] -b "--help"

Systemd integration

Although JuNest has not been designed to be a complete container, it is even possible to virtualize the process tree thanks to the systemd container. The JuNest containter allows to run services inside the container that can be visible from the host OS through the network. The drawbacks of this are that the host OS must use systemd as a service manager, and the container can only be executed using root privileges.

To boot a JuNest container:

sudo systemd-nspawn -bD ~/.junest

Related wiki page:

Internals

Automatic fallback for all the dependent host OS executables

JuNest attempts first to run the executables in the host OS located in different positions (/usr/bin, /bin, /usr/sbin and /sbin). As a fallback it tries to run the same executable if it is available in the JuNest environment.

Automatic building of the JuNest images

There is a periodic automation build of the JuNest images for x86_64 arch only. The JuNest image for arm architecture may not be always up to date because the build is performed manually.

Static QEMU binaries

There are static QEMU binaries included in JuNest image that allows to run JuNest in a different architecture from the host system. They are located in /opt/qemu directory.

Troubleshooting

For Arch Linux related FAQs take a look at the General troubleshooting page.

Cannot use AUR repository

Q: Why do I get the following error when I try to install a package?

Cannot find the gzip binary required for compressing man and info pages.

A: JuNest comes with a very basic number of packages. In order to install AUR packages you need to install the package group base-devel first that contains all the essential packages for compiling from source code (such as gcc, make, patch, etc):

#> pacman -S base-devel

Remember to not install core/sudo as it conflicts with junest/sudo-fake package.

Can't set user and group as root

Q: In ns mode when installing package I get the following error:

warning: warning given when extracting /usr/file... (Can't set user=0/group=0 for
/usr/file...)

A: This is because as fakeroot is not possible to set the owner/group of files as root. The package will still be installed correctly even though this message is showed.

Could not change the root directory in pacman

No servers configured for repository

Q: Why I cannot install packages?

#> pacman -S lsof
Packages (1): lsof-4.88-2

Total Download Size:    0.09 MiB
Total Installed Size:   0.21 MiB

error: no servers configured for repository: core
error: no servers configured for repository: community
error: failed to commit transaction (no servers configured for repository)
Errors occurred, no packages were upgraded.

A: You need simply to update the mirrorlist file according to your location:

# Uncomment the repository line according to your location
#> nano /etc/pacman.d/mirrorlist
#> pacman -Syy

Locate the package for a given file

Q: How do I find which package a certain file belongs to?

A: JuNest is a really small distro, therefore you frequently need to find the package name for a certain file. pkgfile is an extremely useful package that allows you to detect the package of a given file. For instance, if you want to find the package name for the command getopt:

#> pacman -S pkgfile
#> pkgfile --update
$> pkgfile getop
core/util-linux

Alternatively, you can use directly pacman command only. Take a look here.

Kernel too old

Q: Why do I get the error: "FATAL: kernel too old"?

A: This is because the binaries from the precompiled package are compiled for Linux kernel 2.6.32. When JuNest is started without further options, it tries to run a shell from the JuNest chroot. The system sees that the host OS kernel is too old and refuses to start the shell.

The solution is to present a higher "fake" kernel version to the JuNest chroot. PRoot offers the -k option for this, and JuNest passes this option on to PRoot when -p is prepended. For example, to fake a kernel version of 3.10, issue the following command:

$> junest proot -b "-k 3.10"

As Arch Linux ships binaries for kernel version 2.6.32, the above error is not unique to the precompiled package from JuNest. It will also appear when trying to run binaries that were later installed in the JuNest chroot with the pacman command.

In order to check if an executable inside JuNest chroot is compatible with the kernel of the host OS just use the file command, for instance:

$> file ~/.junest/usr/bin/bash
ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked
(uses shared libs), for GNU/Linux 2.6.32,
BuildID[sha1]=ec37e49e7188ff4030052783e61b859113e18ca6, stripped

The output shows the minimum recommended Linux kernel version.

Kernel doesn't support private futexes

Q: Why do I get the warning: "kompat: this kernel doesn't support private futexes and PRoot can't emulate them."?

A: This happens on older host OS kernels when the trick of showing a fake kernel version to the JuNest chroot is applied (see above: Kernel too old).

The consequence of showing a fake kernel version to the JuNest chroot is that in the background, PRoot needs to translate requests from applications in the chroot to the old kernel of the host OS. Some of the newer kernel functionality can be emulated, but private futexes cannot be translated.

Private Futexes were introduced in Linux kernel 2.6.22. Therefore, the above problem likely appears on old Linux systems, for example RHEL5 systems, which are based on Linux kernel 2.6.18. Many of the core tools like which, man, or vim run without problems while others, especially XOrg-based programs, are more likely to show the warning. These are also more likely to crash unexpectedly.

Currently, there is no (easy) workaround for this. In order to be fully compatible with kernels below 2.6.22, both the precompiled package from JuNest and all software that is installed later needs to be compiled for this kernel. Most likely this can only be achieved by building the needed software packages from source, which kind of contradicts JuNest's distro-in-a-distro philosophy.

SUID permissions

Q: Why I do not have permissions for ping?

$> ping www.google.com
ping: icmp open socket: Operation not permitted

A: The ping command uses suid permissions that allow to execute the command using root privileges. The fakeroot mode is not able to execute a command set with suid, and you may need to use root privileges. There are other few commands that have suid permission, you can list the commands from your JuNest environment with the following command:

$> find /usr/bin -perm /4000

No characters are visible on a graphic application

Q: Why I do not see any characters in the application I have installed?

A: This is probably because there are no fonts installed in the system.

To quick fix this, you can just install a fonts package:

#> pacman -S gnu-free-fonts

Differences between filesystem and package ownership

Q: Why do I get warning when I install a package using root privileges?

#> pacman -S systat
...
warning: directory ownership differs on /usr/
filesystem: 1000:100  package: 0:0
...

A: In these cases the package installation went smoothly anyway. This should happen every time you install package with root privileges since JuNest will try to preserve the JuNest environment by assigning ownership of the files to the real user.

Unprivileged user namespace disable at kernel compile time or kernel too old

Q: Why do I get this warning when I run JuNest via Linux namespaces?

$> junest ns
Unprivileged user namespace is disabled at kernel compile time or kernel too old (<3.8). Proceeding anyway...

A: This means that JuNest detected that the host OS either does not have a newer kernel version or the unprivileged user namespace is not enabled at kernel compile time. JuNest does not stop the execution of the program but it attempts to run it anyway. Try to use Proot as backend program in case is not possible to invoke namespaces.

Unprivileged user namespace disabled

Q: Why do I get this warning when I run JuNest via Linux namespaces?

$> junest ns
Unprivileged user namespace disabled. Root permissions are required to enable it: sudo sysctl kernel.unprivileged_userns_clone=1

A: This means that JuNest detected that the host OS either does not have a newer Linux version or the user namespace is not enabled. JuNest does not stop the execution of the program but it attempts to run it anyway. If you have root permissions try to enable it, otherwise try to use Proot as backend program.

More documentation

There are additional tutorials in the JuNest wiki page.

Contributing

Contributions are welcome! You could help improving JuNest in the following ways:

Donating

To sustain the project please consider funding by donations through the GitHub Sponsors page.

Authors

JuNest was originally created in late 2014 by Filippo Squillace ([email protected]).

Here is a list of really appreciated contributors!

junest's People

Contributors

adamcavendish avatar ayaka14732 avatar cfriesicke avatar cosmojg avatar droidfreak32 avatar escape0707 avatar fsquillace avatar hodapp512 avatar neiser avatar schance995 avatar soraxas avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

junest's Issues

Use only the compat proot binaries

Currently, JuJu uses two binaries in order to run the environment with proot.
Using only the compat proot binaries would make the code a bit cleaner.

Deprecate getopt

The main added value of getopt is just the fact that it allows collapsed options (-kv -> -k -v). I am inclined to remove it since I prefer to have JuJu script even more portable.

Building JuJu images in JuJu

As per #65 I think it would be really useful to be able to build JuJu in JuJu. Being able to bootstrap an Arch environment without having the parent is extremely useful. My intention is to completely build this in CI and create packages. pacstrap seemed to die when mounting, or chrooting depending on the distro underneath.

Add support for DBus service and more

Some essentials services for running properly the application could be handled inside JuJu.
This mostly an investigation issue to understand the feasibility of this feature.

Fix the spawn of bash for testing die function

On some bash the test act diferently:
test_build_image_juju...OK
test_check_cli...FAIL
test_delete_juju...OK
test_help...OK
test_run_juju_as_fakeroot...OK
test_run_juju_as_root...OK
test_run_juju_as_user...OK
test_version...OK

Decide whether the juju scripts need to be placed in JuJu image

There are two possible choices.

  • JuJu script are placed in JuJu environment during the setup:
    • Pros: The user will always have the most recent up to date master branch
    • Cons: The image itself will not be self-contained
  • JuJu script are placed in JuJu image directly (current behavior)

Add a config file with user mounts --bind

It would be nice to have a config file (probably inside JUJU_HOME or something) with extra user specified paths to bind, so the command goes from juju -p -b /myworkdir mycommand to juju mycommand with /myworkdir automagically mounted on the fakechroot (the same way proot -S does)

image download fails with Busybox's wget applet

When wget from Busybox is used the download fails with 404, changing the core script not to check for wget and use curl instead worked. If you make the script prefer curl over wget such cases would be avoided.

Second method for installing JuJu

Provide a second method for installing juju based on downloading directly the juju image.
juju image contains the juju script as well.

pacman -Syu hangs at the end

JuJu is really great, I'm using it with Travis-CI as their Ubuntu images are terribly out of date. However, when I try to do a system update (pacman -Syu) the command never seems to complete and just hangs.

Unrelated to the issue: why can't we build a JuJu image in JuJu?

Hard links fail if the host os uses multiple partitions

Since hard links can't go across partitions. I think this could be solved by tarring to symbolic links instead?

I can't seem to solve this with the current tar archive because there doesn't seem to be an option to convert hard links to symbolic links when untarring.

Check if alternatives against the yaourt/makepkg patches can be found

In the previous release of makepkg there was the option --asroot that allowed to build packages directly using root privileges. In the current release makepkg cannot be used with root access anymore for security reasons.

The JuJu uses a patch for yaourt and makepkg scripts that removes the root check of ($EUID == 0)
https://github.com/fsquillace/juju/blob/master/lib/core.sh#L300.
Furthermore, those scripts uses sudo or su depending if sudo is installed or not.

Ideally, it would be nice if we can run smoothly yaourt and makepkg without creating patches on them.

There are two improvements that can be done here:

  • Bypassing sudo and su commands (maybe by intercepting calls for euid and uid?) so that we avoid failures when we run them. The bypass on those commands should work for the entire JuJu session.
  • Mocking the EUID variable . The mock on that variable should not work for the entire JuJu session but should be used only for yaourt/makepkg scripts.

The current behavior for sudo inside juju is the following:

  • As a fakeroot user:

    > sudo ls

    sudo: setresgid() [1419771904, 0, 1419771904] -> [-1, 1419771904, -1]: Operation not permitted

  • As a normal user:

    $> sudo ls
    sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set

The current behavior for su inside juju is the following:

  • As a fakeroot user it works well:

    > su -c ls

    bin lib tests

  • As a normal user it requires password:

    $> su -c ls
    Password:

Reference: http://allanmcrae.com/2015/01/replacing-makepkg-asroot/

Validation test

Create a script in tests/ directory that make a final check of juju:

  • Download the image
  • Install package with pacman using proot
  • Install package with yaourt using proot
  • Access using root user

Wrong fallback to seccomp mode

After a juju session it possible that JuJu tries to fallback to seccomp mode.
The seccomp mode need to be used only when the access to JuJu at the first time fails.

Needs a new name?

Hi-

First off, I just wanted to thank you for making this. JuJu is a super useful tool for allowing users to install packages in a nice sandbox without messing up the rest of the system.

This isn't a technical issue, but I noticed that the project name is basically identical to Canonical's cloud software management tool (which appears to be much older; 2011 vs 2014). You might want to consider switching to another name since, beyond causing confusion, this prevents the two packages from both being installed on the same system (I can't think of a scenario where anyone would want to do this right now, but hey- who knows?).

Cheers,
John Pellman

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.