gramineproject / graphene Goto Github PK

Graphene / Graphene-SGX - a library OS for Linux multi-process applications, with Intel SGX support

License: GNU Lesser General Public License v3.0

Makefile 1.77% Python 6.79% Assembly 1.36% C 88.39% C++ 0.31% Shell 0.32% GDB 0.14% Dockerfile 0.26% Emacs Lisp 0.02% Meson 0.64%

linux compatibility virtualization sgx libraryos

graphene's Introduction

PROJECT MOVED TO A NEW REPOSITORY

Note: Graphene project was renamed to Gramine and moved to a new location: https://github.com/gramineproject/gramine. All development will continue there.

Graphene Library OS with Intel SGX Support

A Linux-compatible Library OS for Multi-Process Applications

NOTE: We are in the middle of transitioning our buildsystem to Meson, and the build procedures are changing. See Building instructions for an up-to-date build tutorial.

What is Graphene?

Graphene is a lightweight library OS, designed to run a single application with minimal host requirements. Graphene can run applications in an isolated environment with benefits comparable to running a complete OS in a virtual machine -- including guest customization, ease of porting to different OSes, and process migration.

Graphene supports native, unmodified Linux binaries on any platform. Currently, Graphene runs on Linux and Intel SGX enclaves on Linux platforms.

In untrusted cloud and edge deployments, there is a strong desire to shield the whole application from rest of the infrastructure. Graphene supports this “lift and shift” paradigm for bringing unmodified applications into Confidential Computing with Intel SGX. Graphene can protect applications from a malicious system stack with minimal porting effort.

Graphene is a growing project and we have a growing contributor and maintainer community. The code and overall direction of the project are determined by a diverse group of contributors, from universities, small and large companies, as well as individuals. Our goal is to continue this growth in both contributions and community adoption.

Release candidate version of Graphene 1.2 available

Graphene has evolved a lot since our last major release. Over the last few months, we have made significant updates to provide a stable version that supports deploying key workloads with Intel SGX. We’ve rewritten major subsystems, done a significant update to the build and packaging scripts, extended test coverage, and improved the CI/CD process. We’ve reviewed and hardened specific security aspects of Graphene, and increased stability for long-running and heavy workloads.

Graphene also includes full SGX Attestation support, protected files support, multi-process support with encrypted IPC, and support for the upstreamed SGX driver for Linux. We’ve introduced a number of performance optimizations for SGX, and provide mechanisms to more easily deploy in cloud environments with full support for automatic Docker container integration using Graphene Shielded Containers (GSC).

We have a growing set of well-tested applications including machine learning frameworks, databases, webservers, and programming language runtimes.

This version of Graphene is tagged 'v1.2-rc1'. We encourage you to try this out with your workloads and let us know if you’re facing any issues. Please see the release page for release notes and installation instructions.

While we have made significant progress, we are continuing to work towards making Graphene better and adding support for more workloads. The items that we are most immediately working on are tracked in #1544.

In the meantime, we are also in the process of transitioning the Graphene project to a new home within the Confidential Computing Consortium under the Linux Foundation. In Q3 2021 we will provide more details on this, and we expect the next version of Graphene to be released once this transition is complete.

Graphene documentation

The official Graphene documentation can be found at https://graphene.readthedocs.io. Below are quick links to some of the most important pages:

Getting help

For any questions, please send an email to [email protected] (public archive).

For bug reports, post an issue on our GitHub repository: https://github.com/oscarlab/graphene/issues.

Acknowledgments

Graphene Project benefits from generous help of fosshost.org: they lend us a VPS, which we use as toolserver and package hosting.

graphene's People

Contributors

Stargazers

Watchers

Forkers

icodetm rkumar2468 ash9211 xzffwy banne01 flyflypeng huwei5 terry2012 brianmcgillion shmeni sogua1980 woonhak zatkh adombeck liusdu nikileshsa jovanbulck arjun4084346 happy-ferret 0xhwang tempbottle xbee michaelborsellino acsl-technion peasentspring chiache rainfld ysminnpu maverick-zhn macawhn haitaohuang hilvert hanjae neilbd mkow jim8y bahaihe qiufd fooc chenmin46 exfalso cupertinodude rutvik-k taihulight dhilip89 elichai scrusader liujiong63 wnagchenghku lacabra sgx-java sgx-naors daveti h0k5 asu-es-courses tatetian miaogecm dimakuv weili1988 aua237032 daydayup40 bigdata-memory zhangckid jjcollinge bart-jansen woachk dcowhig yamahata shvaliuk vampsj thomasknauth zhangyubao5 cocoppang yzz127 mmspide smherwig omeg michaelbarkdoll deertrodis gssxd gtsvetov hw42 jiazhang0 whulxx kanakraju73 lu-biao prekucki alexandermerritt benesse1899 eckelj leesera wckao ddragoti zhouboxiao gautham103 amerritt-intel martinschmatz huandong-gohigh kushg02 boryspoplawski

graphene's Issues

Build sgx-driver issues

I'd like to compile sgx-driver in graphene(Pal/src/host/Linux-SGX/sgx-driver)
However, linux-sgx-driver has been changed and removed many symbols such as ISGX_IOCTL_XXX (changed to ISGX_IOC_XXX), isgx_create_param, and ISGX_ADD_SKIP_EEXTEND.
So I can't build and load graphene-sgx driver.

licensing clarification

What's the license for published code?

Bug in get_norm_path

This line causes get_norm_path to return an error when a component of a path starts with ...

Example:

cd LibOS/shim/test/native
./pal ls.manifest.sgx /lib/..foo

results in file:/bin/ls: cannot access '/lib/..foo': Interrupted system call.

I wonder why you rewrite a function like get_norm_path. IMO this is bound to cause bugs, possibly leading to vulnerabilities. Wouldn't it be better to reuse existing code for this, for example realpath from the glibc?

poll() on socket returned -EAGAIN

I'm trying to run MySQL server on graphene-sgx.

Calling poll() on socketfd seems little weird.
I checked the source code,
and I found that if the status of the socket is SOCK_LISTENED,
then it always returned EAGAIN(-11).
Why it doesn't wait on poll() and just return it ?

Here's the code.
LibOS/shim/src/socket/fs.c:socket_poll()

static int socket_poll (struct shim_handle * hdl, int poll_type)
{
    struct shim_sock_handle * sock = &hdl->info.sock;
    int ret = 0;

    lock(hdl->lock);

    if (poll_type & FS_POLL_RD) {
        if (sock->sock_type == SOCK_STREAM) {
            if (sock->sock_state == SOCK_CREATED ||
                sock->sock_state == SOCK_BOUND ||
                sock->sock_state == SOCK_SHUTDOWN) {
                ret = -ENOTCONN;
                goto out;
            }
            if (sock->sock_state == SOCK_LISTENED) {
                ret = -EAGAIN;
                goto out;
            }
        }

        if (sock->sock_type == SOCK_DGRAM &&
            sock->sock_state == SOCK_SHUTDOWN) {
            ret = -ENOTCONN;
            goto out;
        }
    }

Can't execute shared objects anymore

Since commit 76967f3, the execution of shared objects in Graphene with SGX doesn't work anymore. I used to execute python3.5, which happens to be compiled as a shared object on Ubuntu 14.04, in Graphene. I will have to fix this. Sadly, this is a very big commit, which makes it hard to pin down the change that is responsible. Do you have an idea which change broke this?

Cannot run Python with SGX (missing libs)

Runtime/resolv is missing in SGX mode

Missing dependencies

I needed the following Ubuntu packages to use Graphene-SGX:

python-protobuf
python-crypto

These aren't mentioned in the documentation anywherre.

building openjdk fails

Building the openjdk test app fails:

$ cd LibOS/shim/test/apps/openjdk
$ make
readlink: missing operand
Try 'readlink --help' for more information.
readlink: missing operand
Try 'readlink --help' for more information.
Package freetype2 was not found in the pkg-config search path.
Perhaps you should add the directory containing `freetype2.pc'
to the PKG_CONFIG_PATH environment variable
No package 'freetype2' found
Package freetype2 was not found in the pkg-config search path.
Perhaps you should add the directory containing `freetype2.pc'
to the PKG_CONFIG_PATH environment variable
No package 'freetype2' found
cd openjdk-7 && \
		make \
		ALT_BOOTDIR= \
		ALT_JDK_IMPORT_PATH= \
		ANT=/usr/bin/ant \
		INSANE="true" \
		FT2_CFLAGS= \
		FT2_LIBS= \
		STATIC_CXX="false" \
		EXTRA_CFLAGS="-fPIC" \
		NO_DOCS="true" \
		BUILD_HEADLESS_ONLY="true" \
		BUILD_HOTSPOT="true" BUILD_JDK="true" \
		BUILD_LANGTOOLS="false" \
		BUILD_JAXP="false" BUILD_JAXWS="false" BUILD_CORBA="false" \
		HOTSPOT_BUILD_JOBS="8" \
		TEST_IN_BUILD="false" \
		product_build
make[1]: Entering directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7'
make \
	        SKIP_FASTDEBUG_BUILD=true \
	        SKIP_DEBUG_BUILD=true \
	         \
	        generic_build_repo_series
make[2]: Entering directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7'
/bin/mkdir -p ./build/linux-amd64/j2sdk-image
/bin/mkdir -p /home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir
/bin/mkdir -p /home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/import


########################################################################
########################################################################
##### Entering hotspot for target(s) all_product                   #####
########################################################################

cd  ./hotspot/make && \
	    make JDK_TOPDIR=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/jdk JDK_MAKE_SHARED_DIR=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/jdk/make/common/shared EXTERNALSANITYCONTROL=true SOURCE_LANGUAGE_VERSION=7 TARGET_CLASS_VERSION=7 MILESTONE=internal BUILD_NUMBER=b00 JDK_BUILD_NUMBER=b00 FULL_VERSION=1.7.0-internal-user_2016_11_09_17_37-b00 PREVIOUS_JDK_VERSION=1.6.0 JDK_VERSION=1.7.0 JDK_MKTG_VERSION=7 JDK_MAJOR_VERSION=1 JDK_MINOR_VERSION=7 JDK_MICRO_VERSION=0 PREVIOUS_MAJOR_VERSION=1 PREVIOUS_MINOR_VERSION=6 PREVIOUS_MICRO_VERSION=0 STATIC_CXX=false PAX_COMMAND=/usr/sbin/paxmark.sh PAX_COMMAND_ARGS="-vm" ARCH_DATA_MODEL=64 COOKED_BUILD_NUMBER=0 ALT_OUTPUTDIR=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir ALT_EXPORT_PATH=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/import BUILD_FLAVOR=product ALT_SLASH_JAVA=/NOT-SET ALT_BOOTDIR=/NO_BOOTDIR HOTSPOT_BUILD_JOBS=8 all_product
INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
INFO: /usr/bin/objcopy cmd found so will create .debuginfo files.
INFO: STRIP_POLICY=min_strip
INFO: ZIP_DEBUGINFO_FILES=1
/bin/sh: 1: cd: can't cd to /NO_BOOTDIR
make[3]: Entering directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make'
cd /home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make; \
	make BUILD_FLAVOR=product VM_TARGET=product generic_build2 ALT_OUTPUTDIR=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir
INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
INFO: /usr/bin/objcopy cmd found so will create .debuginfo files.
INFO: STRIP_POLICY=min_strip
INFO: ZIP_DEBUGINFO_FILES=1
/bin/sh: 1: cd: can't cd to /NO_BOOTDIR
make[4]: Entering directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make'
mkdir -p /home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir
cd /home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir; \
	    make -f /home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make/linux/Makefile \
		       LP64=1 JAVA_HOME=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make OUTPUTDIR=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir GAMMADIR=/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot MAKE_VERBOSE=y HOTSPOT_RELEASE_VERSION=24.95-b00 JRE_RELEASE_VERSION=1.7.0-internal-user_2016_11_09_17_37-b00 HOTSPOT_BUILD_VERSION= INCLUDE_TRACE=1 product
INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
INFO: /usr/bin/objcopy cmd found so will create .debuginfo files.
INFO: STRIP_POLICY=min_strip
INFO: ZIP_DEBUGINFO_FILES=1
/bin/sh: 1: cd: can't cd to /NO_BOOTDIR
make[5]: Entering directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir'
make -f /home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make/linux/Makefile checks
INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
INFO: /usr/bin/objcopy cmd found so will create .debuginfo files.
INFO: STRIP_POLICY=min_strip
INFO: ZIP_DEBUGINFO_FILES=1
/bin/sh: 1: cd: can't cd to /NO_BOOTDIR
make[6]: Entering directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir'
/NO_BOOTDIR/bin/javap javax.xml.transform.TransformerFactory > /dev/null 2>&1; \
	if [ $? -ne 0 ]; then \
	   /NO_BOOTDIR/bin/java -version; \
	  echo "*** An XSLT processor (J2SE 1.4.x or newer) is required" \
	  "to bootstrap this build" 1>&2; \
	  exit 1; \
	fi
/bin/sh: 3: /NO_BOOTDIR/bin/java: not found
*** An XSLT processor (J2SE 1.4.x or newer) is required to bootstrap this build
make[6]: *** [check_j2se_version] Error 1
make[6]: Leaving directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir'
make[5]: *** [linux_amd64_compiler2/debug] Error 2
make[5]: Leaving directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir'
make[4]: *** [generic_build2] Error 2
make[4]: Leaving directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make'
make[3]: *** [product] Error 2
make[3]: Leaving directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make'
make[2]: *** [hotspot-build] Error 2
make[2]: Leaving directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7'
make[1]: *** [build_product_image] Error 2
make[1]: Leaving directory `/home/user/projects/graphene/LibOS/shim/test/apps/openjdk/openjdk-7'
make: *** [openjdk-7/build/linux-amd64/j2sdk-image/jre/bin/java] Error 2

OOM error

Hello,

I am trying to run a quite big c++ program with Graphene in SGX and the program seems to start working, until it reaches the following error:

Fatal process OOM in heap setup
[ 1] ---- return from shim_write(...) = 40

I think it is Out Of Memory error and I tried changing the values of the following lines in the manifest to up to 4 times their original value without any success:
sgx.enclave_size=[SIZE]
sys.stack.size=[# of bytes]
sys.brk.size=[# of bytes]

Any help will be welcome :)

epoll_socket problem sending requests

My configuration is Ubuntu 14.04 64bit on an SGX-enabled i7 CPU with a graphene SGX-PAL.
epoll_trace.tar.gz

Reproduce:
Compile graphene with SGX support
go to native directory: make SGX=1; make SGX_RUN=1
./pal epoll_socket 8000

On another terminal:
telnet 8000

The telnet connects to the client, yet NO requests are sent to the client.

Attached a trace with debug=inline manifest

Problem with write()

Hi,

So my enclave program simply creates a file and starts writing to the file. I wanted to add gibberish to fill up a file to 8GB of size. However, write() crashes after 2GB which is a simple problem of int being used within the code. However, even when I changed them to uint64_t, I still can't create a file larger than 4GB. Can you please check why this is the case?

Thanks!

signal_name returns pointer from alloca

In signal_name, you return a pointer obtained from alloca. This is not allowed, as this memory is allocated on the stack and may be immediately overwritten by the calling function. This results in nice debug output such as:

[    1] ---- shim_rt_sigprocmask(SETMASK,[SIGHUP,SIGINT,SIGQUIT,SIGILL,SIGTR..
[    1] ..AP,SIGABRT,SIGBUS,SIGFPE,SIGKILL,SIGUSR1,SIGSEGV,SIGUSR2,SIGPIPE,S..
[    1] ..IGALRM,SIGTERM,SIGSTKFLT,SIGCHLD,SIGCONT,SIGSTOP,SIGTSTP,SIGTTIN,S..
[    1] ..IGTTOU,9����,9����,9����,9����,9����,9����,9����,9����,9��..
[    1] ..��,9����,9����,9����,9����,9����,9����,9����,9����,9����..
[    1] ..,9����,9����,9����,9����,9����,9����,9����,9����,9����,9�..
[    1] ..���,9����,9����,9����,9����,9����,9����,9����,9����,9����..
[    1] ..,9����,9����,BAD SIGNAL,],[]) = 0

C++ support?

Hi,

I'm not too sure where to find the answer to this, but how much support does Graphene-SGX provide for C++ applications? I've tried writing an example helloworld.cpp application and building/running it with Graphene, but I get errors when trying to link to the common libstdc++ libraries, such as iostream etc.

Does Graphene support C++ applications?

LTP testcases for gettimeofday

LTP testcase gettimeofday01 calls syscall 'gettimeofday' with invalid inputs.
"int shim_do_gettimeofday (struct __kernel_timeval * tv, struct __kernel_timezone * tz)" is not "verifying_write" to 'tv' and 'tz', which is returning "Internal memory fault" in Graphene.

The implementation of gettimeofday in Linux kernel does a verify_write ok tv and tz check before setting them with time of day.

Create file in Graphene indepentently from host filesystem

I want to create a file in Graphene that should be inaccessible from the host, i.e. it should not be stored in the host filesystem. Is this currently supported?

exec() fails if executed with SGX and the target's code segment is bigger than 4096 bytes

If the program executed via pal-Linux-SGX contains a call to a function of the exec() family (e.g. execv() or execl()), the call fails if the target's code segment is bigger than 4096 bytes.

For example, the code segment of the exec_victim binary in LibOS/shim/test/native is smaller than 4096 bytes, so the exec binary in the same directory can execute exec_victim without error:

$ SGX=1 ./pal_loader exec
[...]
Hello World (./exec_victim)!
envp['IN_EXECVE'] = 1
[...]

If we simply add some bytes to the exec_victim's code, for example by printing about 1500 additional characters, the execution fails:

Patched exec_victim.c:

/* -*- mode:c; c-file-style:"k&r"; c-basic-offset: 4; tab-width:4; indent-tabs-mode:nil; mode:auto-fill; fill-column:78; -*- */
/* vim: set ts=4 sw=4 et tw=78 fo=cqt wm=0: */

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char ** argv, const char ** envp)
{
    FILE * out = stdout;

    if (argc > 1) {
        int fd = atoi(argv[argc - 1]);
        printf("inherited file descriptor %d\n", fd);
        out = fdopen(fd, "a");
        if (!out) {
            perror("fdopen");
            exit(1);
        }
    }

    fprintf(out, "Hello World (%s)!\n", argv[0]);
    fprintf(out, "envp[\'IN_EXECVE\'] = %s\n", getenv("IN_EXECVE"));
    fprintf(out, "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 \
");

    return 0;
}

Execution:

$ SGX=1 ./pal_loader exec
[...]
[    1] loading file:./exec_victim as runtime object at 0x0
[    1] memory fault at 0x0 (IP = 0xbd98bcb)
[    1] SIGSEGV handled
[    1] run signal handler 0xbabf81e (11, 0x8fcc580, 0x8fcc608)
[    1] killed by SIGSEGV
[...]

I took a look at it with gdb and the bug seems to be triggered when reading the newly mapped process image further than the first page, 0x400000 to 0x401000, but I couldn't figure out why. The bug doesn't get triggered if the process image is smaller than 4096 bytes (= 1 page).

The backtrace:

#0  _wordcopy_fwd_aligned (dstp=4198392, srcp=139856873025536, len=512) at string/wordcopy.c:97
#1  0x000000000bd96575 in memcpy (dstpp=0x400000, srcpp=<optimized out>, len=8192) at string/memcpy.c:41
#2  0x000000000bd7e467 in file_map (handle=<optimized out>, addr=0x8fcce2a, prot=<optimized out>, 
    offset=<optimized out>, size=8192) at db_files.c:237
#3  0x000000000bd78325 in _DkStreamMap (handle=handle@entry=0xcfb9b10 <mem_pool+16861872>, 
    paddr=paddr@entry=0x8fcce7a, prot=<optimized out>, offset=<optimized out>, size=<optimized out>)
    at db_streams.c:549
#4  0x000000000bd78480 in DkStreamMap (handle=0xcfb9b10 <mem_pool+16861872>, addr=<optimized out>, 
    prot=<optimized out>, offset=<optimized out>, size=<optimized out>) at db_streams.c:584
#5  0x000000000baccc74 in chroot_mmap (hdl=0xba904b8, addr=0x8fccf92, size=8192, prot=5, flags=18, 
    offset=0) at fs/chroot/fs.c:856
#6  0x000000000baec260 in __map_elf_object (file=0xba904b8, fbp=0x8fcd0a0, fbp_len=832, addr=0x0, 
    type=1, remap=0x0) at elf/shim_rtld.c:596
#7  0x000000000baeda1d in __load_elf_object (file=0xba904b8, addr=0x0, type=1, remap=0x0)
    at elf/shim_rtld.c:1051
#8  0x000000000baed86a in load_elf_object (file=0xba904b8, addr=0x0, mapped=0) at elf/shim_rtld.c:1001
#9  0x000000000bb32a19 in shim_do_execve_rtld (hdl=0x0, argv=0x0, envp=0x0) at sys/shim_exec.c:134
#10 0x0000000008fcdff0 in ?? ()
#11 0x0000000000000000 in ?? ()

Memory limitation

Hi, I have a question about the memory allocation.
As far as I understand, we can specify the required memory in the manifest like

sgx.enclave_size = 10240M

However, the problem I meet is that the size is limited up to 4GB ..
I think it is a problem of mmap() in sgx device driver.
It seems isgx_vma_open() of sgx driver does not support memory more than 4GB.

If you have any idea to solve this problem (e.g., allocating 10GB), could you please give me some information?

Thank you so much (and thank you for this wonderful project!)

Python urllib2 cannot work in the enclave.

request = urllib2.Request("http://....")
opener = urllib2.build_opener()
response = opener.open(request, timeout=10)

It will report Permission denied.
I have already specified the network rules in the manifest. And I also write a test about HTTP in C and it works.

How the direct syscalls are handled?

I'm the lead developer for the SGX driver that we are eventually trying to upstream. I just recently studied your project and one question popped into my mind.

Couldn't you bypass your sandbox by directly inserting SYSENTER opcode to the program instead of using libc to make the syscall? Of course with SGX this would not be possible because that would
cause #GP.

In order to fight against such attempts two options come up to my mind:

Patch the binary. This is BTW something that Chromium used to do some years ago for JIT before seccomp filters came around.
Block all direct system calls with a seccomp filter. This would thrawt the promise to run unmodified binaries.

If you have a solution for this issue, maybe it would be good to explain in the wiki because this is really the very first question that popped up when I ready it through.

Graphene support for SGX Simulation mode?

Hi,

One question I'm not sure about is whether or not we can run graphene SGX applications without SGX hardware. In the Intel Sample Code, for example, omitting the flag "SGX_MODE=HW" allows you to run the application in simulation mode (see the README here: https://github.com/01org/linux-sgx/tree/master/SampleCode/SampleEnclave). Is the same true with Graphene? If I specify make "SGX=1", does this force me to require SGX enabled hardware?

Thanks for your help!

graphene-sgx does not compile

Hey,
I got the following problem trying to compile graphene right now:

(On Ubuntu 14.04, able to run native SGX apps)
Cloned a fresh copy of graphene, use latest driver, sdk & psw.
Tried to compile with SGX=1 and got an error while signing enclave:

File "/home/user/graphene/Pal/src/host/Linux-SGX/signer/pal-sgx-sign", line 211, in resolve_uri
raise Exception('Cannot resolve ' + orig_uri + ' or the file does not exist.')
Exception: Cannot resolve /src/libsysdb.so or the file does not exist.

Number of trusted files is limited

The number of trusted files you can have in your SGX manifest seems to somehow be limited by CONFIG_MAX. However, increasing CONFIG_MAX (to e.g. 10240) results in SIGBUS.

fork() system call execution

Hi,

I want to run a benchmark which makes some system calls. I have written a small program to test if some system calls were working, But the fork() function seems to be stuck in a infinite loop and never exits the program. The program runs successfully natively on Linux.

I have attached a part of the program here and the manifest file is similar to the manifest.template of opendjk example in the library.

_code snippet_

           switch (fork()) {
           case -1:            /* Error */
               errExit("fork");

           case 0:             /* Child */
               printf("DOne3 \n");
               CPU_SET(childCPU, &set);
               if (sched_setaffinity(getpid(), sizeof(set), &set) == -1)
                     errExit("sched_setaffinity")
               else
                     printf("Setaffinity success!\n");

               for (j = 0; j < nloops; j++)
                   getppid();

               exit(EXIT_SUCCESS);

_manifest.template snippet_

sys.stack.size = 256K
sys.brk.size = 16M
glibc.heap_size = 16M

sgx.enclave_size = 512M
sgx.thread_num = 16

I cloned and installed the library from GitHub just yesterday after seeing 'bugfix: process fork in LibOS' and ran the program. It gave the output and gets stuck.

WARNING: no physical memory support, process creation will be slow.
sched_setaffinity: Function not implemented

I have followed the SGX Quick Start but no PAL binary in /LibOS/shim/test/native

There is a pal_loader but shell saying it is not built, , or security mode is not supported

Can't connect to Unix socket from inside enclave

Connecting to a socket from inside the enclave fails. For an example, see this code.

I open the socket with socat - UNIX-LISTEN:socket.
Then, when I execute the program without Graphene, it connects successfully:

$ ./connect
Success!

When executed with Graphene (after generating the manifest.sgx and signature), it fails to connect:

$ ~/graphene/Pal/src/pal connect.manifest.sgx
manifest file: file:connect.manifest.sgx
enclave created:
[...]
allowed: file:socket
Failed to connect!

Note that I'm currently at commit f2c36a1, because of #45.

Bug in unmapping untrusted files

In db_files.c#L119 the size argument is calculated as map_start - map_end. Obviously, the size of the memory region is map_end - map_start, like it is calculated in db_files.c#L125.

"bug(): bookkeep/shim_vma.c:205"

When I run a python script through Graphene with SGX support, I get the following error info:

"bug(): bookkeep/shim_vma.c:205"

Does this mean a bug in Graphene or some unsupported features?

Thanks.

The files in the '/proc' directory are empty

I've mounted the ‘/proc’ to the file ‘/proc’ and then I try to read the '/proc/stat' file but cannot read anything。
It actually found the file without throwing the "No such file" exception, just read nothing.
How can I find the files in '/proc' directory?

How to write an SGX application with graphene?

Does graphene support unmodified applications? The Helloworld example needs to invoke the function provided by graphene instead of the original functions from libc.

pal crashes with SIGILL when run with gdb

I wanted to debug a program i ran with graphene, but it crashes with SIGILL when debugged with gdb. Then I realized that graphene always crashes when it is debugged with gdb:

~/projects/graphene/LibOS/shim/test/native$ gdb --args ./pal ./helloworld
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./pal...done.
(gdb) run
Starting program: /home/adrian/projects/graphene/LibOS/shim/test/native/pal ./helloworld
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
manifest file: file:helloworld.manifest.sgx
being GDB'ed!!!
enclave created:
    base:         0x0000000000000000
    size:         0x10000000
    attr:         0x0000000000000007
    xfrm:         0x0000000000000007
    ssaframesize: 1
    isvprodid:    0x00000000
    isvsvn:       0x00000000
User addr a8a12000, addr ffff000 (0, ffff000), skip_eexten 0
adding page  to enclave: 000000000ffff000 [REG:R--] (manifest) measured
User addr 0, addr fff6000 (0, fff6000), skip_eexten 1
adding pages to enclave: 000000000fff6000-000000000fffe000 [REG:RW-] (ssa)
User addr a8a0f000, addr fff1000 (0, fff1000), skip_eexten 1
adding pages to enclave: 000000000fff1000-000000000fff5000 [TCS:---] (tcs)
User addr a8a0f000, addr ffec000 (0, ffec000), skip_eexten 1
adding pages to enclave: 000000000ffec000-000000000fff0000 [REG:RW-] (tls)
User addr 0, addr ffdb000 (0, ffdb000), skip_eexten 1
adding pages to enclave: 000000000ffdb000-000000000ffeb000 [REG:RW-] (stack)
User addr 0, addr ffca000 (0, ffca000), skip_eexten 1
adding pages to enclave: 000000000ffca000-000000000ffda000 [REG:RW-] (stack)
User addr 0, addr ffb9000 (0, ffb9000), skip_eexten 1
adding pages to enclave: 000000000ffb9000-000000000ffc9000 [REG:RW-] (stack)
User addr 0, addr ffa8000 (0, ffa8000), skip_eexten 1
adding pages to enclave: 000000000ffa8000-000000000ffb8000 [REG:RW-] (stack)
User addr a89ca000, addr bd77000 (0, bd77000), skip_eexten 0
adding pages to enclave: 000000000bd77000-000000000bda4000 [REG:R-X] (code) measured
User addr a8a11000, addr bfa4000 (0, bfa4000), skip_eexten 0
adding pages to enclave: 000000000bfa4000-000000000bfa6000 [REG:RW-] (data) measured
User addr 0, addr bfa6000 (0, bfa6000), skip_eexten 1
adding pages to enclave: 000000000bfa6000-000000000ffa7000 [REG:RW-] (bss)
User addr a8a12000, addr 400000 (0, 400000), skip_eexten 0
adding page  to enclave: 0000000000400000 [REG:RWX] (code) measured
User addr a8a11000, addr 600000 (0, 600000), skip_eexten 0
adding pages to enclave: 0000000000600000-0000000000602000 [REG:RW-] (data) measured
User addr 0, addr 602000 (0, 602000), skip_eexten 1
adding pages to enclave: 0000000000602000-000000000bd76000 [REG:RWX] (free)
User addr 0, addr 10000 (0, 10000), skip_eexten 1
adding pages to enclave: 0000000000010000-0000000000400000 [REG:RWX] (free)
enclave initializing:
    enclave id:   0x000000000ffff000
    enclave hash: 77 97 75 e5 e8 e0 3f 84 1b 12 29 fb 5e 75 86 b4 5a ae d7 50 e4 18 52 d2 96 32 c7 f1 61 91 73 03
add symbol table from file "/home/adrian/projects/graphene/Pal/src/libpal-enclave.so" at
	.text_addr = 0xbd79500
	.rodata_addr = 0xbd9ee80
	.dynamic_addr = 0xbfa4000
	.data_addr = 0xbfa42e0
	.bss_addr = 0xbfa5100
enclave (software) key hash: d10d6cbe509c55ec182e28ba53bef17f848871e6a8af4450300753d11ecd2a5
trusted: [1] c7e5dfb61b78f4310ce42869c9f51464ec8b25b3c7954b5abb9037abcadc9742 file:./helloworld
trusted: [2] 8122fe16b669284c52fa9d76ea30bba226290700912d60fe6fd5d35e64cf1290 file:../../src/libsysdb.so
trusted: [3] f968b9fcbaaafdffd9284b032f438c4c8158137f5f3d4ff8d040223392fec653 file:../../../glibc-build/ld-linux-x86-64.so.2
trusted: [4] 694bce3dd24069c05fb181de5f67c9b4120b3506f0ca8c48577c0c382f519413 file:../../../glibc-build/libc.so.6
trusted: [5] 3ea7c71efc7dee902522363aad88636d906a049f2e52afc7b5d539547dc505aa file:../../../glibc-build/libdl.so.2
trusted: [6] 190cfd9fce1d8a84320455f9489cf253d659baa00c8f74da6c5e7f311691a81e file:../../../glibc-build/libm.so.6
trusted: [7] d0a6b136b6c071800941d59811c7c3cafab76d4b4534a2d9e8bc93dcfe74b79d file:../../../glibc-build/libpthread.so.0
trusted: [8] bd034c8e97a65c93b7872260f38189178b6c4dfed257d46b8a8899c7c10f946c file:exec_victim
trusted: [9] 72f89326444f56c4c39af100aa8f8974f08ed2c3a2e4d1292330a285bcb77927 file:unix.c
trusted: 4f036eb89f7b1eb0767b7d6b7e88102df4f1821ee33f924dfa964ed2c0be3329 file:exec_victim.sig
add symbol table from file "../../src/libsysdb.so" at
	.text_addr = 0xbab7660
	.hash_addr = 0xbaa7120
	.dynsym_addr = 0xbaa7290
	.dynstr_addr = 0xbaa7788
	.gnu.version_addr = 0xbaa7b6c
	.gnu.version_d_addr = 0xbaa7bd8
	.gnu.version_r_addr = 0xbaa7c10
	.rela.dyn_addr = 0xbaa7c30
	.rela.plt_addr = 0xbab6f18
	.plt_addr = 0xbab7370
	.rodata_addr = 0xbb47b80
	.eh_frame_addr = 0xbb511d0
	.dynamic_addr = 0xbd64000
	.got_addr = 0xbd641a0
	.got.plt_addr = 0xbd641a8
	.data_addr = 0xbd64340
	.bss_addr = 0xbd70000
add symbol table from file "helloworld" at
	.text_addr = 0x400430
	.interp_addr = 0x400238
	.note.ABI-tag_addr = 0x400254
	.note.gnu.build-id_addr = 0x400274
	.gnu.hash_addr = 0x400298
	.dynsym_addr = 0x4002b8
	.dynstr_addr = 0x400318
	.gnu.version_addr = 0x400358
	.gnu.version_r_addr = 0x400360
	.rela.dyn_addr = 0x400380
	.rela.plt_addr = 0x400398
	.init_addr = 0x4003c8
	.plt_addr = 0x4003f0
	.plt.got_addr = 0x400420
	.fini_addr = 0x4005d4
	.rodata_addr = 0x4005e0
	.eh_frame_hdr_addr = 0x4005f8
	.eh_frame_addr = 0x400630
	.init_array_addr = 0x600e10
	.fini_array_addr = 0x600e18
	.jcr_addr = 0x600e20
	.dynamic_addr = 0x600e28
	.got_addr = 0x600ff8
	.got.plt_addr = 0x601000
	.data_addr = 0x601028
	.bss_addr = 0x601038
[P1620] set tcb to 0xffea538
[P1620] initial random value: 1463882903f36108
[P1620] allocate internal heap at 0xb9a6000 - 0xbaa6000
[P1620] bkeep_mmap: 0xb9a6000-0xbaa6000
[P1620] shim loaded at 0xbaa7000, ready to initialize
[P1620] bkeep_mmap: 0xffff000-0x10000000
[P1620] mounting as proc filesystem: /proc
[P1620] mounting as dev filesystem: /dev
[P1620] mounting as chroot filesystem: from dev:tty to /dev
[P1620] adding port (handle 0xc7b8ad4) for process 0 (type 0001)
[P1620] creating pipe: pipe.srv:66281736
[P1620] adding port (handle 0xc7b8e18) for process 0 (type 0002)
[P1620] walk_thread_list(callback=0xbae7244)
[P1620] allocated pid: 1
[    1] mounting as chroot filesystem: from file:../../../glibc-build to /lib
[    1] mounting as chroot filesystem: from file:/bin to /bin
[    1] get unmapped: 0xb5a5000-0xb9a6000
[    1] bkeep_mmap: 0xb5a6000-0xb9a6000
[    1] bkeep_mprotect: 0xb5a6000-0xb9a6000
[    1] bkeep_mmap: 0xb5a5000-0xb5a6000
[    1] bkeep_mmap: 0xb5a5000-0xb5a6000
[    1] allocated stack at 0xb5a6000 (size = 4194304)
[    1] adding file:helloworld as runtime object loaded at 0x400000-0x602000
[    1] bkeep_mmap: 0x400000-0x401000
[    1] bkeep_mmap: 0x600000-0x602000
[    1] search interpreter: /lib/ld-linux-x86-64.so.2
[    1] bkeep_mmap: 0xb5a1000-0xb5a5000
[    1] get unmapped: 0xb37e000-0xb5a1000
[    1] bkeep_mmap: 0xb37e000-0xb5a1000
[    1] bkeep_mprotect: 0xb39f000-0xb59e000
[    1] bkeep_mmap: 0xb37e000-0xb39f000
[    1] bkeep_mmap: 0xb59e000-0xb5a1000
[    1] bkeep_mmap: 0xb59e000-0xb5a0000
[    1] bkeep_mprotect: 0xb59e000-0xb5a0000
[    1] bkeep_mmap: 0xb5a0000-0xb5a1000
[    1] bkeep_mmap: 0xb5a0000-0xb5a1000
[    1] add a library for gdb: file:../../../glibc-build/ld-linux-x86-64.so.2
add symbol table from file "../../../glibc-build/ld-linux-x86-64.so.2" at
	.text_addr = 0xb37ec40
	.note.gnu.build-id_addr = 0xb37e1c8
	.hash_addr = 0xb37e1f0
	.gnu.hash_addr = 0xb37e2c0
	.dynsym_addr = 0xb37e3b0
	.dynstr_addr = 0xb37e6c8
	.gnu.version_addr = 0xb37e8a4
	.gnu.version_d_addr = 0xb37e8e8
	.rela.dyn_addr = 0xb37e9a8
	.rela.plt_addr = 0xb37eb28
	.plt_addr = 0xb37ebc0
	.plt.got_addr = 0xb37ec30
	.rodata_addr = 0xb397180
	.eh_frame_hdr_addr = 0xb39b640
	.eh_frame_addr = 0xb39bca0
	.data.rel.ro_addr = 0xb59ec00
	.dynamic_addr = 0xb59ee60
	.got_addr = 0xb59efd0
	.got.plt_addr = 0xb59f000
	.data_addr = 0xb59f060
	.bss_addr = 0xb59ffe0
[New Thread 0x7f91a7f4b700 (LWP 5624)]
[    1] shim process initialized
[    1] get unmapped: 0x937e000-0xb37e000
[P1620] set tcb to 0xffd9a00
[P1620] ipc helper thread started
[P1620] allocated stack at 0xb9d0000 (size = 16384)
[P1620] listen to process 0 on port 0xb9c60c8 (handle 0xc7b8e18, type 0002)
[P1620] listen to process 0 on port 0xb9c6048 (handle 0xc7b8ad4, type 0001)
[    1] brk area: 0x937e000 - 0x937f000
[    1] brk reserved area: 0x937f000 - 0xb37e000
[    1] bkeep_mmap: 0x937e000-0x937f000
[    1] bkeep_mprotect: 0x937e000-0x937f000
[    1] bkeep_mmap: 0x937f000-0xb37e000
[    1] bkeep_mmap: 0x937f000-0xb37e000
[    1] ---- shim_brk(0x0) = 0x937e000
[    1] ---- shim_uname(0xb9a51f4) = 0
[    1] ---- shim_mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,4294967295,0) ...
[    1] get unmapped: 0x937d000-0x937e000
[    1] bkeep_mmap: 0x937d000-0x937e000
[    1] ---- return from shim_mmap(...) = 0x937d000
[    1] ---- shim_open("/lib/tls/libc.so.6",O_RDONLY|2000000,1326400530) ...
[    1] ---- return from shim_open(...) = -2
[    1] ---- shim_stat("/lib/tls",0xb9a4c4a) = -2
[    1] ---- shim_open("/lib/libc.so.6",O_RDONLY|2000000,1326400530) ...
[    1] ---- return from shim_open(...) = 3
[    1] ---- shim_read(3,0xb9a4d9a,832) ...
[    1] bkeep_mmap: 0x9379000-0x937d000
[    1] ---- return from shim_read(...) = 832
[    1] ---- shim_fstat(3,0xb9a4c4a) = 0
[    1] ---- shim_mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,4294967295,0) ...
[    1] get unmapped: 0x9378000-0x9379000
[    1] bkeep_mmap: 0x9378000-0x9379000
[    1] ---- return from shim_mmap(...) = 0x9378000
[    1] ---- shim_mmap(0x0,3852760,PROT_READ|PROT_EXEC,MAP_PRIVATE|0x800,3,0) ...
[    1] get unmapped: 0x8fcb000-0x9378000
[    1] bkeep_mmap: 0x8fcb000-0x9378000
[    1] ---- return from shim_mmap(...) = 0x8fcb000
[    1] ---- shim_mprotect(0x916e000,2097152,PROT_NONE) ...
[    1] bkeep_mprotect: 0x916e000-0x936e000
[    1] bkeep_mmap: 0x8fcb000-0x916e000
[    1] bkeep_mmap: 0x936e000-0x9378000
[    1] ---- return from shim_mprotect(...) = 0
[    1] ---- shim_mmap(0x936e000,24576,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|0x800,3,1716224) ...
[    1] bkeep_mmap: 0x936e000-0x9374000
[    1] bkeep_mprotect: 0x936e000-0x9374000
[    1] bkeep_mmap: 0x9374000-0x9378000
[    1] ---- return from shim_mmap(...) = 0x936e000
[    1] ---- shim_mmap(0x9374000,14808,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_FIXED,4294967295,0) ...
[    1] bkeep_mmap: 0x9374000-0x9378000
[    1] ---- return from shim_mmap(...) = 0x9374000
[    1] bkeep_unmmap: 0x9379000-0x937d000
[    1] ---- shim_close(3) = 0
[    1] glibc register library /lib/libc.so.6 loaded at 0x8fcb000
[    1] add a library for gdb: file:../../../glibc-build/libc.so.6
add symbol table from file "../../../glibc-build/libc.so.6" at
	.text_addr = 0x8fea3d0
	.note.gnu.build-id_addr = 0x8fcb230
	.note.ABI-tag_addr = 0x8fcb254
	.gnu.hash_addr = 0x8fcb278
	.dynsym_addr = 0x8fcece8
	.dynstr_addr = 0x8fdbd68
	.gnu.version_addr = 0x8fe1648
	.gnu.version_d_addr = 0x8fe27a8
	.gnu.version_r_addr = 0x8fe2af0
	.rela.dyn_addr = 0x8fe2b30
	.rela.plt_addr = 0x8fea240
	.plt_addr = 0x8fea320
	.plt.got_addr = 0x8fea3c0
	__libc_freeres_fn_addr = 0x9116ac0
	__libc_thread_freeres_fn_addr = 0x91177a0
	.rodata_addr = 0x91179a0
	.interp_addr = 0x913a760
	.eh_frame_hdr_addr = 0x913a7a0
	.eh_frame_addr = 0x9141060
	.gcc_except_table_addr = 0x916a334
	.hash_addr = 0x916a730
	.tdata_addr = 0x936e760
	.tbss_addr = 0x936e770
	.init_array_addr = 0x936e770
	__libc_subfreeres_addr = 0x936e778
	__libc_atexit_addr = 0x936e838
	__libc_thread_subfreeres_addr = 0x936e840
	.data.rel.ro_addr = 0x936e860
	.dynamic_addr = 0x9371b60
	.got_addr = 0x9371d40
	.got.plt_addr = 0x9371fc0
	.data_addr = 0x9372020
	.bss_addr = 0x9373700
[    1] ---- shim_mmap(0x0,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,4294967295,0) ...
[    1] get unmapped: 0x937b000-0x937d000
[    1] bkeep_mmap: 0x937b000-0x937d000
[    1] ---- return from shim_mmap(...) = 0x937b000
[    1] set tcb to 0x937b700
[    1] ---- shim_arch_prctl(4098,0x937b700) = 0x0

Thread 1 "pal" received signal SIGILL, Illegal instruction.
0x000055fefeaf5994 in async_exit_pointer ()

LTP testcase chdir02 fails to chdir to "/" or "/tmp"

Failed LTP test : chdir02
I see that, path_lookupat() in shim_namei.c is not setting the DENTRY_ISDIRECTORY bit in dent->state, returning -ENOTDIR at
https://github.com/oscarlab/graphene/blob/master/LibOS/shim/src/sys/shim_getcwd.c#L82

Can not make OPENJDK in SGX mode

Hi,

The OPENJDK app can not be compiled using the command "make SGX=1", and the following is the error msg. Do you know what I should do? Thank you in advance.

make JDK_TOPDIR=/home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/jdk JDK_MAKE_SHARED_DIR=/home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/jdk/make/common/shared EXTERNALSANITYCONTROL=true SOURCE_LANGUAGE_VERSION=7 TARGET_CLASS_VERSION=7 MILESTONE=internal BUILD_NUMBER=b00 JDK_BUILD_NUMBER=b00 FULL_VERSION=1.7.0-internal-user_2017_02_01_14_24-b00 PREVIOUS_JDK_VERSION=1.6.0 JDK_VERSION=1.7.0 JDK_MKTG_VERSION=7 JDK_MAJOR_VERSION=1 JDK_MINOR_VERSION=7 JDK_MICRO_VERSION=0 PREVIOUS_MAJOR_VERSION=1 PREVIOUS_MINOR_VERSION=6 PREVIOUS_MICRO_VERSION=0 STATIC_CXX=false PAX_COMMAND=/usr/sbin/paxmark.sh PAX_COMMAND_ARGS="-vm" ARCH_DATA_MODEL=64 COOKED_BUILD_NUMBER=0 ALT_OUTPUTDIR=/home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir ALT_EXPORT_PATH=/home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/import BUILD_FLAVOR=product ALT_SLASH_JAVA=/NOT-SET ALT_BOOTDIR=/usr/lib/jvm/java-7-openjdk-amd64 HOTSPOT_BUILD_JOBS=8 all_product INFO: ENABLE_FULL_DEBUG_SYMBOLS=1 make[3]: Entering directory/home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make'
cd /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make;
make BUILD_FLAVOR=product VM_TARGET=product generic_build2 ALT_OUTPUTDIR=/home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir
INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
make[4]: Entering directory /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make' mkdir -p /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir cd /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/build/linux-amd64/hotspot/outputdir; \ MAKEFLAGS= MFLAGS= EXTRA_userLAGS="-fPIC" nmake -NOLOGO -f \\home\\user\\graphene\\LibOS\\shim\\test\\apps\\openjdk\\openjdk-7\\hotspot\\make\\windows\\build.make \ Variant=compiler2 \ WorkSpace=\\home\\user\\graphene\\LibOS\\shim\\test\\apps\\openjdk\\openjdk-7\\hotspot \ BootStrapDir=\\usr\\lib\\jvm\\java-7-openjdk-amd64 \ BuildUser= \ ENABLE_FULL_DEBUG_SYMBOLS=1 ZIP_DEBUGINFO_FILES=1 RM="rm -f" ZIPEXE=zip JDK_MKTG_VERSION=7 JDK_MAJOR_VER=1 JDK_MINOR_VER=7 JDK_MICRO_VER=0 JDK_BUILD_NUMBER=0 BUILD_WIN_SA=1 JAVA_HOME=\\usr\\lib\\jvm\\java-7-openjdk-amd64 OUTPUTDIR=\\home\\user\\graphene\\LibOS\\shim\\test\\apps\\openjdk\\openjdk-7\\build\\linux-amd64\\hotspot\\outputdir GAMMADIR=\\home\\user\\graphene\\LibOS\\shim\\test\\apps\\openjdk\\openjdk-7\\hotspot MAKE_VERBOSE=y HOTSPOT_RELEASE_VERSION=24.95-b00 JRE_RELEASE_VERSION=1.7.0-internal-user_2017_02_01_14_24-b00 HOTSPOT_BUILD_VERSION= INCLUDE_TRACE=1 product /bin/sh: 2: nmake: not found make[4]: *** [generic_build2] Error 127 make[4]: Leaving directory /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make'
make[3]: *** [product] Error 2
make[3]: Leaving directory /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7/hotspot/make' make[2]: *** [hotspot-build] Error 2 make[2]: Leaving directory /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7'
make[1]: *** [build_product_image] Error 2
make[1]: Leaving directory /home/user/graphene/LibOS/shim/test/apps/openjdk/openjdk-7' make: *** [openjdk-7/build/linux-amd64/j2sdk-image/jre/bin/java] Error 2

Test apps fail with segfault

Today I pulled the commits since f2c36a1 and rebuilt everything. Now most of the test apps fail.

Example:

cd LibOS/shim/test/apps/python
make SGX=1 && make SGX_RUN=1
./python.manifest.sgx scripts/helloworld.py

Then it just hangs without output, on ^C it prints Segmentation fault. The same happens with lighttpd. But native/helloworld.manifest.sgx works.

Is it possible to keep the unmodified Applications outside SGX while the libraries inside SGX?

While it is amazing to keep both the unmodified application binary and supporting libraries inside SGX, sometimes when the applications are not trusted we may only want to isolate (some of) the libraries inside an SGX enclave. Is it possible, for example based on a minimum modification of graphene-SGX? I am having this question, as sometimes we don't need to protect the entire address space. Thanks.

memcached multi-threading issue

Hey,

First, thanks for all the previous help. I'm trying to run the memcached benchmark yet the applications hangs unexpectedly in my experiments.

The configuration is:
Memcached: running the newly added example in the app dir with the cmdline (disabling UDP and working only with TCP for simplicity)
./memcached.manifest.sgx -u root -t 4 -m 500 -U 0 -p 11211

Memslap, with the cmdline:
./memslap --debug --concurrency=24 --servers=localhost --initial-load=100 --execute-number=10000
(memslap was compiled with VALUE_BYTES 4000 (in memslap.cc)

I'm getting the following outputs:

The application either hangs (showing to be stuck either after printing epoll_wait/futex_wait/futex_wake in the inline debugger).
assert failed bookkeep/shim_vma.c:128 !prev || prev->addr + prev->length <= tmp->addr (value:0)
Application pass (very rarely).

This is a small test that should show the bug which is more frequent on my larger test. The bigger experiment I'm attempting to run is with a working set of ~400MB. I.e., having --initial-load=80000 and executing either get or set commands against it with arbitrary number of threads in the client and the server.

Lastly, I've also tried running different configuration with memslap, memaslap and YCSB. All attempts behaves similarly.

LTP testcase alarm03, alarm05, alarm06

Fixed in branch abora-local
File changed : shim_async.c

System call alarm is supposed to return number of seconds remaining in any previously set alarm.
It was returning 0 all the time. Code to calculate correct return value merged in branch abora-local
System call alarm is supposed to cancel any pending alarm. Code to cancel any pending alarm merged in branch abora-local

memcached multi-threading

Hey,
I got the following problem trying to run memcached in graphene-sgx.
I'm running Ubuntu 14.04, with SGX-enabled HW (I'm able to run native enclaves)

memcached.manifest.tar.gz

I tried running memcached with the attached manifest file after signing it.

Memcached runs correctly with the following command line:
./memcached.manifest.sgx -u user -l 132.68.52.219 -p 9000 -c 1024 -t 2 -U 0 -m 256 -C

Increasing memcached to support more threads, e.g. the following command line results in the application getting stuck.
./memcached.manifest.sgx -u user -l 132.68.52.219 -p 9000 -c 1024 -t 2 -U 0 -m 256 -C

Using the inline debug_type of the loader suggests that it stuck after poll, though I'm not sure I read it correctly.

Initializing enclave fails for all tests in LibOS/shim/test/native

Example:

graphene/LibOS/shim/test/native$ ./pal helloworld.manifest.sgx
initializing enclave failed: init_enclave: 1

The reason seems to be the missing loader.exec entry in helloworld.manifest.sgx.

I think the right fix would be a line loader.exec = $(EXEC) in LibOS/shim/test/native/manifest.template and an additional rule in LibOS/shim/test/Makefile to replace the $(EXEC) with the correct executable name. But I was not able to fix this myself, because I can't wrap my head around this makefile.

bash, lmbench fail with "bug() bookkeep/shim_vma.c:205"

Running bash with SGX produces the following output and then the process becomes unresponsive (has to be killed with SIGKILL):

$ ./bash.manifest.sgx bash_test.sh 
+ exec /home/user/projects/graphene/Pal/src/pal ./bash.manifest.sgx bash_test.sh
WARNING: no physical memory support, process creation will be slow.
bug() bookkeep/shim_vma.c:205

The same happens with lmbench in step lat_proc exec:

~/projects/graphene/LibOS/shim/test/apps/lmbench$ make test-graphene
[skipped]
lat_proc exec
WARNING: no physical memory support, process creation will be slow.
bug() bookkeep/shim_vma.c:205

parse_ioctlop prints decimal with 0x prefix

https://github.com/oscarlab/graphene/blob/1a1e199/LibOS/shim/src/shim_parser.c#L977

glibc fails to build if ld.gold is the default linker

As the bug described here:
https://sourceware.org/bugzilla/show_bug.cgi?id=14995

When configuring glibc-2.17:

configure: error:
*** These critical programs are missing or too old: ld
*** Check the INSTALL file for required versions.
uh-oh: <type 'exceptions.OSError'>
chiache@gonzo$ ls -l /usr/bin/ld                                                                                                                  
lrwxrwxrwx 1 root root 7 Jun 24  2013 /usr/bin/ld -> ld.gold

file operation not work at python with SGX

Hello there,
I'm new for Graphene, I just build graphene on SGX.
Firstly I go to Libos/shim/test/apps/python and add those lines to python.manifest.template

 fs.mount.storage.type = chroot
 fs.mount.storage.path = /storage
 fs.mount.storage.uri = file:/home/shenyouren/storage

Then I run

make SGX=1
make SGX_RUN=1

After that I modify script/helloworld.py and want to chdir to /storage.
But the python code

os.chdir("/storage")

returns

OSError: [Errno 20] Not a directory

I tried to change to other dir like /bin /host but failed too except /usr

I don't know why and can't figure out what's the different between /usr and other chroot entries.

Would you mind to help me?

Really appreciate for reply!
Thank you very much.

attestation

I was wondering if graphene has any attestation mechanism to make sure the enclave is not compromised?

Thank you for your work.

LTP testcases exit01, exitgroup01

Fixed in branch abora-local
File changed : shim_exit.c

exit is supposed to set process's exit code to the provided value. It was not using the provided value earlier. Code to do so merged in abora-local.

Vulnerabilities in SGX trusted enclave_entry procedure

I identified several security-critical vulnerabilities in Graphene's trusted runtime for SGX enclaves. It should be noted that the containing host application, including the unprotected Graphene runtime, is completely untrusted in the SGX attacker model.

Vulnerabilities

As a general rule, the trusted intra-enclave runtime should properly check all arguments/return values passed from the untrusted runtime. This is currently not the case in at least the following places:

enclave_entry.S: does not perform bounds checking on the provided index for the entry function lookup table. The untrusted runtime can overflow/underflow the ecall_table lookup by providing an out-of-bounds entry index in %rdi:

.Lhandle_ecall:
	lea ecall_table(%rip), %rbx
	mov (%rbx,%rdi,8), %rbx
        ...
	call *%rbx

An attacker controlling the unprotected application can redirect control flow to non-entry functions by abusing intra-enclave function pointers. She could even jump to arbitrary in-enclave code by over/underflowing the ecall_table lookup to unprotected memory (or enclave memory that holds user input).

enclave_entry.S: allows to "return" into an enclave thread that is not waiting for a previous ocall return. Untrusted runtime can simply provide the special return entry index in %rdi:

	cmp $RETURN_FROM_OCALL, %rdi
	je .Lreturn_from_ocall
        ...
.Lreturn_from_ocall
	mov %gs:SGX_LAST_STACK, %rsp
        popfq
        ...
        ret

An attacker can abuse this to initialize the in-enclave stack pointer of a newly started thread with the value of the last ocall. The memory content at these locations determine register values, and ultimately control flow.

enclave_ecalls.c contains an entry function enclave_ecall_thread_start that redirects control flow to a provided function address. The function pointer is not restricted in any way, allowing an attacker to jump to arbitrary code locations within the enclave. The attacker also has control over the first argument provided to the function.

Proof-of-Concept Exploit

I included a proof-of-concept exploit at https://github.com/jovanbulck/graphene.

To demonstrate the danger of allowing arbitrary non-entry code execution, I wrote an elementary application that decides access to an access_allowed function by comparing a user-provided number with a secret PIN code. After confirming the application has been loaded, the untrusted runtime abuses the ecall_table lookup in enclave_entry.S to call access_allowed, regardless of the provided PIN. This shows that an attacker can redirect control flow to arbitrary enclaved code, including the loaded application binary:

Hello world from enclaved application binary!
	--> ac.allowAccess at 0x601078 is 0x4007e1 (access_allowed_handler)
app: enter PIN..
> 1233
user entered 1233
app: checking acess..
===> app: access denied! <===
app: ocall: opening dummy_file

[urts] entering enclave with illegal ecall idx -24332361 (ecall_table_adrs=0xbfa52c0)
enclave_handle_ecall: asm stub passed function pointer:
[ocall_dump] 0x4007e1
===> app: access allowed! <===
destroying enclave...

I also included a proof-of-concept exploit that redirects control flow to an arbitrary non-entry function of the trusted runtime by abusing the argument of enclave_ecall_thread_start. It can easily be understood that this allows for code abuse attacks that could break the confidentiality/integrity of in-enclave data.

Security Patches

It should be clear from the above explanation and the exploit that Graphene's trusted runtime should restrict enclave entry to a few well-defined entry points. I did not write a patch since it is not yet entirely clear what the best solution would be:

enclave_entry.S: given the security-sensitive role of the entry assembly code, I suggest reducing it to the absolute minimum. After switching to the secure stack, a normal C function could be called to take care of the ecall/return from ocall. At this point, entry index bounds checking and illegal returns could be properly handled. This approach resembles the official SGX SDK, where an enter_enclave C function is called from the enclave_entry asm stub.
enclave_ecalls.c: the current implementation of thread_start is fundamentally flawed. Entry code execution should be limited to predefined entry points. It is unclear to me how this fits in Graphene's untrusted runtime/glibc threading model (?)

Regards,

<jo.vanbulck at cs.kuleuven dot be>

Segfault with current Graphene tests

Hi,

The native tests (all of them) in the LibOS directory fail with a segfault:

[_DkResumeSighandler @sgx_exception.c:238] Segmentation Fault in Untrusted Code (RIP = 7fffed794980)

I'm guessing there is some issue with compatibility with SGX SDK and SGX Drivers. Can you please tell me which commit (of SDK and drivers) I should use?

I can run the tests in the native SDK without any trouble but Graphene tests now fail.

Adil

LTP testcase getpgid01

parent and child processes are using different instances of 'thread_list' in 'shim_thread.c'
'thread_list' of parent process contains both threads of parent process and child process,
while 'thread_list' of child process contains only thread of child process.
Thus getting 'parent process group ID' is returning 'ESRCH No such process' error in child process
address of thread_list is different in child process.

Linux-SGX file_read() TOCTTOU vulnerability

I think the file_read() implementation in Linux-SGX/db_files.c#L86 has a time-of-check-to-time-of-use vulnerability:

In L108 the target file is mapped to untrusted memory.
In verify_trusted_file() in L114, the memory is verified using a CMAC.
If the verification was successful, the memory is copied in L124.

Because the file is mapped to untrusted memory, the untrusted OS can modify the content of the memory after the verification and before the memory is copied.

Maybe I got this wrong, because I don't really understand your file verification process in the first place. First you try to verify the file with the SHA256 checksum in load_trusted_file() called by file_open(), generate a CMAC in load_trusted_file(), and then try to verify the CMAC in file_read(). Why the CMAC?
Also, IIUC, load_trusted_file() operates on untrusted files. Why perform a check in load_trusted_file() in the first place, if the untrusted OS can modify the file afterwards?

fsync on directory

It is not implemented in graphene and SGX file ops.
Are there any reasons on this ?

Can't map large files

Case 1

#include <stdio.h>
#include <sys/mman.h>

int main() {
    FILE*fp=fopen("testfil","a+");
    if (!fp) { perror("fopen"); return 1; }
    void* a=mmap(NULL, 0x10000f000, PROT_READ|PROT_WRITE, MAP_SHARED, fileno(fp), 0);
    if (!a) { perror("mmap"); return 1; }
    ((char*)a)[0x100000000]=0xff;
    return 0;
}

Expected output of truncate -s 0 testfil && truncate -s $((0x10000f000)) testfil && hexdump -C -s $((0x100000000)) testfil && ./a.out && hexdump -C -s $((0x100000000)) testfil:

100000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
10000f000
100000000  ff 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
100000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
10000f000

Getting:

[    1] ---- shim_mmap(0x0,61440,PROT_READ|PROT_WRITE,MAP_SHARED,3,0) ...
[    1] get unmapped: 0x3dd2d3e93000-0x3dd2d3ea2000
assert failed sys/shim_mmap.c:63 cur_stack < addr || cur_stack > addr + length (value:0)

Case 2

#include <stdio.h>
#include <sys/mman.h>

int main() {
    FILE*fp=fopen("testfil","a+");
    if (!fp) { perror("fopen"); return 1; }
    void* a=mmap(NULL, 0x8000f000, PROT_READ|PROT_WRITE, MAP_SHARED, fileno(fp), 0);
    if (!a) { perror("mmap"); return 1; }
    ((char*)a)[0x80000000]=0xff;
    return 0;
}

Expected output of truncate -s 0 testfil && truncate -s $((0x8000f000)) testfil && hexdump -C -s $((0x80000000)) testfil && ./a.out && hexdump -C -s $((0x80000000)) testfil:

80000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
8000f000
80000000  ff 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
80000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
8000f000

Getting:

[    1] ---- shim_mmap(0x0,-2147422208,PROT_READ|PROT_WRITE,MAP_SHARED,3,0) ..
[    1] .....
[    1] get unmapped: 0x81ceef56000-0x81c6ef65000
[    1] bkeep_unmmap: 0x81ceef56000-0x81c6ef65000
assert failed bookkeep/shim_vma.c:127 tmp->length > 0 (value:0)

gramineproject / graphene Goto Github PK

graphene's Introduction

PROJECT MOVED TO A NEW REPOSITORY

Graphene Library OS with Intel SGX Support

What is Graphene?

Release candidate version of Graphene 1.2 available

Graphene documentation

Getting help

Acknowledgments

graphene's People

Contributors

Stargazers

Watchers

Forkers

graphene's Issues

Vulnerabilities

Proof-of-Concept Exploit

Security Patches

Case 1

Case 2

Recommend Projects

Recommend Topics

Recommend Org