GSC conversion for REST API about gsc HOT 10 CLOSED

Please update your GSC to the latest version. We recently made a change to core Gramine and to GSC.

from gsc.

I tried app.run(debug=False,host='0.0.0.0', port=5000) without debugger.
It can work.

Gramine is starting. Parsing TOML manifest file, this may take some time...
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:

  - sgx.debug = true                           (this is a debug enclave)
  - loader.insecure__use_cmdline_argv = true   (forwarding command-line args from untrusted host to the app)
  - sgx.allowed_files = [ ... ]                (some files are passed through from untrusted host without verification)

Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------

 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [29/Jul/2022 08:05:12] "GET / HTTP/1.1" 200 -

$ curl localhost:5000
{"Result":"GET result"}

from gsc.

Since it seems to work, I will close this issue.

from gsc.

• Can I use some pre-configured docker images (like python:3.7-slim-stretch) or it has to be ubuntu:18.04 and then install python3 in the docker image?

Yes you can. You can use any Ubuntu-based Docker image as your base image in the ubuntu18.04-flask.dockerfile file. The only requirement is that Gramine's base image (specified in config.yaml of GSC) is compatible with your Dockerfile's base image.

• Is it possible to run a docker container (with Flask server) and also provide arguments in the docker run command ? something like this:
  docker run -p 5000:5000 -it --device=/dev/sgx_enclave -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket gsc-<some-converted-image> '$ARG'

Yes. For this, you need to use the --insecure-args option during the gsc build step. Which you already do. So it should just work.

• Do you have some working examples with Python Flask server (or any RESTFUL API) I can refer to?

Unfortunately no. But I don't see any specific problems with REST API or with Flask -- I mean, I don't think your problem is about network connectivity. Your problem is most probably with some manifest option that is missing from your minimal test/generic.manifest.

So the next step for you:

1. Please show us your config.yaml GSC file.
2. Please tell us which GSC commit you're using (cd gsc && git show --summary).
3. Please try to re-build your application with gsc build --debug, to get debug information. This will force GSC and Gramine to print a lot of info. You can analyze this log yourself and find the potential issues, or you can attach the log here, so we can take a look.

from gsc.

Thanks @dimakuv , please see my responses in the step below

my test/generic.manifest

# Some workloads like Python may generate huge manifest files
loader.pal_internal_mem_size = "128M"

sgx.enclave_size = "4G"
sgx.thread_num = 8

sgx.trusted_files = [
  "file:/gramine/app_files/entrypoint.manifest",  # unused entry, only to test merging of manifests
]

So the next step for you:

1. Please show us your config.yaml GSC file.

# Specify the OS distro. Currently tested distros are
# ``ubuntu:18.04``, ``ubuntu:20.04``, ``ubuntu:21.04`` and ``centos:8``.
Distro: "ubuntu:18.04"

# If the image has a specific registry, define it here.
# Empty by default; example value: "registry.access.redhat.com/ubi8".
Registry: ""

# If you're using your own fork and branch of Gramine, specify the GitHub link and the branch name
# below; typically, you want to keep the default values though
Gramine:
    Repository: "https://github.com/gramineproject/gramine.git"
    Branch:     "master"

# Specify the Intel SGX driver installed on your machine (more specifically, on the machine where
# the graminized Docker container will run); there are several variants of the SGX driver:
#
#   - legacy out-of-tree driver: use something like the below values, but adjust the branch name
#         Repository: "https://github.com/01org/linux-sgx-driver.git"
#         Branch:     "sgx_driver_1.9"
#
#   - DCAP out-of-tree driver: use something like the below values
#         Repository: "https://github.com/intel/SGXDataCenterAttestationPrimitives.git"
#         Branch:     "DCAP_1.11 && cp -r driver/linux/* ."
#
#   - DCAP in-kernel driver: use empty values like below
#         Repository: ""
#         Branch:     ""
#
SGXDriver:
    Repository: ""
    Branch:     ""

2. Please tell us which GSC commit you're using (cd gsc && git show --summary).
   commit 72b296d5272863aea5200d4cda5b9e240d08151c (HEAD -> master, origin/master, origin/HEAD)

3. Please try to re-build your application with gsc build --debug, to get debug information. This will force GSC and Gramine to print a lot of info. You can analyze this log yourself and find the potential issues, or you can attach the log here, so we can take a look.

I made with the debug and received some warnings like this:

{"log":"[P1:T1:python3] trace: ---- openat(AT_FDCWD, \"/usr/lib/python3.6/__pycache__/warnings.cpython-36.pyc.1638345408\", O_WRONLY|O_CREAT|O_EXCL|0x80000, 0644) = -13\n","stream":"stderr","time":"2022-07-13T09:39:14.239605538Z"}
{"log":"warning: Disallowing access to file '/usr/lib/python3.6/__pycache__/shutil.cpython-36.pyc.1635719976'; file is not trusted or allowed.\n","stream":"stderr","time":"2022-07-13T09:39:14.246526475Z"}
{"log":"warning: Disallowing access to file '/usr/lib/python3.6/__pycache__/bz2.cpython-36.pyc.1638721720'; file is not trusted or allowed.\n","stream":"stderr","time":"2022-07-13T09:39:14.250241313Z"}
{"log":"warning: Disallowing access to file '/usr/lib/python3.6/__pycache__/_compression.cpython-36.pyc.1639286240'; file is not trusted or allowed.\n","stream":"stderr","time":"2022-07-13T09:39:14.253179997Z"}

I suspect that there must be more allow files/folders need to be added in the manifest file ?

Additionally, would it be possible to add the gsc to main path so that I can run it anywhere instead of inside gsc folder? Perhaps a soft link would work?

Thank you for your suggestions

Here's the log file
7f5a0178ef612c072784e479e51839cbe153bb49c8ebfbe150109f87469f7838-json.log

from gsc.

Additionally, would it be possible to add the gsc to main path so that I can run it anywhere instead of inside gsc folder? Perhaps a soft link would work?

You can just copy it yourself, if you want to. Something like ln -sf gsc /usr/local/bin should work. We didn't implement the installation procedure, because nobody seemed to care (until your request now :)).

Here's the log file

The log file has this as the last syscall executed:
pselect6(0, 0, 0, 0, 0x4aa11bc0, 0) ...

This is the way to sleep for some time (we can't see for how much time because we only see the pointer to the "time struct", which is at address 0x4aa11bc0). Are you sure your program is not just sleeping for like several minutes?

Other than that, I didn't find anything special in the log...

from gsc.

This is the way to sleep for some time (we can't see for how much time because we only see the pointer to the "time struct", which is at address 0x4aa11bc0). Are you sure your program is not just sleeping for like several minutes?

That's kind of strange, as my python program given above didn't make any sleep time, or maybe the Flask library does that without my understanding.

Anyways, should I include all necessary python libraries in the trusted files of the manifest, just to make sure it can read/write the libraries ?

Thank you

from gsc.

Anyways, should I include all necessary python libraries in the trusted files of the manifest, just to make sure it can read/write the libraries ?

This actually shouldn't be necessary, as GSC does it by itself.

If you think that reading/writing some files (libraries) can be a root cause, then you can try sgx.file_check_policy = "allow_all_but_log" manifest option.

from gsc.

I've tried with the given manifest option, but it was the same result, the app hanging but not receiving GET request from the curl command.

Also recently, when I tried to build new gsc images, there's an error appears:

Program protoc-c found: YES (/usr/bin/protoc-c)
Program protoc found: NO
meson.build:262:4: ERROR: Program 'protoc' not found or not executable
A full log can be found at /gramine/build/meson-logs/meson-log.txt
NOTICE: You are using Python 3.6 which is EOL. Starting with v0.62.0, Meson will require Python 3.7 or newer
Failed to build unsigned graminized docker image gsc-python3.7-flask-demo-unsigned.

Do you have any suggestion for this ? (I'm running local python3 version at 3.6.9)
Thank you
Update: I've installed 3.7 and update alternative python3 to 3.7 and re-installed all needed pip3 libs, but the issue is still there.

from gsc.

I tried app.run(debug=False,host='0.0.0.0', port=5000) without debugger. It can work.

Gramine is starting. Parsing TOML manifest file, this may take some time...
-----------------------------------------------------------------------------------------------------------------------
Gramine detected the following insecure configurations:

  - sgx.debug = true                           (this is a debug enclave)
  - loader.insecure__use_cmdline_argv = true   (forwarding command-line args from untrusted host to the app)
  - sgx.allowed_files = [ ... ]                (some files are passed through from untrusted host without verification)

Gramine will continue application execution, but this configuration must not be used in production!
-----------------------------------------------------------------------------------------------------------------------

 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [29/Jul/2022 08:05:12] "GET / HTTP/1.1" 200 -

$ curl localhost:5000
{"Result":"GET result"}

Hi @llly ,
Yes it seems removing the debugging option made the Flask server code running (at least with the GET).
Thank you for the suggestion!

from gsc.