zephyr: unable to compile with `WAMR_BUILD_LIBC_WASI` flag. about wasm-micro-runtime HOT 7 OPEN

Status

1. Abstracting libc primitives: After looking at how the POSIX primitives where abstracted in core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/ to support Windows. I think that we could do the same to support Zephyr, with the final idea to have an implementation entirely relying on the os_ APIs.

   • functions:
     • poll --> os_poll
     • ioctl --> os_ioctl
   • structs/types:
     • timespec --> os_timespec
     • pollfd --> os_poll_file_handle
     • nfds_t --> os_nfds_t

2. Extending platform_api_extension.h: The following API were added.

   __wasi_errno_t
   os_ioctl(os_file_handle *handle, int request, void *argp);
   
   // Not sure for return type
   __wasi_errno_t
   os_poll(os_poll_file_handle *pfds, os_nfds_t nfs, int timeout);

3. Changing the sandboxed system primitives to use the new types and functions.

   • random.c:

     #elif defined(BH_PLATFORM_ZEPHYR)
     static void
     open_urandom(void)
     {
         // Not implemented
     }
     __wasi_errno_t
     random_buf(void *buf, size_t len)
     {
         return __WASI_ENOSYS;
     }

   • blocking.h:

     __wasi_errno_t
     - os_poll(pollfd *pfds, os_nfds_t nfds, int timeout)
     + os_poll(os_poll_file_handle *pfds, os_nfds_t nfds, int timeout)

   • locking.h: change every struct timespec to struct os_timespec
   • posix.c: change every struct pollfd to struct os_poll_file_handle

4. Defining new types in platform_internal.h: Based on the os_file_handle type implementation we define the following types (and flags).

   typedef struct zsock_pollfd os_poll_file_handle;
   typedef unsigned int os_nfds_t;
   
   // To use posix flags
   #define POLLIN ZSOCK_POLLIN
   #define POLLPRI ZSOCK_POLLPRI
   #define POLLOUT ZSOCK_POLLOUT
   #define POLLERR ZSOCK_POLLERR
   #define POLLHUP ZSOCK_POLLHUP
   #define POLLNVAL ZSOCK_POLLNVAL
   
   #define FIONREAD ZFD_IOCTL_FIONREAD
   
   typedef struct {
           time_t tv_sec;
           long tv_nsec;
   } os_timespec;
   
   #define CLOCK_REALTIME 1
   #define CLOCK_MONOTONIC 4
   
   typedef struct {
       struct k_mutex mtx;  // Mutex for exclusive access
       struct k_sem sem;    // Semaphore for shared access
       int read_count;      // Number of readers
   } korp_rwlock;

5. Implementing the new functions in platform_socket.c:

   __wasi_errno_t
   os_ioctl(os_file_handle *handle, int request, void *argp)
   {
       __wasi_errno_t wasi_errno = __WASI_ESUCCESS;
   
       if(zsock_ioctl_wrapper(handle, request, argp) < 0){
           wasi_errno = zephyr_to_wasi_errno(errno);
       }
       return wasi_errno;
   }
   
   __wasi_errno_t
   os_poll(os_poll_file_handle *fds, os_nfds_t nfs, int timeout)
   {
       __wasi_errno_t wasi_errno = __WASI_ESUCCESS;
       int rc = 0;
   
       rc = zsock_poll(fds, nfs, timeout) 
       if(rc < 0){
           wasi_errno = zephyr_to_wasi_errno(errno);
       }
       switch(rc){
           case 0:
               wasi_errno = __WASI_ETIMEOUT;
               break;
           case -1:
               wasi_errno = zephyr_to_wasi_errno(errno);
               break;
           default:
               break;
       }
       return wasi_errno;
   }

6. compilation errors The compilation still fail with the same warnings as before. The unknown flags errors have disappeared. But the warning about the struct timespec and struct pollfd still persist.

   • Error on locking.h:

     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:200:12: error: variable 'ts' has initializer but incomplete type
     200 |     struct os_timespec ts = {
         |            ^~~~~~~~~~~
     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:201:10: error: 'struct os_timespec' has no member named 'tv_sec'
     201 |         .tv_sec = (time_t)(timeout / 1000000000),
         |          ^~~~~~
     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:201:19: warning: excess elements in struct initializer
     201 |         .tv_sec = (time_t)(timeout / 1000000000),
         |                   ^
     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/locking.h:201:19: note: (near initialization for 'ts')

   • Error on posix.c:

     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2153:61: error: invalid application of 'sizeof' to incomplete type 'struct os_poll_file_handle'
     2153 |         wasm_runtime_malloc((uint32)(nsubscriptions * sizeof(*pfds)));
         |                                                             ^
     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2177:25: error: invalid use of undefined type 'struct os_poll_file_handle'
     2177 |                     pfds[i] = (struct os_poll_file_handle){
         |                         ^
     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2178:26: error: 'struct os_poll_file_handle' has no member named 'fd'
     2178 |                         .fd = fos[i]->file_handle,
         |                          ^~
     /home/user/wasm-micro-runtime/core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c:2178:31: warning: excess elements in struct initializer
     2178 |                         .fd = fos[i]->file_handle,
         |

   This look like the compiler is not able to find the definition of the new types.

   • I tried to include the platform_internal.h in the locking.h and posix.c files to see if it solves the issue. But it didn't work.
   • I also tried to directly define the types in a new header and directly include it in the locking.h and posix.c files. But the issue still persist.
   • I even defined the whole file System API in a zephyr_file.c file (just returning __WASI_ENOSYS for now), and changed shared_platform.cmake. But the issue still persist.

   I'm still stuck on this issue. I will try to find a solution.

   If anyone is familiar with the implemenation of the sanboxed system primitives, or the how the abstract POSIX APIs are passed to sanboxed system primitives.

   Any help on this would be appreciated.

from wasm-micro-runtime.

@lucasAbadFr could you upload the patch or create a PR and change it to draft or WIP, so that others can have a try according to your work?

from wasm-micro-runtime.

@wenyongh thanks for your reply.

A new zephyr sample called simpe-http was added to demonstrate socket usage on zephyr. Refer to the sample readme for build purpose.

I've just opened a draft and noticed that I didn't adhere to the code guideline or other guidelines.

These changes will cause most tests to fail.

Please note that the work wasn't intended to be shared in this state.

from wasm-micro-runtime.

Hi @wenyongh,

I managed to make a simple wasi module work (by breaking a lot of things).

I would be interested to know if it work on someone else board.

I updated the readme.md under the new sample to give a brief overview of what was done.

Also i'm open to any tips to compile a WASI module with libc (from wasi-sdk) linked.

from wasm-micro-runtime.

@lucasAbadFr I am not sure who is using zephyr but I believe there are some developers requiring libc-wasi on zephyr. I found there are many modifications in wasm_runtime_common.c and core/iwasm/libraries/libc-wasi and wonder whether it is necessary? It would be better to add APIs and structures in zephyr platform and leave some APIs empty (or return false), and reduce the modification on core/iwasm and make the CIs run successfully.

from wasm-micro-runtime.

Thanks for the feedback.

The CI fail due to the new struct and API defined in platform_api_extension.h, to make it pass I could just typedef the new struct in platform_internal.h, and declare the API in their respective folder.

As for the changes in core/iwasm/libraries/libc-wasi/sandboxed-system-primitives, they are mostly here to abstract POSIX functions and use os_ abstraction. I think this is necessary to have at least a full abstraction.

The main changes are in core/iwasm/libraries/libc-wasi/sandboxed-system-primitives/src/posix.c where I had an offset due to struct size being different on both platform.

I'm curently investigating how to solve this issue and working to maintain compatibility with POSIX platform, which is the most used/maintained one.

from wasm-micro-runtime.

OK, I read the code and added some comments, it should be OK to abstract some os_xxx structures and functions. I think maybe you can enable the basic functionality and make CI run fine first, and then we can let others help review this PR and give suggestions, like how to abstract the APIs.

from wasm-micro-runtime.