mirage / mirage-platform Goto Github PK

Archived, see https://github.com/mirage/mirage/issues/1159 for details. Old: Core platform libraries for Mirage (UNIX and Xen). This provides the `OS` library which handles timers, device setup and the main loop, as well as the runtime for the Xen unikernel.

Home Page: https://mirage.io

License: Other

Shell 5.90% Makefile 0.30% C 93.80%

mirage-platform's Introduction

Status update, March 2018

mirage-platform is currently undergoing major refactoring. Please read this issue for detailed information.

The goal is to reuse the ocaml-freestanding work for the xen backend as well:

unix moved to mirage-unix (release pending) and switched to topkg and ocamlbuild
xen and bindings subdirectories moved to mirage-xen (release pending), also switched to topkg and ocamlbuild
the remaining xen-ocaml and xen-posix subdirectories will be deprecated once the move to ocaml-freestanding is complete

MirageOS

Mirage OS is a library operating system that constructs unikernels for secure, high-performance network applications across a variety of cloud computing and mobile platforms. Code can be developed on a normal OS such as Linux or MacOS X, and then compiled into a fully-standalone, specialised unikernel that runs under the Xen hypervisor.

Since Xen powers most public cloud computing infrastructure such as Amazon EC2 or Rackspace, this lets your servers run more cheaply, securely and with finer control than with a full software stack.

Mirage uses the OCaml language, with libraries that provide networking, storage and concurrency support that work under Unix during development, but become operating system drivers when being compiled for production deployment. The framework is fully event-driven, with no support for preemptive threading.

This repository contains an OCaml runtime used on Xen - the opam packages mirage-xen-posix and mirage-xen-ocaml.

For documentation, visit http://openmirage.org.

Repository Contents

xen-posix/ contains the header files to pretend a posix system (required to compile the OCaml runtime), plus minilibc and float formating -- this is the home for the mirage-xen-posix OPAM package. Installation goes into .opam/x/lib/mirage-xen-posix and .opam/x/include/mirage-xen-posix
xen-ocaml/ contains only the OCaml runtime (patches and build system), installation into .opam/x/lib/mirage-xen-ocaml and .opam/x/include/mirage-xen-ocaml

mirage-platform's People

Contributors

Stargazers

Watchers

mirage-platform's Issues

Remove all the outdated signatures

Now that we have https://github.com/mirage/mirage-types a lot of signatures are either duplicated or outdated.

I think that's also a good opportunity to split that repo a little further with the remaining implementations.

Missing `strtod`

val float_of_string (and the corresponding Scanf function) uses strod (see the relevant OCaml code). However It seems that we do not support it.

Include a LICENSE file

LICENSE files are easier for machines to see because the letters are bigger. Also, the opam file in ocaml/opam-repository is lacking a license field.

No visible output from OS.Console.log on Unix

Apparently due to buffering - this is just a duplicate of Mort's pull request to mirage/mirage, But it still isn't fixed so the mirage-skeleton tutorial examples appear not to work when you try them! (e.g. basic doesn't visibly print hello...)

Makefile PREFIX set incorrectly

...when using the opam system switch -- -- build failed because mirage-platform Makefile sets PREFIX ?= /usr/local which results in ocamlfind not being found. (since it's actually at ~/.opam/system/bin/ocamlfind)

Clock.time () inconsistent on ARM

Clock.time () / default_clock seems to return monotonic time since the unikernel started running in Xen mode on ARM, while in Unix mode and on x86 (Xen/Unix) it returns seconds since epoch.

xen: missing symbols on 4.01 snapshot-- runtime upgrade needed?

It looks like the code produced by the 4.01 snapshot doesn't link with the 4.00.1 runtime. If we are going to recommend people use the snapshot then we should bite the bullet and upgrade the runtime?

[mirari] + Executing: ld -d -nostdlib -m elf_x86_64 -T /local/scratch/djs/.opam/xen/lib/mirage-xen/mirage-x86_64.lds /local/scratch/djs/.opam/xen/lib/mirage-xen/x86_64.o _build/main.native.o /local/scratch/djs/.opam/xen/lib/mirage-xen/libocaml.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libxen.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libxencaml.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libdiet.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libm.a /local/scratch/djs/.opam/xen/lib/mirage-xen/longjmp.o -o _build/main.xen
/local/scratch/djs/.opam/xen/lib/mirage-xen/libdiet.a(sprintf.o): In function `sprintf':
sprintf.c:(.text+0x83): warning: warning: Avoid *sprintf; use *snprintf. It is more secure.
/local/scratch/djs/.opam/xen/lib/mirage-xen/libdiet.a(vsnprintf.o): In function `vsnprintf':
vsnprintf.c:(.text+0xc4): warning: warning: the printf functions add several kilobytes of bloat.
_build/main.native.o: In function `camlPrintexc__fun_1124':
(.text+0x4aa1a): undefined reference to `caml_get_current_callstack'
_build/main.native.o: In function `camlPrintexc__fun_1126':
(.text+0x4aa4a): undefined reference to `caml_get_exception_raw_backtrace'
_build/main.native.o: In function `camlPrintexc__print_raw_backtrace_1091':
(.text+0x4b07e): undefined reference to `caml_convert_raw_backtrace'
_build/main.native.o: In function `camlPrintexc__print_backtrace_1094':
(.text+0x4b0c2): undefined reference to `caml_get_exception_raw_backtrace'
_build/main.native.o: In function `camlPrintexc__raw_backtrace_to_string_1101':
(.text+0x4b23a): undefined reference to `caml_convert_raw_backtrace'
_build/main.native.o: In function `camlPrintexc__get_backtrace_1103':
(.text+0x4b26e): undefined reference to `caml_get_exception_raw_backtrace'
_build/main.native.o: In function `camlPrintexc__get_backtrace_1103':
(.text+0x4b287): undefined reference to `caml_convert_raw_backtrace'
_build/main.native.o: In function `camlPrintexc__35':
(.data+0x2c4b0): undefined reference to `caml_get_current_callstack'
_build/main.native.o: In function `camlPrintexc__35':
(.data+0x2c4c8): undefined reference to `caml_convert_raw_backtrace'
_build/main.native.o: In function `camlPrintexc__35':
(.data+0x2c4d0): undefined reference to `caml_get_exception_raw_backtrace'
[mirari] ERROR: The command "ld -d -nostdlib -m elf_x86_64 -T /local/scratch/djs/.opam/xen/lib/mirage-xen/mirage-x86_64.lds /local/scratch/djs/.opam/xen/lib/mirage-xen/x86_64.o _build/main.native.o /local/scratch/djs/.opam/xen/lib/mirage-xen/libocaml.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libxen.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libxencaml.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libdiet.a /local/scratch/djs/.opam/xen/lib/mirage-xen/libm.a /local/scratch/djs/.opam/xen/lib/mirage-xen/longjmp.o -o _build/main.xen" exited with code 1.

Improve README

In particular, make it clear that OS is only intended to be used directly by the auto-generated main.ml files.

(as discussed on IRC on 2016-07-27)

multi-page rings are broken because...

... we allocate a contiguous Io_page.t of size 2 pages. We then chop it into views using Bigarray.Array1.sub, we then 'grant' each of these views, the C stub uses a function 'base_page_of' which looks to the underlying C buffer and grants that. The result is that we grant the first page twice.

The base_page_of behaviour is convenient if your view points in the middle of the single page, since we want to grant access to the enclosing page.

I think what we really want to do is redefine base_page_of to return the start address of the current page containing the view, not the start of the underlying (much larger) C buffer.

add support for 4.04

See ocaml/opam-repository#7231 (problem report in opam-repository that lead to the current < 4.04 version constraint on mirage-xen-ocaml) and 59b58dc (the commit that enabled 4.03 support, which was also reported broken in the opam-repository PR above).

Add Ethif IP/Mac binding functions

From https://github.com/crotsos/signpostd

Grab the C bindings for MacOS X/Linux, and there are Xen equivalents also via xenstore.

unix backend needs a package dependency on tuntap

This needs an opam package dependency adding, once we've checked it doesn't mess up the xen backend.

Io_page.string_blit is broken (on all platforms)

let string_blit src srcoff dst dstoff len =
  for i = srcoff to srcoff + len - 1 do
    dst.{i+dstoff} <- src.[i]
  done

should be something like

let string_blit src srcoff dst dstoff len =
  for i = 0 to len - 1 do
    dst.{i+dstoff} <- src.[i+srcoff]
  done

Update dtoa.c in xen-posix/ to correct string_of_float operation

Now that I have a fix for mirage/ocaml-solo5#16, I've also reproduced the problem with Tim's test code (mirage/ocaml-solo5#16 (comment)) and can confirm that it exists on Xen.

The fix is to update xen-posix/src/dtoa.c to the version used by ocaml-freestanding, see mirage/ocaml-solo5#28 for details.

unix: ignore sigpipe

It is currently not ignored in the Unix backend

future of mirage-platform

during a discussion between @reynir @mato @yomimono we decided for a way forward with this repository:

split unix parts into mirage-unix
split xen parts into mirage-xen
superseed the remaining bits by ocaml-freestanding

the transcript of our meeting follows:

packages:
minios-xen (just the minios.o + package config stuff for linker flags)
(no external depenencies)
ocaml-freestanding, replacing mirage-xen-posix, mirage-xen-ocaml,
mirage-xen-minios
then we'll have mirage-xen and mirage-unix in mirage-platform (and can
move them to separate repositories)

build systems in mirage-platform could be moved to topkg/ocamlbuild (we
can copy mirage-solo5's setup for this)

needs to be some sysdeps_minios.c (like sysdeps_solo5.c) which includes
a console write (for printf and stdio) and exit, abort, gettimeofday,
times. now that there's no malloc in solo5, that's where sbrk and
malloc are. in the minios case, we need to provide some sbrk type thing
that can talk to minios's page allocator.

upstream minios doesn't namespace symbols, fixable with objcopy stuff
(and we might be able to upstream if we hide it behind a conf option),
but need to clean up the headers which is annoying. why do we care
about the namespacing of minios? we care because there might be conflict
with ocaml-freestanding (although we're not sure without trying it). we
could do a hack for sysdeps_minios where we include only minios headers,
and repeat any declarations that are internal to ocaml-freestanding, or
something like that

gmp and zarith are the ony things with native code that will have this
problem. if we do this, zarith-xen and gmp-xen will go away, but we need
to get them to compile, which is a matter of massaging pkgconfig

how much c code do we still have in mirage-xen that uses some specific
minios interface? that's the only code that needs to live both in
ocaml-freestanding and mirage-xen (IIUC)

homes of stubs: ocaml-cstruct & such have stubs both in solo5 and in
repositories like cstruct, which is strange; we would like to have that
cross-compiling only at a single point. hannes has changed his mind from
putting them in separate packages to into the backend repo

net and block stubs in solo5 will move from mirage-solo5 to
mirage-net-solo5 etc

where are those hooks in the xen side? it's a bit unclear, they are kind
of all over the place.

we might not need any of these changes for x86, but we don't want to
retire arm (although we can ignore it for now, maybe?)

(go through commit messages, mato thinks we might need like 10 of them
but maybe we can just start with upstream minios, see what's really
needed and put it back in)

it's unclear how our stuff interacts with the minios thread scheduler -
probably originally it did but now maybe not

so out of the stubs in mirage-platform/bindings (which will end up in
mirage-xen, with the dead code thrown out). from mirage-platform
xen-ocaml goes away, xen-posix goes away. there is stuff in xen, it
mirrors what's in mirage-solo5, including the magic which we'll still need

bindings -> mirage-xen/lib/mirage-bindings, mirage-xen -> mirage-xen/lib

hannes is willing to pull the mirage-xen and mirage-unix stuff into its
own repositories without making major changes

looking at dependencies for mirage-unix, shared-memory-ring "has stubs
but only barrier stubs", those should go into mirage-xen

martin is willing to do this, mindy also willing to learn a bunch of
stuff and try it in may if not already fixed by someone more familiar
with this area

gntshr stubs unnecessarily inefficient

The stub does this:

CAMLprim value
stub_gntshr_grant_access(value v_ref, value v_iopage, value v_domid, value v_wri
table)
{
    grant_ref_t ref = Int_val(v_ref);
    void *page = base_page_of(v_iopage);
    gntshr_grant_access(ref, page, Int_val(v_domid), !Bool_val(v_writable));

    return Val_unit;
}

where base_page_of looks up the data pointer of the Caml_ba_array_val. This means that to share a contiguous batch of pages we need to use Io_page.get_n (allocating a large page-aligned bigarray) and then use Io_page.to_pages to chop the buffer into separate pages, via Bigarray.Array1.sub.

It would be better to call Io_page.(to_cstruct (get_n ...)) and then to chop the Cstruct into pages since Cstruct.sub is cheaper than Bigarray.Array1.sub. We would modify the stub to take a Cstruct and compute the real page pointer by adding the buffer.Cstruct.off field.

Our code is using a mixture of Io_page.t and Cstruct.t which is a bit confusing. I think we sometimes use Io_page.t to guarantee alignment but this means we end up doing Bigarray.Array1.sub more often than we should and, if we don't do that and use Cstruct.sub instead, we have hard-to-find bugs where the wrong page is granted because of base_page_of.

Network packet fragments are being prematurely transmitted

If you add a 'Time.sleep 1' between pushing network packet fragments onto the xen VIF transmission ring then the backend sends the fragments as if they are complete ethernet frames, even though the more_data flag is set. Without the 'Time.sleep 1' the backend usually transmits the full packet.

Is this a race in the backend? Or something broken in the frontend?

(In my tests the backend is a debian wheezy running 3.3.0)

xen: domain crashing on startup

Here's what's on the console:

[root@dt52 ~]# cat /consoles/5
kernel.c: Mirage OS!
kernel.c:   start_info: 0x11ab000(VA)
kernel.c:     nr_pages: 0x10000
kernel.c:   shared_inf: 0xbf2dd000(MA)
kernel.c:      pt_base: 0x11ae000(VA)
kernel.c: nr_pt_frames: 0xd
kernel.c:     mfn_list: 0x112b000(VA)
kernel.c:    mod_start: 0x0(VA)
kernel.c:      mod_len: 0
kernel.c:        flags: 0x0
kernel.c:     cmd_line: 
x86_setup.c:   stack:      0x122f40-0x922f40
mm.c: MM: Init
x86_mm.c:       _text: 0x0(VA)
x86_mm.c:      _etext: 0xa2e9f(VA)
x86_mm.c:    _erodata: 0xc7000(VA)
x86_mm.c:      _edata: 0x10ab60(VA)
x86_mm.c: stack start: 0x122f40(VA)
x86_mm.c:        _end: 0x112b000(VA)
x86_mm.c:   start_pfn: 11be
x86_mm.c:     max_pfn: 10000
x86_mm.c: Mapping memory range 0x1400000 - 0x10000000
x86_mm.c: setting 0x0-0xc7000 readonly
x86_mm.c: skipped 0x1000
mm.c: MM: Initialise page allocator for 0x1234000 -> 0x10000000

Ticket created for @vbmithr (who built this kernel)

Possibly the kernel is not completely linked? That crash happened very early.

mirage-xen-ocaml uninstall is broken

[WARNING] failure in package uninstall script, some files may remain:
          # opam-version 1.2.1
          # os           linux
          # command      make xen-ocaml-uninstall
          PREFIX=/home/travis/.opam/system
          # path         /home/travis/.opam/system/build/mirage-xen-ocaml.2.3.0
          # compiler     system (4.02.1)
          # exit-code    2
          # env-file
          /home/travis/.opam/system/build/mirage-xen-ocaml.2.3.0/mirage-xen-ocaml-18906-548b09.env
          # stdout-file
          /home/travis/.opam/system/build/mirage-xen-ocaml.2.3.0/mirage-xen-ocaml-18906-548b09.out
          # stderr-file
          /home/travis/.opam/system/build/mirage-xen-ocaml.2.3.0/mirage-xen-ocaml-18906-548b09.err
          ### stdout ###
          # cd xen-ocaml && make uninstall
          # make[1]: Entering directory
          `/home/travis/.opam/system/build/mirage-xen-ocaml.2.3.0/xen-ocaml'
          # ./uninstall.sh
          # make[1]: Leaving directory
          `/home/travis/.opam/system/build/mirage-xen-ocaml.2.3.0/xen-ocaml'
          ### stderr ###
          # [...]
          # +prefix=
          # +[  =  ]
          # +opam config var prefix
          # +prefix=/home/travis/.opam/system
          # +odir=/home/travis/.opam/system/lib
          # +rm -f /home/travis/.opam/system/lib/pkgconfig/mirage-xen-ocaml.pc
          # +rm -f /home/travis/.opam/system/lib/mirage-xen-ocaml
          # rm: cannot remove `/home/travis/.opam/system/lib/mirage-xen-ocaml':
          Is a directory
          # make[1]: *** [uninstall] Error 1
          # make: *** [xen-ocaml-uninstall] Error 2
[WARNING] Directory /home/travis/.opam/system/lib/mirage-xen-ocaml is not
          empty, not removing

cut a 3.0.5 release for mirage-xen

#195 fixed support for 4.05.0; need to cut a 3.0.5 release so it's usable.

prepare release notes (#197)
tag the commit reflecting the release
make PR to opam-repository for mirage-xen-ocaml 3.0.5 (ocaml/opam-repository#10812)

appveyor tests seem not to work

They used to work: https://ci.appveyor.com/project/djs55/mirage-platform/build/1.0.38 is the passing test for #165 from May of this year, but it appears that no PR since then has successfully built on Appveyor.

It appears to my untrained eye that the combination of us not setting PACKAGE and there not being a single opam file for this repository are causing the usual build path to fail.

CFLAGS confusion

MiniOS exports (via pkg-config) some CFLAGS (using ARCH_CFLAGS) -- more precisely x86 and arm

MiniOS is built with DEF_FLAGS, which does not include any optimization level
bindings, xen-ocaml, and xen-posix repeat parts of the logic (esp -fno-stack-protector, which should be exported from MiniOS) and define some custom CFLAGS... shouldn't this be more unified (and use pkg-config)?
we should use a good optimization level (or does something not compile with -O3?)

Opam2: Calling opam config exec -- ocamlfind query foo fails in the sandbox

The build script calls opam config exec -- ocamlfind query foo and this fails in opam2 because ~/opam/log/ is read-only in the sandbox. I'm not sure if I should file this as a bug in opam or here. Filing it here for now.

install cmx for the pack

Only the cmxa is installed atm

4.02.2 support for tracing patch

The tracing patch in mirage-xen-ocaml no longer applies due to significant changes in minor_gc.c in ocaml 4.02.2 and so the build fails.

/cc @talex5 : it looks like support has been added for GC hooks in 4.02.2, so I thought I'd leave a bug open since the tracing patch needs to be rewritten.

Simplify the build system

This implies removing all the shell scripts and the symlink hacks (and use proper #ifdfef directives)

Installation fails if multiple findlib paths are defined.

If findlib.conf contains a path with multiple components, then the mirage-xen package fails with a series of "cp: omitting directory..." as it receives more than 2 arguments. I think it's correct to use destdir instead of path for XEN_LIB in xen/Makefile, since the makefile installs without passing any -destdir.

There is a similar issue in mirage. The same fix would be good enough, but if you prefer I can send a pull request for replacing the configure-time substitution into the makefile with a XEN_LIB = $(shell ocamlfind query mirage-xen).

#require "mirage";; fails on Linux with 4.00.1

Just so that we keep a track of this somewhere:

utop $ #require "mirage";;
/usr/lib/ocaml/threads: added to search path                                                                                                                                                                     /usr/lib/ocaml/dynlink.cma: loaded                                                                                                                                                                               /usr/lib/ocaml/camlp4: added to search path                                                                                                                                                                      
/home/vb/.opam/system/lib/optcomp: added to search path
/home/vb/.opam/system/lib/ocplib-endian: added to search path
/home/vb/.opam/system/lib/ocplib-endian/ocplib_endian.cma: loaded
/home/vb/.opam/system/lib/ocplib-endian/bigstring.cma: loaded
/home/vb/.opam/system/lib/cstruct: added to search path
/home/vb/.opam/system/lib/cstruct/cstruct.cma: loaded
/home/vb/.opam/system/lib/lwt: added to search path
/home/vb/.opam/system/lib/mirage: added to search path
/home/vb/.opam/system/lib/mirage/oS.cma: loaded
Error: The external function `pcap_opendev' is not available

And here is Haris answer:

This is something I did a while back in order to allow interfaces to open in Mac environments. It was the only way to open raw ethernet sockets in MacOSX. The equivalent action in Linux should open a raw ethernet socket using Libc (Something along this lines http://aschauf.landshut.org/fh/linux/udp_vs_raw/ch01s03.html). At the moment I haven't fixed the Linux code to work. In the case of Linux I had an ifdef function which implements an empty function for this.

It is a bit complicated to handle this case because there is a different approach on how to read packets from pcap fd in Macosx and from raw ethernet sockets in Linux. The difference is that in pcap you need to skip the pcap header. This is not the case in Linux. I can fix quickly the code if you could give me some hint on how I could detect in Ocaml the OS type.

Support OCaml 4.02.3

The new release of OCaml breaks our patches. See mirage/mirage#440 (comment)

Remove dependency on xenstore

Now that blkif and netif are out-of-tree, it should be possible to remove the dependency on xenstore. We would need to add some kind of pre-suspend hook (xenstore is the only service which is shutdown pre-suspend, everything else just re-issues requests post-resume); and functorise netif over xenstore (like blkif).

Retain exception information - ?

Hey!

I'm using mirage-xen.

I frequently have the problem that exceptions raised here https://github.com/mirage/mirage-platform/blob/master/xen/lib/main.ml#L60 do not retain the original source in the backtrace.

Instead I get something like:

main: (Invalid_argument "String.sub / Bytes.sub")
Raised at file "src/core/lwt.ml" line 2719, characters 18-25
Called from file "lib/main.ml", line 60, characters 15-25

Does anybody know a way to obtain a more precise error location?

travis ci broken. fix

Can't build mirage-xen.2.3.2

w/ opam:

#=== ERROR while installing mirage-xen.2.3.2 ==================================#
# opam-version 1.2.2
# os           linux
# command      make xen-build
# path         /home/self/.opam/system/build/mirage-xen.2.3.2
# compiler     system (4.02.1)
# exit-code    2
# env-file     /home/self/.opam/system/build/mirage-xen.2.3.2/mirage-xen-21134-9088e2.env
# stdout-file  /home/self/.opam/system/build/mirage-xen.2.3.2/mirage-xen-21134-9088e2.out
# stderr-file  /home/self/.opam/system/build/mirage-xen.2.3.2/mirage-xen-21134-9088e2.err
### stdout ###
# [...]
# touch lib/OS.mli  ; if  /home/self/.opam/system/bin/ocamlfind ocamlopt -pack -g -annot -bin-annot -I lib lib/activations.cmx lib/device_state.cmx lib/env.cmx lib/time.cmx lib/main.cmx lib/start_info.cmx lib/xs.cmx lib/sched.cmx lib/xenctrl.cmx -o lib/OS.cmx  ; then  rm -f lib/OS.mli  ; else  rm -f lib/OS.mli  ; exit 1; fi
# /home/self/.opam/system/bin/ocamlfind ocamlopt -a lib/OS.cmx -o lib/OS.cmxa
# /home/self/.opam/system/bin/ocamlfind ocamlopt -shared lib/OS.cmxa lib/OS.cmx -o lib/OS.cmxs
# make[1]: Leaving directory '/home/self/.opam/system/build/mirage-xen.2.3.2/xen'
# cd bindings && make build
# make[1]: Entering directory '/home/self/.opam/system/build/mirage-xen.2.3.2/bindings'
# ./build.sh
# Makefile:4: recipe for target 'build' failed
# make[1]: Leaving directory '/home/self/.opam/system/build/mirage-xen.2.3.2/bindings'
# Makefile:37: recipe for target 'xen-build' failed
### stderr ###
# In file included from /home/self/.opam/system/include/minios-xen/inttypes.h:22:0,
# [...]
#                  from start_info_stubs.c:17:
# /home/self/.opam/system/include/minios-xen/stdint.h:1:24: fatal error: stdint-gcc.h: No such file or directory
# compilation terminated.
# In file included from /home/self/.opam/system/include/minios-xen/os.h:15:0,
#                  from xb_stubs.c:14:
# /home/self/.opam/system/include/minios-xen/types.h:22:20: fatal error: stddef.h: No such file or directory
# compilation terminated.
# make[1]: *** [build] Error 1
# make: *** [xen-build] Error 2

Same when working on a source checkout:

$ mirage-platform  make xen-build
cd xen && make
make[1]: Entering directory '/home/self/coad/projects/mirleft/mirage-platform/xen'
ocaml setup.ml -configure

Configuration:
ocamlfind: ........................................... /home/self/.opam/system/bin/ocamlfind
ocamlc: .............................................. /usr/bin/ocamlc.opt
ocamlopt: ............................................ /usr/bin/ocamlopt.opt
ocamlbuild: .......................................... /usr/bin/ocamlbuild
Package name: ........................................ mirage-xen
Package version: ..................................... 2.3.1
os_type: ............................................. Unix
system: .............................................. linux
architecture: ........................................ amd64
ccomp_type: .......................................... cc
ocaml_version: ....................................... 4.02.1
standard_library_default: ............................ /usr/lib/ocaml
standard_library: .................................... /usr/lib/ocaml
standard_runtime: .................................... /usr/bin/ocamlrun
bytecomp_c_compiler: ................................. gcc -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC
native_c_compiler: ................................... gcc -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT
model: ............................................... default
ext_obj: ............................................. .o
ext_asm: ............................................. .s
ext_lib: ............................................. .a
ext_dll: ............................................. .so
default_executable_name: ............................. a.out
systhread_supported: ................................. true
Install architecture-independent files dir: .......... /usr/local
Install architecture-dependent files in dir: ......... $prefix
User executables: .................................... $exec_prefix/bin
System admin executables: ............................ $exec_prefix/sbin
Program executables: ................................. $exec_prefix/libexec
Read-only single-machine data: ....................... $prefix/etc
Modifiable architecture-independent data: ............ $prefix/com
Modifiable single-machine data: ...................... $prefix/var
Object code libraries: ............................... $exec_prefix/lib
Read-only arch-independent data root: ................ $prefix/share
Read-only architecture-independent data: ............. $datarootdir
Info documentation: .................................. $datarootdir/info
Locale-dependent data: ............................... $datarootdir/locale
Man documentation: ................................... $datarootdir/man
Documentation root: .................................. $datarootdir/doc/$pkg_name
HTML documentation: .................................. $docdir
DVI documentation: ................................... $docdir
PDF documentation: ................................... $docdir
PS documentation: .................................... $docdir
findlib_version: ..................................... 1.5.5
is_native: ........................................... true
suffix_program: ......................................
Remove a file.: ...................................... rm -f
Remove a directory.: ................................. rm -rf
Turn ocaml debug flag on: ............................ true
Turn ocaml profile flag on: .......................... false
Compiler support generation of .cmxs.: ............... true
OCamlbuild additional flags: .........................
Create documentations: ............................... true
Compile tests executable and library and run them: ... false
pkg_lwt: ............................................. /home/self/.opam/system/lib/lwt
pkg_cstruct: ......................................... /home/self/.opam/system/lib/cstruct
pkg_xen_evtchn: ...................................... /home/self/.opam/system/lib/xen-evtchn
pkg_xen_gnt: ......................................... /home/self/.opam/system/lib/xen-gnt
pkg_xenstore_client: ................................. /home/self/.opam/system/lib/xenstore
pkg_shared_memory_ring: .............................. /home/self/.opam/system/lib/shared-memory-ring
pkg_shared_memory_ring_lwt: .......................... /home/self/.opam/system/lib/shared-memory-ring
pkg_shared_memory_ring_xenstore: ..................... /home/self/.opam/system/lib/shared-memory-ring
pkg_shared_memory_ring_console: ...................... /home/self/.opam/system/lib/shared-memory-ring
pkg_mirage_types: .................................... /home/self/.opam/system/lib/mirage-types
pkg_mirage_clock_xen: ................................ /home/self/.opam/system/lib/mirage-clock-xen
pkg_io_page: ......................................... /home/self/.opam/system/lib/io-page
pkg_mirage_profile: .................................. /home/self/.opam/system/lib/mirage-profile
pkg_lwt_syntax: ...................................... /home/self/.opam/system/lib/lwt
pkg_cstruct_syntax: .................................. /home/self/.opam/system/lib/cstruct

ocaml setup.ml -build
W: Cannot find source file matching module 'OS' in library OS
Finished, 1 target (0 cached) in 00:00:00.
Finished, 50 targets (0 cached) in 00:00:02.
make[1]: Leaving directory '/home/self/coad/projects/mirleft/mirage-platform/xen'
cd bindings && make build
make[1]: Entering directory '/home/self/coad/projects/mirleft/mirage-platform/bindings'
./build.sh
++ opam config var prefix
+ export PKG_CONFIG_PATH=/home/self/.opam/system/lib/pkgconfig
+ PKG_CONFIG_PATH=/home/self/.opam/system/lib/pkgconfig
+ PKG_CONFIG_DEPS='mirage-xen-minios mirage-xen-ocaml'
+ pkg-config --print-errors --exists mirage-xen-minios mirage-xen-ocaml
++ pkg-config --cflags mirage-xen-ocaml
+ CFLAGS='-isystem/home/self/.opam/system/lib/pkgconfig/../../include/minios-xen -isystem/home/self/.opam/system/lib/pkgconfig/../../include/minios-xen/posix -m64 -mno-red-zone -fno-reorder-blocks -fno-stack-protector -fno-asynchronous-unwind-tables -nostdinc -isystem /usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/include -U__linux__ -U__FreeBSD__ -U__sun__ -U__linux -D__MINIOS__ -I/home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-ocaml/include -I/home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-posix/include -I/home/self/.opam/system/include '
++ pkg-config --cflags mirage-xen-minios mirage-xen-ocaml
+ MINIOS_CFLAGS='-D__XEN_INTERFACE_VERSION__=0x00030205 -D__INSIDE_MINIOS__ -isystem/home/self/.opam/system/lib/pkgconfig/../../include/minios-xen -isystem/home/self/.opam/system/lib/pkgconfig/../../include/minios-xen/posix -m64 -mno-red-zone -fno-reorder-blocks -fno-stack-protector -fno-asynchronous-unwind-tables -nostdinc -isystem /usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/include -U__linux__ -U__FreeBSD__ -U__sun__ -U__linux -D__MINIOS__ -I/home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-ocaml/include -I/home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-posix/include -I/home/self/.opam/system/include '
++ gcc -dumpversion
++ cut -f2 -d.
+ GCC_MVER2=1
+ '[' 1 -ge 8 ']'
+ CC=cc
+ cc -Wall -Wno-attributes -D__XEN_INTERFACE_VERSION__=0x00030205 -D__INSIDE_MINIOS__ -isystem/home/self/.opam/system/lib/pkgconfig/../../include/minios-xen -isystem/home/self/.opam/system/lib/pkgconfig/../../include/minios-xen/posix -m64 -mno-red-zone -fno-reorder-blocks -fno-stack-protector -fno-asynchronous-unwind-tables -nostdinc -isystem /usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/include -U__linux__ -U__FreeBSD__ -U__sun__ -U__linux -D__MINIOS__ -I/home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-ocaml/include -I/home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-posix/include -I/home/self/.opam/system/include -c barrier_stubs.c eventchn_stubs.c exit_stubs.c gnttab_stubs.c main.c sched_stubs.c start_info_stubs.c xb_stubs.c
In file included from barrier_stubs.c:17:0:
/home/self/.opam/system/include/minios-xen/stdint.h:1:24: fatal error: stdint-gcc.h: No such file or directory
compilation terminated.
In file included from /home/self/.opam/system/include/minios-xen/os.h:15:0,
                 from eventchn_stubs.c:16:
/home/self/.opam/system/include/minios-xen/types.h:22:20: fatal error: stddef.h: No such file or directory
compilation terminated.
In file included from /home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-ocaml/include/caml/mlvalues.h:21:0,
                 from exit_stubs.c:16:
/home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-ocaml/include/caml/misc.h:26:20: fatal error: stddef.h: No such file or directory
compilation terminated.
In file included from /home/self/.opam/system/include/minios-xen/os.h:15:0,
                 from gnttab_stubs.c:19:
/home/self/.opam/system/include/minios-xen/types.h:22:20: fatal error: stddef.h: No such file or directory
compilation terminated.
In file included from /home/self/.opam/system/include/minios-xen/os.h:15:0,
                 from main.c:17:
/home/self/.opam/system/include/minios-xen/types.h:22:20: fatal error: stddef.h: No such file or directory
compilation terminated.
In file included from /home/self/.opam/system/include/minios-xen/os.h:15:0,
                 from sched_stubs.c:15:
/home/self/.opam/system/include/minios-xen/types.h:22:20: fatal error: stddef.h: No such file or directory
compilation terminated.
In file included from /home/self/.opam/system/include/minios-xen/inttypes.h:22:0,
                 from /home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-posix/include/sys/types.h:4,
                 from /home/self/.opam/system/lib/pkgconfig/../../include/mirage-xen-posix/include/stdio.h:5,
                 from start_info_stubs.c:17:
/home/self/.opam/system/include/minios-xen/stdint.h:1:24: fatal error: stdint-gcc.h: No such file or directory
compilation terminated.
In file included from /home/self/.opam/system/include/minios-xen/os.h:15:0,
                 from xb_stubs.c:14:
/home/self/.opam/system/include/minios-xen/types.h:22:20: fatal error: stddef.h: No such file or directory
compilation terminated.
Makefile:4: recipe for target 'build' failed
make[1]: *** [build] Error 1
make[1]: Leaving directory '/home/self/coad/projects/mirleft/mirage-platform/bindings'
Makefile:37: recipe for target 'xen-build' failed
make: *** [xen-build] Error 2

$ mirage-platform  ll =cc
lrwxrwxrwx 1 root root 3 May 21 10:12 /usr/bin/cc -> gcc*

$ mirage-platform  gcc --version                                                                      :(
gcc (GCC) 5.1.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Running Arch Linux.

Both stdint-gcc.h and stddef.h live in /usr/lib/gcc/x86_64-unknown-linux-gnu/5.1.0/include/.

Worked the last time around. I had gcc-4.9.2 then, got 5.1.0 in the meantime. I assume include path trickery has changed since. Then again, minios headers directly including compiler-specific headers looks a little short of being... robust.

Will investigate as time permits.

Unsupported function times called in Mini-OS kernel

Seems that Sys.time is not supported by the runtime. We should provide some monotonic time there.

Unsupported function times called in Mini-OS kernel
Do_exit called!
base is 0x4efc80 caller is 0x290930
base is 0x3 Page fault at linear address b, rip 2ae747, regs 00000000004efba8, sp 4efc50, our_sp 00000000004efb68, code 0
RIP: e030:[<00000000002ae747>]
RSP: e02b:00000000004efc50  EFLAGS: 00010006
RAX: 000000000000000c RBX: 0000000000000003 RCX: 0000000000000c0c
RDX: 0000000000000c0c RSI: 00000000004efab0 RDI: 0000000000000004
RBP: 00000000004efc80 R08: 000000000000040b R09: 0000000000000020
R10: 000000000000000c R11: 0000000000000000 R12: 000000000011d967
R13: 0000000000000005 R14: 00000000004efd90 R15: 000000000088f190
base is 0x4efc80 caller is 0x290930
base is 0x3 Page fault in pagetable walk (access to invalid memory?).

mirage-platform disagreggation

We should aim to move as much as C stubs as possible to their own repository.

This means:

Move cstruct_stubs to cstruct
Move unsafe_load_uint32, unsafe_save_uint32 to cstruct
Maybe move atomic_or_fetch_uint8 and atomic_fetch_and_uint8 to cstruct
Move clock_stubs to mirage-clock
Move eventch_stubs to xen-evtchn
Move gnttab_stubs to xen-gnt
Move xb_stubs to xenstore (or somewhere else which handle the Xenbus protocol)?

mirage-xen does not work with OCaml 4.05

Linking the noop example from mirage-skeleton gives:

$ make
mirage build
_build/main.native.o: In function `camlBigarray__map_file_inner_1994':
/home/user/.opam/4.05.0/build/ocaml/otherlibs/bigarray/bigarray.ml:137: undefined reference to `caml_ba_map_file'
_build/main.native.o: In function `camlBigarray__49':
:(.data+0x67c70): undefined reference to `caml_ba_map_file'