hyperhq / hyperstart Goto Github PK
View Code? Open in Web Editor NEWThe tiny Init service for HyperContainer
Home Page: https://www.hypercontainer.io
License: Apache License 2.0
The tiny Init service for HyperContainer
Home Page: https://www.hypercontainer.io
License: Apache License 2.0
directly use symlink() and unlink().
hyper_create() also needs to be removed. see #204
Is there any tutorial that could tell us how to build a qboot image?
[root@centos-0318811974 /]# chgrp adm /a
chgrp: changing group of '/a': Operation not permitted
found in centos and ubuntu image, overlay and aufs driver.
and
[root@h8s-single centos]# grep clear /etc/libvirt/qemu.conf
clear_emulator_capabilities = 0
This issue is filed by @feiskyer , the original bug link is hyperhq/hyperd#71
I created a pod using following spec:
{
"containers": [
{
"envs": [
{
"env": "SERVICE_HOST",
"value": "10.254.0.1"
},
{
"env": "SERVICE_PORT",
"value": "443"
}
],
"image": "nginx",
"name": "nginx",
"ports": [
{
"containerPort": 80,
"protocol": "TCP"
}
],
"tty": false,
}
],
"resource": {
"memory": 192,
"vcpu": 1
},
"tty": true,
"type": "pod",
}
But there is no SERVICE_HOST
or SERVICE_PORT
found in environment variables:
# hyper exec pod-TToRTANXBb env
HOME=/
TERM=linux
PATH=/bin:/sbin/:/usr/bin/:/usr/sbin/
PWD=/
I try to run docker in hyper container, but failed. The problems are:
Storage driver
No storage driver (dm, aufs, overlay, etc...) is supported by hyper kernel.
Bridge module
bash-4.1# /sbin/modprobe -v bridge
install /sbin/modprobe --ignore-install bridge && /sbin/sysctl -q -w net.bridge.bridge-nf-call-arptables=0 net.bridge.bridge-nf-call-iptables=0 net.bridge.bridge-nf-call-ip6tables=0
FATAL: Module bridge not found.
FATAL: Error running install command for bridge
bash-4.1#
The init binary executes /usr/sbin/rtas_errd
to configure the VM network adapters in ppc64le. This comes from build/arch/ppc64le/binary/rtas.tar
in the repository but the shared libraries it requires are not copied into lib64 in build/make-initrd.sh
resulting in these errors in the vmconsole output:
I0417 14:24:20.944302 47891 watcher.go:74] vmconsole: executing cmd /usr/sbin/rtas_errd -d
I0417 14:24:20.945676 47891 watcher.go:74] vmconsole: create directory /tmp/hyper/shared
I0417 14:24:20.946671 47891 watcher.go:74] vmconsole: /usr/sbin/rtas_errd: error while loading shared libraries: librtas.so.2: cannot open shared object file: No such file or directory
I0417 14:24:20.946690 47891 watcher.go:74] vmconsole: create directory /tmp/hyper/shm
I0417 14:24:20.946824 47891 watcher.go:74] vmconsole: pod init pid 1707
I0417 14:24:20.947125 47891 watcher.go:74] vmconsole: /usr/sbin/rtas_errd -d cmd exit normally, status 127
I0417 14:24:20.947243 47891 watcher.go:74] vmconsole: cmd /usr/sbin/rtas_errd -d exit unexpectedly, status 32512
I0417 14:24:20.947359 47891 watcher.go:74] vmconsole: rtas start failed: No such file or directory
Environment
Fedora 26 x86_64
clang 3.9.1
git faef420
Error
/usr/bin/clang -DHAVE_CONFIG_H -I. -I.. -Wall -Werror -DVERSIONCOMMIT=\""0.8.1, commit: `git describe --dirty --always --tags 2> /dev/null || true`"\" -g -O2 -MT util.o -MD -MP -MF $depbase.Tpo -c -o util.o util.c &&\
mv -f $depbase.Tpo $depbase.Po
util.c:251:9: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare]
if (l < 0) {
~ ^ ~
1 error generated.
make[2]: *** [Makefile:362: util.o] Error 1
Environment
Fedora 26 x86_64
gcc 7.1.1
git hash faef420
Error
gcc -DHAVE_CONFIG_H -I. -I.. -Wall -Werror -DVERSIONCOMMIT=\""0.8.1, commit: `git describe --dirty --always --tags 2> /dev/null || true`"\" -g -O2 -MT util.o -MD -MP -MF $depbase.Tpo -c -o util.o util.c &&\
mv -f $depbase.Tpo $depbase.Po
util.c: In function ‘online_cpu’:
util.c:514:42: error: ‘%s’ directive writing up to 255 bytes into a region of size 232 [-Werror=format-overflow=]
sprintf(path, "/sys/devices/system/cpu/%s/online", entry->d_name);
^~
util.c:514:3: note: ‘sprintf’ output between 32 and 287 bytes into a destination of size 256
sprintf(path, "/sys/devices/system/cpu/%s/online", entry->d_name);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
util.c: In function ‘online_memory’:
util.c:550:45: error: ‘%s’ directive writing up to 255 bytes into a region of size 229 [-Werror=format-overflow=]
sprintf(path, "/sys/devices/system/memory/%s/online", entry->d_name);
^~
util.c:550:3: note: ‘sprintf’ output between 35 and 290 bytes into a destination of size 256
sprintf(path, "/sys/devices/system/memory/%s/online", entry->d_name);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libevent is much mature and easy to use.
refer: https://github.com/kgabis/parson
example: #123
apis: https://github.com/kgabis/parson/blob/master/parson.h
JSON_Value *value = hyper_json_parse(json, length); #parse, return root value
json_object_get_string(json_object(value), "container"); # retrieve a filed
(int)json_object_get_number(json_object(value), "signal");
json_object_dotget_string(json_object(value), "container.process.workdir"); # retrieve a filed via dot
json_value_free(value); # free
JSON_Value * json_object_get_value (const JSON_Object *object, const char *name);
const char * json_object_get_string (const JSON_Object *object, const char *name);
JSON_Object * json_object_get_object (const JSON_Object *object, const char *name);
JSON_Array * json_object_get_array (const JSON_Object *object, const char *name);
double json_object_get_number (const JSON_Object *object, const char *name); /* returns 0 on fail */
int json_object_get_boolean(const JSON_Object *object, const char *name); /* returns -1 on fail */
/* dotget functions enable addressing values with dot notation in nested objects,
just like in structs or c++/java/c# objects (e.g. objectA.objectB.value).
Because valid names in JSON can contain dots, some values may be inaccessible
this way. */
JSON_Value * json_object_dotget_value (const JSON_Object *object, const char *name);
const char * json_object_dotget_string (const JSON_Object *object, const char *name);
JSON_Object * json_object_dotget_object (const JSON_Object *object, const char *name);
JSON_Array * json_object_dotget_array (const JSON_Object *object, const char *name);
double json_object_dotget_number (const JSON_Object *object, const char *name); /* returns 0 on fail */
int json_object_dotget_boolean(const JSON_Object *object, const char *name); /* returns -1 on fail */
JSON_Value * json_array_get_value (const JSON_Array *array, size_t index);
const char * json_array_get_string (const JSON_Array *array, size_t index);
JSON_Object * json_array_get_object (const JSON_Array *array, size_t index);
JSON_Array * json_array_get_array (const JSON_Array *array, size_t index);
double json_array_get_number (const JSON_Array *array, size_t index); /* returns 0 on fail */
int json_array_get_boolean(const JSON_Array *array, size_t index); /* returns -1 on fail */
size_t json_array_get_count (const JSON_Array *array);
NFS via vsock is under developing, and is promising for the future FS sharing of hyper, because:
Some blocker of adopting vsock NFS
Some references:
Related repos:
Add script to build the kernel with the config in a hypercontainer or native environment.
A Dockerfile is also required for defining the hypercontainer's image.
The SETUPINTERFACE command parses the json for a single interface and configures it. It would be nice to have a command (setupmultiinterface/setupinterfaces?) to configure multiple interfaces or modify the existing command to support multiple interfaces. Helps reduce the round trips in case of multiple interfaces. I see that the SETUPROUTE command, currently supports multiple routes.
/ # ls -l /proc/$$/fd
total 0
lrwx------ 1 root root 64 Aug 29 08:36 0 -> /dev/pts/0
lrwx------ 1 root root 64 Aug 29 08:36 1 -> /dev/pts/0
lrwx------ 1 root root 64 Aug 29 08:36 10 -> /dev/tty
lrwx------ 1 root root 64 Aug 29 08:36 2 -> /dev/pts/0
lrwx------ 1 root root 64 Aug 29 08:36 6 -> socket:[701]
socket:[701] is sure to be a leaking fd. it might be netlink fd. @gao-feng
/dev/tty is unsure what is it. It might be opened by busybox.
[root@sbox runv]# docker exec b3ce4aaa72c0 ls -l '/proc/self/fd'
total 0
lr-x------ 1 root root 64 Aug 29 08:42 0 -> pipe:[874]
l-wx------ 1 root root 64 Aug 29 08:42 1 -> pipe:[875]
l-wx------ 1 root root 64 Aug 29 08:42 2 -> pipe:[876]
ls: /proc/self/fd/3: cannot read link: No such file or directory
lr-x------ 1 root root 64 Aug 29 08:42 3
lrwx------ 1 root root 64 Aug 29 08:42 6 -> socket:[701]
what is the fd/3 ?
While hyperstart try to run command to setup route, the guest kernel will crash:
hyper_ctlmsg_handle SETUPROUTE
init[1]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006
pgd = ffffffc005767000
[00000000] *pgd=0000000085769003, *pud=0000000085769003
, *pmd=0000000000000000
CPU: 0 PID: 1 Comm: init Not tainted 4.9.36 #3
Hardware name: linux,dummy-virt (DT)
task: ffffffc00744ad00 task.stack: ffffffc00744c000
PC is at 0x406ba8
LR is at 0x4079f0
pc : [<0000000000406ba8>] lr : [<00000000004079f0>] pstate: 60000000
sp : 0000007ffec98cb0
x29: 0000007ffec98cb0 x28: 0000000000000000
x27: 000000000042c000 x26: 000000002f2131c0
x25: 0000000000000015 x24: 0000000000416000
x23: 000000000042c000 x22: 0000007ffec99170
x21: 000000000042c000 x20: 000000000042c000
x19: 0000000000000000 x18: 0000000000000001
x17: 0000007f95058988 x16: 000000000042c2a8
x15: 0000000000000001 x14: 0000000000000003
x13: 0000000000417b58 x12: 00000000ffffffff
x11: 000000000000000a x10: 0000000000000000
x9 : 0000000000000001 x8 : 00000000ffffffff
x7 : 0000000000000002 x6 : 000000002f2131d0
x5 : 0000000000000001 x4 : 0000000000000001
x3 : 0000000000000000 x2 : 000000000042c448
x1 : 000000000042c640 x0 : 0000000000000000
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
CPU: 0 PID: 1 Comm: init Not tainted 4.9.36 #3
Hardware name: linux,dummy-virt (DT)
Call trace:
[] dump_backtrace+0x0/0x198
[] show_stack+0x14/0x20
[] dump_stack+0x94/0xb8
[] panic+0x110/0x258
[] complete_and_exit+0x0/0x20
[] do_group_exit+0x38/0xa8
[] get_signal+0x234/0x4d0
[] do_signal+0x354/0x508
[] do_notify_resume+0x90/0xb0
[] work_pending+0x8/0x10
Kernel Offset: disabled
Memory Limit: none
---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
E0904 15:08:58.579042 2185 network.go:184] rpc error: code = Internal desc = transport is closing
E0904 15:09:58.939408 2185 vm_states.go:246] SB[vm-hBJcxNbNru] Shutting down because of an exception: %!(EXTRA string=Destroy pod failed: &grpc.rpcError{code:0xe, desc:"grpc: the connection is unavailable"})
Create new container failed: rpc error: code = Internal desc = transport is closing
We have created a PR# #342 to fix this issue.
I am using runv and need to use my own kernel for the vm rather than the default one of hyperstart.
Then I tried to build my own kernel using hyperstart.
Is this the right way to build local kernel?
So this is the source for init, but I would also like to see the source for Hyperkernel. Is it possible to have the source for that as well? I'm mainly just curious about which build system was used for it - Buildroot? Yocto? Something custom? Thanks in advance!
Although vm is strong isolation, but capabilities might be still need for some other cases...
all containers need to share the same /dev/shm, the same tmpfs mount with bindmount for sharing.
since all containers share the same ipc ns in the sandbox.
More investigation is needed.
ref: setupIpcDirs() in moby/daemon/container_operations_unix.go
Hyperstart doesn't implement pid-namespacing when containers share a pod (i.e. when you have shared container network namespaces).
make will failed since 'make-initrd.sh' has more unexpectledy 'cd ../'.
[ray@localhost hyperstart]$ make
......
make[2]: Entering directory '/home/ray/go/src/github.com/hyperhq/hyperstart/build'
bash ./make-initrd.sh vbox
build initrd for vbox
28547 blocks
cp: cannot stat '../vbox/kernel': No such file or directory
cp: cannot stat '../hyper-initrd.img': No such file or directory
cp: cannot stat '../vbox/isolinux/isolinux.bin': No such file or directory
cp: cannot stat '../vbox/isolinux/ldlinux.c32': No such file or directory
I: -input-charset not specified, using utf-8 (detected in locale settings)
genisoimage: Uh oh, I cant find the boot image 'isolinux/isolinux.bin' !
make[2]: Leaving directory '/home/ray/go/src/github.com/hyperhq/hyperstart/build'
make[2]: Entering directory '/home/ray/go/src/github.com/hyperhq/hyperstart'
make[2]: Leaving directory '/home/ray/go/src/github.com/hyperhq/hyperstart'
make[1]: Leaving directory '/home/ray/go/src/github.com/hyperhq/hyperstart'
In exec.c:265
:
// append additional groups to supplementary groups
int i;
reallocgroups = realloc(groups, sizeof(gid_t) * (ngroups + exec->nr_additional_groups));
if (reallocgroups == NULL)
goto fail;
groups = reallocgroups;
for (i = 0; i < exec->nr_additional_groups; i++) {
fprintf(stdout, "try to find the group: %s\n", exec->additional_groups[i]);
struct group *gr = hyper_getgrnam(exec->additional_groups[i]);
if (gr == NULL) {
perror("can't find additional group");
goto fail;
}
groups[ngroups] = gr->gr_gid;
ngroups++;
}
This stanza of code is broken when using runv with docker - the problem is docker passes in a huge stanza of additional groups by default from the user running it (docker 1.12.1 at least):
I0106 16:43:13.550083 25707 vm_console.go:46] SB[vm-rDTiYKRrYK] [CNL] call hyper_new_container, json {"id":"1fa52cef9d6a715e905ae2c07e178f31ca87671a0c706f690a1f973717b2286e","rootfs":"rootfs","image":"1fa52cef9d6a715e905ae2c07e178f31ca87671a0c706f690a1f973717b2286e","process":{"id":"init","additionalGroups":["0","1","2","3","4","6","10","11","20","26","27"],"terminal":true,"stdio":1,"args":["sleep","10"],"envs":[{"env":"PATH","value":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"},{"env":"HOSTNAME","value":"1fa52cef9d6a"},{"env":"TERM","value":"xterm"}],"workdir":"/"},"restartPolicy":"never","initialize":false}
Obviously none of these groups exist in the client container, so the lookup fails and the group startup fails.
Probably the way this needs to be handled is do the lookup, and if it fails, try a string to int conversion, and if that succeeds use the integer number without the lookup. If both succeed for some reason, then fail.
There's a secondary issue which is a failure in this part of the code does not cleanly message back to the runv containerd daemon, so docker gets stuck waiting for a container to start without failing.
hyperstart uses devtmpfs for containers' /dev, but devtmpfs is the singleton in the whole system,
it is shared with all the containers and the rest of the system. we can convert it to tmpfs as the same as libcontainer.
[HYPER INFO 1105 01:43:^@41 00456 pod.go] [:298] Process the Containers section in POD SPEC
[HYPER INFO 1105 01:43:^@41 00456 pod.go] [:301] trying to init container kube_5e212388-8385-11e5-bb0e-063553ee1923_influxdb-grafana-ex8d0_kube-system-service-discovery
[HYPER INFO 1105 01:43:^@41 00456 tags.go] [:133] LookupImage Name is haproxy:latest
[HYPER INFO 1105 01:43:^@41 00456 server.go] [:1052] Calling GET /list
[HYPER INFO 1105 01:43:^@41 00456 server.go] [:190] List type is pod, specified pod: [], list auxiliary pod:
[HYPER INFO 1105 01:43:^@41 00456 job.go] [:78] +job list(pod, , )
[HYPER INFO 1105 01:43:^@41 00456 container.go] [:10] ready to get the container(4a0753465d925c5237e21a53c05dce7d1dbf09b2dba9d56f63f4c3f3810337df) info
[HYPER INFO 1105 01:43:^@41 00456 pod.go] [:301] trying to init container kube_5e212388-8385-11e5-bb0e-063553ee1923_influxdb-grafana-ex8d0_kube-system_influxdb.cb712a1b_47d8cbdf
[HYPER INFO 1105 01:43:^@41 00456 tags.go] [:133] LookupImage Name is kubernetes/heapster_influxdb:v0.5
[HYPER ERROR 1105 01:43:^@41 00456 pod.go] [:315] mkdir /var/lib/hyper/overlay/6211ea67e7be674977fa0826145ab159ffa4f00c0bf7fb8c3fb365275403c030-init/merged/dev/shm: invalid argument
see https://www.togaware.com/linux/survivor/Standard_Groups.html
tty | 5 | The terminal devices with names beginning with /dev/tty are group accessible to group tty. Programs such as write and wall need access to /dev/tty and they set their group id (sgid) to tty. |
---|
Current streams are multiplexed and sent/received via the serial port (named: "sh.hyper.channel.1")
And the format for multiplexing is:
/*
* stream message format
* | stream sequence | length | payload (length-12) |
* | . . . . . . . . | . . . . | . . . . . . . . . . . . |
* 0 8 12 length
*/
#define STREAM_HEADER_SIZE 12
#define STREAM_HEADER_LENGTH_OFFSET 8
Both stream sequence and length are encoding in bigendian.
And length=0 indicates the command of close the stream (only one direction)
(and there is an additional ugly data is sent from hyperstart to runv for the exitcode,
this part of the protocol will be removed soon(scheduled after the big refactor(hyperhq/runv#295)))
The major problem is that payload will be discarded in hyperstart if the buffer if full,
and stream service is blocked in runv if the buffer if full.
We need change the protocol after the big refactor(hyperhq/runv#295) as:
num
bytes of data(ack for the earlier message). A should record how much the data that B had received. A shouldn't send any more data to B util A get ALL the ack.if the container creates the processes very frequent and quickly. the looping killer(#96) can not kill all the processes.
Is there any proposal to solve it? (it is not bug, it is user's responsibility if the container can't end, but it will be better if hyperstart can help).
(in my test, it loops util the test script stopped by OOM, if the script does fork processes carefully, the container will not end.)
see #96
I'll write some code to make hyperstart support such situation that
there are more than one container in the pod and I want to stop one of these containers and the pod is still running.
Step 6 : RUN apt-get update
---> Running in 33968afa2cb0
Ign http://archive.ubuntu.com trusty InRelease
Get:1 http://archive.ubuntu.com trusty-updates InRelease [65.9 kB]
E: Unable to determine file size for fd 7 - fstat (2: No such file or directory)
Let me describe the detail:
(1) init new container
|
\/
(2) fork() ----->(3b) fork again for namespace ----> (B) init stdio & exec process
|
\/
(3a) return success
|
\/
(A) next command (signal container with SIGKILL)
Some failed logs:
The (A)
was shown (signal and return normally) in log before (B)
(hyper_install_process_stdio
), and the container process was not killed.
SB[vm-QGyxKjgyXa] [CNL] hyper send type 8
SB[vm-QGyxKjgyXa] [CNL] prerequisite process pid 332
SB[vm-QGyxKjgyXa] [CNL] create child process pid=333 in the sandbox
SB[vm-QGyxKjgyXa] [CNL] hyper send type 333
SB[vm-QGyxKjgyXa] [CNL] hyper_init_event exec stdin event 0x867eb8, ops 0x61d5c0, fd 7
SB[vm-QGyxKjgyXa] [CNL] hyper_add_event add event fd 7, 0x61d5c0
SB[vm-QGyxKjgyXa] [CNL] hyper_init_event exec stdout event 0x867ef8, ops 0x61d5a0, fd 8
SB[vm-QGyxKjgyXa] [CNL] hyper_add_event add event fd 8, 0x61d5a0
SB[vm-QGyxKjgyXa] [CNL] hyper_init_event exec stderr event 0x867f38, ops 0x61d580, fd 11
SB[vm-QGyxKjgyXa] [CNL] hyper_add_event add event fd 11, 0x61d580
SB[vm-QGyxKjgyXa] [CNL] hyper_run_process process pid 333
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 9, len 0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLOUT, he 0x61d648, fd 3, 0x61d4c0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLIN, he 0x61d648, fd 3, 0x61d4c0
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_modify_event modify event fd 3, 0x61d648, event 8197
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlfd_read: get length 116
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlmsg_handle, type 24, len 116
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 9, len 0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLOUT, he 0x61d648, fd 3, 0x61d4c0
type 24
is SIGNALPROCESS
, and the command payload is
{"container":"d9f81367f4ad974fb2ba3f60f38917602d28914a15764f01c117be082aad9e39","process":"init","signal":9}
SB[vm-QGyxKjgyXa] [CNL] pid 332 exit normally, status 0
SB[vm-QGyxKjgyXa] [CNL] hyper_install_process_stdio
(more than 9 minutes later, timeout)
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_modify_event modify event fd 3, 0x61d648, event 8197
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlfd_read: get length 117
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 14, len 4
SB[vm-QGyxKjgyXa] [CNL] hyper_ctlmsg_handle, type 24, len 117
SB[vm-QGyxKjgyXa] [CNL] hyper ctl append type 9, len 0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLOUT, he 0x61d648, fd 3, 0x61d4c0
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLHUP or EPOLLERR, he 0x867f38, fd 11, 10
SB[vm-QGyxKjgyXa] [CNL] stderr_hup, seq 1, id init
SB[vm-QGyxKjgyXa] [CNL] still have 3 user of exec
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLHUP or EPOLLERR, he 0x867ef8, fd 8, 10
SB[vm-QGyxKjgyXa] [CNL] stdout_hup, seq 1, id init
SB[vm-QGyxKjgyXa] [CNL] still have 2 user of exec
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_event event EPOLLHUP or EPOLLERR, he 0x867eb8, fd 7, 8
SB[vm-QGyxKjgyXa] [CNL] stdin_hup, seq 1, id init
SB[vm-QGyxKjgyXa] [CNL] still have 1 user of exec
SB[vm-QGyxKjgyXa] [CNL] pid 333 exit by signal, status 15
SB[vm-QGyxKjgyXa] [CNL] hyper_handle_exec_exit exec exit pid 333, seq 1, container d9f81367f4ad974fb2ba3f60f38917602d28914a15764f01c117be082aad9e39
SB[vm-QGyxKjgyXa] [CNL] container init process 333
SB[vm-QGyxKjgyXa] [CNL] last user of exec exit, release
Failed: http://ci.hypercontainer.io:8080/job/hyperd-auto/288/consoleFull
Success: http://ci.hypercontainer.io:8080/job/hyperd-auto/295/consoleFull
and fix the access mode of the /etc/hostname.
(it is now 0755, but expected 0644)
clean up hyper_create() and hyper_create_file().
don't over mount on /etc/hostname when run container for docker, since docker provides and is keeping updating this file.
Problem1: how to distinguish whether /etc/hostname ?
Problem2: hyperstart should be also notified when hostname need to be updated. hyperstart can also update this /etc/hostname. (So, is this issue a real problem?)
otherwise container can access the vm-rootfs via /proc/1/root/ or /proc/1/cwd/
don't use the host /lib64/libc.so
the build system should ensure every one creates the initrd with the same libs.
hyperstart doesn't setup the /dev/net/tun interface inside containers, which means using tun devices does not work unless it is explicitely setup in the image:
mkdir /dev/net
mknod /dev/net/tun c 10 200
solves the problem, but it seems like hyperstart should probably handle this?
looks this is not the same with #74 , and I have already applied #86 and #87
[root@centos-rpm-build SPECS]# yum install strace
Loaded plugins: fastestmirror, ovl
base | 3.6 kB 00:00
extras | 3.4 kB 00:00
updates | 3.4 kB 00:00
Loading mirror speeds from cached hostfile
* base: mirror.stjschools.org
* extras: mirror.keystealth.org
* updates: mirror.hmc.edu
Resolving Dependencies
--> Running transaction check
---> Package strace.x86_64 0:4.8-11.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
strace x86_64 4.8-11.el7 base 265 k
Transaction Summary
================================================================================
Install 1 Package
Total download size: 265 k
Installed size: 950 k
Is this ok [y/d/N]: y
Downloading packages:
strace-4.8-11.el7.x86_64.rpm | 265 kB 00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
Installing : strace-4.8-11.el7.x86_64 1/1
Rpmdb checksum is invalid: dCDPT(pkg checksums): strace.x86_64 0:4.8-11.el7 - u
There is a warning: Warning: RPMDB altered outside of yum.
, and failed at last
During testing hyperhq/hyperd#585 (fix hyperhq/hyperd#539), hyperstart #290 failed when restart container, while the restart works with #287 .
#287 result:
➜ sudo ./hyperctl list container
Container ID Name POD ID Status
2c8abf13b126d71e8fdfdf21d117633178f6f2635f4d884f3179c3e7e306a54c busybox-9326359365 busybox-9326359365 succeeded
➜ sudo ./hyperctl start -c busybox-9326359365
Successfully started container busybox-9326359365
➜ sudo ./hyperctl attach busybox-9326359365
/ # ls
bin dev etc home lib proc root sys tmp usr var
/ # exit
➜
#290 result:
➜ sudo ./hyperctl list container
Container ID Name POD ID Status
3ccb3256b304f77602b159cdb75dd748e9675d8e2fc6526c94ba31296888fd01 busybox-7322297518 busybox-7322297518 succeeded
➜ sudo ./hyperctl start -c busybox-7322297518
./hyperctl ERROR: Error from daemon's response: Create new container failed: Error:
255 ➜
console logs:
I0417 23:24:00.703250 [CNL] hyper_handle_event event EPOLLIN, he 0x61b5a8, fd 3, 0x61b460
I0417 23:24:00.704053 [CNL] hyper ctl append type 14, len 4
I0417 23:24:00.705311 [CNL] hyper_modify_event modify event fd 3, 0x61b5a8, event 8197
I0417 23:24:00.706092 [CNL] hyper_ctlfd_read: get length 482
I0417 23:24:00.706898 [CNL] hyper ctl append type 14, len 4
I0417 23:24:00.707836 [CNL] hyper_ctlmsg_handle, type 17, len 482
I0417 23:24:00.716334 [CNL] call hyper_new_container, json {"id":"3ccb3256b304f77602b159cdb75dd748e9675d8e2fc6526c94ba31296888fd01","rootfs":"rootfs","image":"/3bb3a3600dd818aec2c01ba2ffc474442ab43c3c30fff69fb5e7a63cf73a4a3f","fsmap":[{"source":"JbzXCiRQfJ","path":"/etc/hosts","readOnly":false,"dockerVolume":false}],"process":{"id":"init","terminal":true,"stdio":2,"args":["sh"],"envs":[{"env":"PATH","value":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"}],"workdir":"/"},"restartPolicy":"never","initialize":true}, len 474
I0417 23:24:00.717478 [CNL] hyper send mntns referenced event: normal
I0417 23:24:00.718539 [CNL] create child process pid=338 in the sandbox
I0417 23:24:00.720520 [CNL] src directory /tmp/hyper/shared//3bb3a3600dd818aec2c01ba2ffc474442ab43c3c30fff69fb5e7a63cf73a4a3f/
I0417 23:24:00.723649 [CNL] root directory for container is /tmp/hyper/3ccb3256b304f77602b159cdb75dd748e9675d8e2fc6526c94ba31296888fd01/root//rootfs, init task sh
I0417 23:24:00.724620 [CNL] recreate file ./etc/hosts
I0417 23:24:00.725966 [CNL] recreate file ./etc/hostname
I0417 23:24:00.727686 [CNL] recreate symlink ./etc/mtab to /proc/mounts
I0417 23:24:00.729321 [CNL] container sets up init layer failed
I0417 23:24:00.730327 [CNL] hyper send container inited event: error
I0417 23:24:00.731311 [CNL] wait for setup container rootfs failed
I0417 23:24:00.733253 [CNL] create child process pid=340 in the sandbox
I0417 23:24:00.734420 [CNL] fail to enter container ns: Bad file descriptor
I0417 23:24:00.735414 [CNL] hyper send enter container ns event: error
I0417 23:24:00.736197 [CNL] hyper ctl append type 10, len 0
I0417 23:24:00.737698 [CNL] hyper_handle_event event EPOLLOUT, he 0x61b5a8, fd 3, 0x61b460
麻烦问一下,现在http://www.visualops.io/还维护么,注册总是返回401。 谢谢!
Test log at http://ci.hypercontainer.io:8080/job/hyperd-auto/436/consoleFull
hyperstart got HUP event resulting container to stop. The source of HUP event needs further investigation.
cannot find vsock device
scan /sys/class/virtio-ports/ failed: No such file or directory
cannot find ctl channel
fail to setup hyper serial channel
It'd be nice to document the protocol between the host and hyperstart for projects wanted to explore hyperstart. In particular:
Hello all,
I'm newbie on this project.
and I got below error on build time.
$ make
make all-recursive
make[1]: Entering directory '/home/keyolk/.gvm/pkgsets/go1.6.2/container/src/github.com/hyperhq/hyperstart'
Making all in src
make[2]: Entering directory '/home/keyolk/.gvm/pkgsets/go1.6.2/container/src/github.com/hyperhq/hyperstart/src'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/keyolk/.gvm/pkgsets/go1.6.2/container/src/github.com/hyperhq/hyperstart/src'
Making all in build
make: relocation error: libm.so.6: symbol __get_cpu_features, version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference
make[1]: *** [Makefile:352: all-recursive] Error 1
make[1]: Leaving directory '/home/keyolk/.gvm/pkgsets/go1.6.2/container/src/github.com/hyperhq/hyperstart'
make: *** [Makefile:293: all] Error 2
$ objdump -T /usr/lib/libc.so.6 | grep GLIBC_PRIVATE | grep cpu
00000000000f4300 g DF .text 0000000000000036 GLIBC_PRIVATE __clock_getcpuclockid
to solve this,
is there any something workaround ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.