Giter Club home page Giter Club logo

kvm's Introduction

Docker KVM simple container

Generic container for launching a Virtual Machine inside a Docker container.

Features:

  • Non libvirt dependant.
  • It uses QEMU/KVM to launch the VM directly with PID 1.
  • It attaches to the VM as many NICs as the docker container has.
  • The VM gets the original container IPs.
  • Uses macvtap tun devices for best network throughput.
  • Outputs serial console to stdio, thus visible using docker logs

Partially based on RancherVM project.

Running:

  • It is mandatory to define the AUTO_ATTACH variable:
    • If AUTO_ATTACH is set to yes, then all the container interfaces are attached to the VM. This is the typical use case.
    • If AUTO_ATTACH is set to no, a list of interfaces have to be declared in the ATTACH_IFACES variable. This is useful when launching the container with net=host flag, and only a subset of network interfaces need to be attached to the container.
  • The VM image needs to be located in /image/image (no extension)
  • Any additional parameters for QEMU/KVM can be specified as CMD argument when launching the container.
  • When launching the VM, its serial port is accesible through docker attach
$ docker run                                     \
      --name kvm                                 \
      -td                                        \
      --privileged                               \
      -v /path_to/image_file.qcow2:/image/image  \
      -e AUTO_ATTACH=yes                         \
      bbvainnotech/kvm:latest

Using more than one interface for the container (and the VM)

Before running the container, it is needed to create the networks:

$ docker network create --driver=bridge network1 --subnet=172.19.0.0/24
$ docker network create --driver=bridge network2 --subnet=172.19.2.0/24

Then, create the container and attach the network prior to start the container:

$ docker create                                 \
      --name container_name                     \
      -td                                       \
      --privileged                              \
      --network=network1                        \
      -v /path_to/image_file.qcow2:/image/image \
      -e AUTO_ATTACH=yes                        \
      bbvainnotech/kvm:latest

$ docker network connect network2 container_name
$ docker start container_name

Using the dockerhost interfaces

$ docker run                                    \
      --name container_name                     \
      -net=host                                 \
      -td                                       \
      --privileged                              \
      -v /path_to/image_file.qcow2:/image/image \
      -e AUTO_ATTACH=yes                        \
      bbvainnotech/kvm:latest

Debug mode

Passing bash keyword as argument to the container will launch a bash shell:

$ docker run                                    \
      -ti                                       \
      --privileged                              \
      -v /path_to/image_file.qcow2:/image/image \
      -e AUTO_ATTACH=yes                        \
      bbvainnotech/kvm:latest bash

Environment variables

SELECTED_NETWORK

If the container has more than one IP configured in a given interface, the user can select which one to use. The SELECTED_NETWORK environment variable is used to select that IP. This env variable must be in the form IP/MASK (e.g. 1.2.3.4/24). If this env variable is not set, the IP to be given to the VM is the first in the list for that interface (default behaviour).

This usecase is found when working with Kubernetes: Kubernetes assigns two IP addresses to the docker eth0 interface.

AUTO_ATTACH

When this env variable is set to yes, the entrypoint will scan all the vNICs present in the Docker container, and it will configure the hosted VM to get as many vNICs as the host container.

If this variable is set to no, only the interface names specified in the env variable $ATTACH_IFACES will be connected to the guest VM. Interfaces shall be separated by spaces (eg. ATTACH_IFACES='eth0 eth2').

If AUTO_ATTACH is set to no and no interfaces are defined, the VM will start with no NICs (and thus no vtap devices connected to container interfaces).

DNSMASQ_OPTS

This var controls the invocation parameters for dnsmasq daemon, used to give IP addresses to the VM. See dnsmasq's man page for info about available options.

It's specially useful the following options when debugging dnsmasq behaviour:

--log-facility=/var/log/dnsmasq.log --log-dhcp

DEBUG

When this env varable is set to yes, the verbosity is increased.

USE_NET_BRIDGES

This container uses macvlan devices to setup network connectivity. If an old kernel or limited host is used, it is possible to use linux bridge by setting the variable USE_NET_BRIDGES to yes.

Notes / Troubleshooting

  • Privileged mode (--privileged) is needed in order for KVM to access to macvtap devices see issue #3 for further information.

  • If you get the following error from KVM:

    qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: vhost-net requested but could not be initialized
    qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: Device 'tap' could not be initialized
    
    

    you will need to load the vhost-net kernel module in your dockerhost (as root) prior to launch this container:

    # modprobe vhost-net
    

    This is probed to be needed when using RancherOS.

License

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Authors

  • BBVA Innotech - Fernando Alvarez (@methadata)
  • BBVA Innotech - Pancho Horrillo (@panchoh)
  • BBVA Innotech - Rodrigo de la Fuente (@rodrigofuente)

kvm's People

Contributors

methadata avatar panchoh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kvm's Issues

Not able to add second network interface

Hi,

I have two docker networks created as explained in README. But when I follow the steps I don't see the second network getting added. It updates same eth0 with the last network connected. I have brwan1 and brlan1 networks created and I create container using below command

docker create
--name gw-vm
-it
--privileged
--network=network1
-v /images/gw-vm.qcow2:/image/image
-e AUTO_ATTACH=yes
bbvainnotech/kvm:latest

and then run docker network connect network2 gw-vm && docker start gw-vm. But when I docker attach gw-vm and check for ifconfig I only see eth0 and it updated with brlan1 ip range but not two nics added.

Can you please suggest what might be wrong here?

docker run simply hangs in this step char device redirected to /dev/pts/1 (label charserial0)

Firstly, thanks for this wonderful tool :)

When i run my qcow2 image on centos7 using this command

[root@localhost test]# docker run --name kvm --rm -it --privileged -v /var/test/myimage.qcow2:/image/image -e AUTO_ATTACH=yes -e DEBUG=y bbvainnotech/kvm:latest /bin/sh

The command simply hangs at this point

INFO: Launching qemu-kvm
DEBUG: Launching /usr/libexec/qemu-kvm -drive if=virtio,file=/image/image -nodefaults -device virtio-balloon-pci,id=balloon0 -realtime mlock=off -msg timestamp=on -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -serial stdio -enable-kvm -machine accel=kvm,usb=off -vga qxl -display none -m 1024 -smp 4,sockets=4,cores=1,threads=1 /bin/sh -device virtio-net-pci,netdev=net0,mac=02:42:ac:11:00:02 -netdev tap,id=net0,vhost=on,fd=3 3<>/dev/macvtap1c06f8
char device redirected to /dev/pts/1 (label charserial0)

Is there anything i need to set in my host to have this working?

Container VM Connection

I need the container have connection because I am trying to add the SMB feature of KVM to bypass docker volume to SMB Service inside the Guest VM, is a way to simulate the Docker volume inside the VM's

But It seems doesn't work because the container it's not visible from the VM Guest

Some guest OS don't get IP address

Some guest OS are not getting the IP address offered by dnsmasq in container service.
If IP address is configured manually, network connectivity works properly.

Guest OS tested:

Support non-hardware-accelerated Docker Hosts

CircleCI VMs do not have nested hardware virtualization support, so it's not possible to run a test that launches this container with a real VM image.

This arises the lack of support for non-hardware accelerated VMs when the docker hosts does not have the virtualization extensions enabled.

Manage more than one interface with ATTACH_IFACES

Could you add some examples with AUTO_ATTACH=no and ATTACH_IFACES?

I get mknod / macvtap0 errors

Command line is not complete. Try option "help"
Command line is not complete. Try option "help"
Cannot find device "name"
Cannot find device "macvtap0"
Cannot find device "name"
Cannot find device "macvlan0"
cat: /sys/class/net/macvtap0/address: No such file or directory
cat: /sys/devices/virtual/net/macvtap0/tap*/dev: No such file or directory
mknod: missing operand after 'c'
Special files require major and minor device numbers.
Try 'mknod --help' for more information.
/usr/local/bin/startvm: line 75: /8: syntax error: operand expected (error token is "/8")

Tested with different settings (host iface names, docker network names, AUTO_ATTACH=yes, ...

Attach VGA interface

To boot most images (most Linux distros, Windows...) it is needed to attach a VGA on boot.

Consider to render VGA output to serial port, or simply output serial port to stdout, if the guest vm has been configured to do so.

No internet connection inside guest

Hi,

my guest system is unable to connect to any host outside the given docker-network. I can connect to docker containers within the same networks, but even ping to 8.8.8.8 fails without response. I even can ping the gateway defined for the network with no problem. The other docker containers can also connect to my guest via it's docker ip. Have you got any idea?

My log output:

INFO: KVM acceleration enabled
DEBUG: Container original MAC address: 02:42:0a:fe:65:fa
++ local iface=eth0
++ local mode=macvlan
+++ generateNetdevNames macvlan
+++ devicetype=macvlan
+++ netdevinterfaces=($(ip link show | awk "/$devicetype/ { print $2 }" | cut -d '@' -f 1 | tr -d :))
++++ ip link show
++++ awk '/macvlan/ { print $2 }'
++++ cut -d @ -f 1

++++ tr -d :
+++ local netdevinterfaces
++++ cat /dev/urandom
++++ tr -dc a-f0-9
++++ fold -w 6
++++ head -n 1
+++ local randomID=81b16a
+++ containsElement macvlan81b16a
+++ local e

+++ return 1
+++ echo 81b16a
++ local deviceID=81b16a
++ local bridgeName=macvlan81b16a
++ [[ macvlan == \b\r\i\d\g\e ]]
++ vtapdev=macvtap81b16a
+++ ip link add link eth0 name macvtap81b16a type macvtap mode bridge
++ ip link set macvtap81b16a address 02:42:0a:fe:65:fa
++ ip link set macvtap81b16a up
++ ip link add link eth0 name macvlan81b16a type macvlan mode bridge
++ ip link set macvlan81b16a up
++ IFS=:
++ read major minor
+++ cat /sys/devices/virtual/net/macvtap81b16a/tap2/dev
++ mknod /dev/macvtap81b16a c 247 1

x
INFO: DHCP configured to serve IP 10.254.101.250/24 via macvlan81b16a (attached to container's eth0)
DEBUG: bridgeName: macvlan81b16a
INFO: Lauching dnsmasq
DEBUG: dnsmasq options: --dhcp-range=10.254.101.250,10.254.101.250 --dhcp-host=02:42:0a:fe:65:fa,,10.254.101.250,b74e57122277,infinite --dhcp-option=option:netmask,255.255.255.0 --dhcp-option=option:dns-server,127.0.0.11 --dhcp-option=option:router,10.254.101.254 --dhcp-option=option:domain-search, --dhcp-option=option:domain-name,
INFO: Launching qemu-kvm
DEBUG: Launching /usr/libexec/qemu-kvm -drive if=virtio,file=/image/image -nodefaults -device virtio-balloon-pci,id=balloon0 -realtime mlock=off -msg timestamp=on -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -serial stdio -enable-kvm -machine accel=kvm,usb=off -vga qxl -display none -m 1024 -smp 4,sockets=4,cores=1,threads=1 -vnc 10.254.250.71:1 -k de -device virtio-net-pci,netdev=net0,mac=02:42:0a:fe:65:fa -netdev tap,id=net0,vhost=on,fd=3 3<>/dev/macvtap81b16a
char device redirected to /dev/pts/0 (label charserial0)

IP config inside docker container:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: macvtap81b16a@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 500
link/ether 02:42:0a:fe:65:fa brd ff:ff:ff:ff:ff:ff
3: macvlan81b16a@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether be:39:f7:05:5f:74 brd ff:ff:ff:ff:ff:ff
inet 10.254.100.250/23 scope global macvlan81b16a
valid_lft forever preferred_lft forever
139: eth0@if140: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether fe:05:f4:4d:f0:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0
141: eth1@if142: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 02:42:0a:fe:fa:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.254.250.71/24 brd 10.254.250.255 scope global eth1
valid_lft forever preferred_lft forever

Remove launcher command line break?

It looks like network interface(s) will be ignored because of the line break in front of -m 1024?

/usr/libexec/qemu-kvm -enable-kvm   -drive file=/image/image.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=writethrough   -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-1,bootindex=1      -machine rhel6.0.0,accel=kvm,usb=off   -nodefaults   -no-acpi   -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2   -realtime mlock=off   -msg timestamp=on   -chardev pty,id=charserial0   -device isa-serial,chardev=charserial0,id=serial0   -serial stdio
   -m 1024 -smp 4,sockets=4,cores=1,threads=1   -device virtio-net-pci,netdev=net0,mac=da:ec:9a:19:71:64 -netdev tap,id=net0,vhost=on,fd=3 3<>/dev/macvtap0 -device virtio-net-pci,netdev=net1,mac=aa:7b:f2:12:f9:51 -netdev tap,id=net1,vhost=on,fd=4 4<>/dev/macvtap1 -device virtio-net-pci,netdev=net2,mac=4e:db:4c:e5:a9:35 -netdev tap,id=net2,vhost=on,fd=5 5<>/dev/macvtap2
: ${KVM_OPTS:="\
  -machine rhel6.0.0,accel=kvm,usb=off \
  -nodefaults \
  -no-acpi \
  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 \
  -realtime mlock=off \
  -msg timestamp=on \
  -chardev pty,id=charserial0 \
  -device isa-serial,chardev=charserial0,id=serial0 \
  -serial stdio
"}

I think it should be:

: ${KVM_OPTS:="\
  -machine rhel6.0.0,accel=kvm,usb=off \
  -nodefaults \
  -no-acpi \
  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 \
  -realtime mlock=off \
  -msg timestamp=on \
  -chardev pty,id=charserial0 \
  -device isa-serial,chardev=charserial0,id=serial0 \
  -serial stdio"}

Improve debugging

Enable a verbose mode that outputs debug information.

To do it, use a function in bash (called log?) instead of echo when printing to stdout.

Replacing virt-install with bare qemu-kvm

Is there a libvirtd supported container available ? If not, can someone please help me with the qemu-kvm equivalent command for the below virt-install command so that I can create automated installation of VM inside the KVM CentOS container ?

virt-install --location http://download.eng.bos.redhat.com/released/RHEL-7/7.6/Server/x86_64/os/
--disk /var/tmp/install$1.img,size=10,format=raw
--os-variant rhel7.6
--initrd-inject ./ks.cfg
--extra-args="ks=file:/ks.cfg"
--vcpus 2
--memory 2048
--noreboot
--name azure-install-$1
--channel tcp,host=127.0.0.1:6080,mode=connect,target_type=virtio,name=org.fedoraproject.anaconda.log.0

wrong IP?

this docker image is setting wrong IP to the docker macvlan

172.16.0.3/15 should be 172.17.0.3/16

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: macvtapa0761e@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff
inet6 fe80::42:acff:fe11:3/64 scope link
valid_lft forever preferred_lft forever
3: macvlana0761e@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1
link/ether ba:49:20:45:26:86 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.3/15 scope global macvlana0761e
valid_lft forever preferred_lft forever
inet6 fe80::b849:20ff:fe45:2686/64 scope link
valid_lft forever preferred_lft forever
3824: eth0@if3825: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether fe:05:93:f4:a6:e4 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::fc05:93ff:fef4:a6e4/64 scope link
valid_lft forever preferred_lft forever

Exposed ports?

Hi,

I tried to use it with exposed ports to reach the VM (-p 2222:22 -p 8080:8080 -p 5900:5900), but it doesn't work. So I can't reach / enter the VM.

Any hints how to reach the VM ports from outside?

Project alive?

The project was very active, but since 2nd Nov there is no progress / commit.

How to connect to qemu VNC without assigning network manually after container ran?

I can't connect to VNC if I try to start container with two networks(the default bridge and calico network), I can't connect with any IP(In my case the bridge IP is 172.17.0.2, calico network(net1) IP is 20.20.100.182)

#!/bin/bash

img=hd01.qcow2

docker create --rm --name kvmtest -e DEBUG=y -t --privileged -v /usr/hpc/vd/registered/$img:/image/image -v /opt/docker-winxp/mock.iso:/image/drv.iso -e AUTO_ATTACH=yes harbor.hpc.com/images/hpc/kvm

docker network connect net1 kvmtest
docker start -i kvmtest

But it works well if I connect one network after the container ran with command:

docker run --rm --name kvmtest -e DEBUG=y -td --privileged -v /usr/hpc/vd/registered/$img:/image/image -v /opt/docker-winxp/mock.iso:/image/drv.iso -e AUTO_ATTACH=yes harbor.hpc.com/images/hpc/kvm
# sleep to wait container running.
sleep 3 
docker network connect net1 kvmtest

Then I can connect VNC with 20.20.100.182
How can I get two networks that are started at the same time to work properly?

dnsmasq error with latest commit

After docker pull the current commit I get an error:

dnsmasq: bad dhcp-range at line 1 of /etc/dnsmasq.d/eth0.conf

cat /etc/dnsmasq.d/eth0.conf

dhcp-range=,
dhcp-host=5a:bf:ec:52:d1:cb,,,infinite
dhcp-option=option:netmask,255.255.0.0

"The container gets non-conflicting IPs"

The VM gets the original container IPs. The container gets non-conflicting IPs

You increase the subnet mask to have some additional non-conflicting IPs?
Could be conflicting with routed subnets (docker host or external)?

Error when building the container image using CircleCI

CircleCI build fails when launching container image creation in the dependencies section:

docker build --rm=false -t bbvainnotech/kvm .

According to this bug report from CircleCI , it seems to be related to the internal VM that CircleCI uses for building docker images.

Next steps:

  • Track the issue from CircleCI
  • Try to build the container in another branch without doing a yum update

Allow to run more image types

Currently only qcow2 images are allowed, because qcow2 format is hardcoded in startvm script:

: ${KVM_BLK_OPTS:="\
  -drive file=/image/image.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=writethrough \
  -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-1,bootindex=1 \
  "}

Using the following image invocation shall be enough to allow kvm to launch any disk type:

: ${KVM_BLK_OPTS:="\
  -drive if=virtio,file=/image/image \
"}

Before pushing to master test it with VxWorks as well as Linux images to not loose backwards compatibility!

VM not working

I used the following command:

docker run -p 3391:3389 -td --name kvm34 --privileged -v /var/lib/libvirt/images/image.qcow2:/image/image.qcow2 -e AUTO_ATTACH=y bbvainnotech/kvm:latest

image

I have the following answer:

image

Error while running under kubernetes

RTNETLINK answers: File exists
Cannot find device "macvtapa85bcf"
Cannot find device "macvtapa85bcf"
cat: /sys/devices/virtual/net/macvtapa85bcf/tap*/dev: No such file or directory
mknod: missing operand after 'c'
Special files require major and minor device numbers.
Try 'mknod --help' for more information.
INFO: DHCP configured to serve IP 192.168.75.19/32 via macvtapa85bcf (attached to eth0)
INFO: Lauching dnsmasq ......
INFO: Launching /usr/libexec/qemu-kvm .......
2016-11-30T17:38:19.804224Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: TUNGETIFF ioctl() failed: Inappropriate ioctl for device
TUNSETOFFLOAD ioctl() failed: Inappropriate ioctl for device

bridge-utils not available in centos:latest

Hey!

Bridge-utils is not available anymore, since it got replaced by the buildin ip bridge.
Because the yum package does not exist anymore, docker-build will fail:

Last metadata expiration check: 0:00:06 ago on Sat Mar 27 09:57:24 2021.
No match for argument: bridge-utils
Package iproute-5.3.0-5.el8.x86_64 is already installed.
Error: Unable to find a match: bridge-utils
The command '/bin/sh -c yum clean all     && yum update -y     && yum install -y qemu-kvm bridge-utils iproute dnsmasq     && yum clean all' returned a non-zero code: 1

Rename kvm-macvlanN with random id

We have discovered that when launching a second KVM container, the macvtap/macvlan devices creation fails, because container says that

RTNETLINK answers: File exists
Cannot find device "kvm-macvtap0"
cat: /sys/class/net/kvm-macvtap0/address: No such file or directory
cat: /sys/devices/virtual/net/kvm-macvtap0/tap*/dev: No such file or directory  

Although this is something that needs further investigation (why does a vNIC device in a container affects to a different one in another container?), the proposed workaround is to ad a random id to the vNIC name, so it will be named like kvm-macvtapf50d

Run without privileged container

--privileged´is needed because of /dev/macvtap0?
RancherVM works fine without it and doesn't need privileged containers.

I read something about performance reasons? Is it really needed or could it changed to run without --privileged?

Running on kubernetes

I am trying to run my qcow2 via bbva on Kubernetes. What do you suggest or tested for multiple network scenarios on Kubernetes. We tried with CNI-Genie, but VM doesn't get second network although we can see multiple networks on container.

Do not modify the qcow-image on runtime?

At the moment the changes written to the qcow image directly I think? Because delete and recreate the container keeps all the changes.

A option to keep the qcow (base) image unchanged would be great, but save the changes to be container restart save?

Move image to alpine?

Could be easy to do and reduce the image size?

Tested qemu kvm with a simple alpine container

docker run --rm -ti --name kvm --cap-add NET_ADMIN -v /path-to-my-qcow2.img:/image/image.qcow2 --device /dev/kvm:/dev/kvm alpine sh
apk -U add qemu-system-x86_64 qemu bridge-utils dnsmasq

qemu-system-x86_64 -enable-kvm -cpu host -m 1024 -curses -drive file=/image/image.qcow2,format=qcow2,cache=none -usb -usbdevice tablet

Clean macvtap0 / macvlan0 after container is removed?

Is it possible to remove interfaces after the container is stopped and removed?

Tried to attach host interfaces:

docker run -p 5900:5900 -p 2222:22 -p 4444:4444 -td --name kvm --privileged -v /home/rancher/image.qcow2:/image/image.qcow2 -e AUTO_ATTACH=n -e ATTACH_IFACES=eth1 --net=host bbvainnotech/kvm:latest

The created interfaces are persistent after container is removed. Will this cause problems with new created KVM containers?

23: macvtap0@eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state LOWERLAYERDOWN group default qlen 500
    link/ether d6:e7:a3:65:21:04 brd ff:ff:ff:ff:ff:ff
24: macvlan0@eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1
    link/ether aa:16:25:51:22:11 brd ff:ff:ff:ff:ff:ff

RancherOS: tap could not be initialized

Try to start the container with (AUTO_ATTACH=y, working with container interfaces):

docker run -p 5900:5900 -p 2222:22 -p 4444:4444 -td --name kvm --privileged -v /home/rancher/image.qcow2:/image/image.qcow2 -e AUTO_ATTACH=y bbvainnotech/kvm:latest

docker logs kvm:

Lauching dnsmasq                                    --dhcp-option=option:dns-server,8.8.8.8,8.8.4.4                  --dhcp-option=option:router,172.17.0.1                    

dnsmasq: bad dhcp-range at line 1 of /etc/dnsmasq.d/eth0.conf
Launching /usr/libexec/qemu-kvm -enable-kvm   -drive file=/image/image.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=writethrough   -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-1,bootindex=1      -machine rhel6.0.0,accel=kvm,usb=off   -nodefaults   -no-acpi   -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2   -realtime mlock=off   -msg timestamp=on   -chardev pty,id=charserial0   -device isa-serial,chardev=charserial0,id=serial0   -serial stdio    -m 1024 -smp 4,sockets=4,cores=1,threads=1    -device virtio-net-pci,netdev=net0,mac=16:a5:26:d0:e8:43 -netdev tap,id=net0,vhost=on,fd=3 3<>/dev/macvtap0
char device redirected to /dev/pts/0 (label charserial0)
2016-10-05T08:19:47.042107Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: vhost-net requested but could not be initialized
2016-10-05T08:19:47.043419Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: Device 'tap' could not be initialized

Tried additional options alone and also combined like that:

docker run -p 5900:5900 -p 2222:22 -p 4444:4444 -td --name kvm --privileged -v /home/rancher/image.qcow2:/image/image.qcow2 -e AUTO_ATTACH=y -v /lib/modules:/lib/modules -v /lib64/modules:/lib64/modules -v /dev:/dev -v /sys:/sys bbvainnotech/kvm:latest

But tap problem still there:

2016-10-05T08:19:47.042107Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: vhost-net requested but could not be initialized
2016-10-05T08:19:47.043419Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: Device 'tap' could not be initialized

I don't know how I solved this yesterday with many different attempts...

So I added bash as CMD and manually executed /usr/local/bin/startvm.

cat: /sys/class/net/macvtap0/address: No such file or directory
cat: /sys/devices/virtual/net/macvtap0/tap*/dev: No such file or directory
mknod: missing operand after 'c'
Special files require major and minor device numbers.
Try 'mknod --help' for more information.
Lauching dnsmasq                                    --dhcp-option=option:dns-server,8.8.8.8,8.8.4.4                  --dhcp-option=option:router,172.17.0.1                    

dnsmasq: bad dhcp-range at line 1 of /etc/dnsmasq.d/eth0.conf
Launching /usr/libexec/qemu-kvm -enable-kvm   -drive file=/image/image.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=writethrough   -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-1,bootindex=1      -machine rhel6.0.0,accel=kvm,usb=off   -nodefaults   -no-acpi   -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2   -realtime mlock=off   -msg timestamp=on   -chardev pty,id=charserial0   -device isa-serial,chardev=charserial0,id=serial0   -serial stdio    -m 1024 -smp 4,sockets=4,cores=1,threads=1    -device virtio-net-pci,netdev=net0,mac= -netdev tap,id=net0,vhost=on,fd=3 3<>/dev/macvtap0
char device redirected to /dev/pts/6 (label charserial0)
2016-10-05T08:32:10.319878Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: TUNGETIFF ioctl() failed: Inappropriate ioctl for device
TUNSETOFFLOAD ioctl() failed: Inappropriate ioctl for device
2016-10-05T08:32:10.321238Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: vhost-net requested but could not be initialized
2016-10-05T08:32:10.321937Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: Device 'tap' could not be initialized

Now it tried it with --net=host to attach just the host eth1 to the VM.
docker run -p 5900:5900 -p 2222:22 -p 4444:4444 -td --name kvm --privileged -v /home/rancher/image.qcow2:/image/image.qcow2 -e AUTO_ATTACH=n -e ATTACH_IFACES=eth1 --net=host bbvainnotech/kvm:latest bash

Entered the container with docker exec -ti kvm bash and launched the startvm script:

/usr/local/bin/startvm: line 75: /8: syntax error: operand expected (error token is "/8")
Lauching dnsmasq                                    --dhcp-option=option:dns-server,8.8.8.8,8.8.4.4                  --dhcp-option=option:router,192.168.200.1                     --dhcp-option=option:domain-name,fam-hoeg.de

dnsmasq: bad dhcp-range at line 1 of /etc/dnsmasq.d/eth1.conf
Launching /usr/libexec/qemu-kvm -enable-kvm   -drive file=/image/image.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=writethrough   -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-1,bootindex=1      -machine rhel6.0.0,accel=kvm,usb=off   -nodefaults   -no-acpi   -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2   -realtime mlock=off   -msg timestamp=on   -chardev pty,id=charserial0   -device isa-serial,chardev=charserial0,id=serial0   -serial stdio    -m 1024 -smp 4,sockets=4,cores=1,threads=1    -device virtio-net-pci,netdev=net0,mac=76:20:fd:95:1e:b8 -netdev tap,id=net0,vhost=on,fd=3 3<>/dev/macvtap0
char device redirected to /dev/pts/0 (label charserial0)
2016-10-05T08:35:38.277312Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: vhost-net requested but could not be initialized
2016-10-05T08:35:38.277966Z qemu-kvm: -netdev tap,id=net0,vhost=on,fd=3: Device 'tap' could not be initialized

So what's missing here? Any kernel module to load? Tested it with macvtap un-/loaded. Both with the same result...

Reverse Network Interface Assignment

Following the example at https://hub.docker.com/r/bbvainnotech/kvm/ when I connect my container to more than one networks, the network assignment inside the container is random. What I expect as eth0 becomes eth1 and vice-versa.

QEMU literally takes the first interface as one that is passed first on the command line, the networking of my VM is revered from what I expect.

What would be an idea way to fix this ? Is just "sorting" based on interface number sufficient ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.