In qemu options, there is -pidfile where it saves its pid of main qemu process.

Sometimes we need this pid, and applying cgroups or prlimits for resources limitations.

I would like to add an option to Config struct so that user can configure it before LaunchCustomQemu()~

I was curious what blocks this package from compiling, on Windows.

$ go build -i -v -x
WORK=C:\Users\Worker\AppData\Local\Temp\go-build762776886
/C/Users/Worker/Desktop/govmm/qemu
mkdir -p $WORK\b001
cat >$WORK\b001\importcfg << 'EOF' # internal

import config

packagefile bufio=c:\go\pkg\windows_amd64\bufio.a
packagefile bytes=c:\go\pkg\windows_amd64\bytes.a
packagefile container/list=c:\go\pkg\windows_amd64\container\list.a
packagefile context=c:\go\pkg\windows_amd64\context.a
packagefile encoding/json=c:\go\pkg\windows_amd64\encoding\json.a
packagefile errors=c:\go\pkg\windows_amd64\errors.a
packagefile fmt=c:\go\pkg\windows_amd64\fmt.a
packagefile io=c:\go\pkg\windows_amd64\io.a
packagefile io/ioutil=c:\go\pkg\windows_amd64\io\ioutil.a
packagefile log=c:\go\pkg\windows_amd64\log.a
packagefile net=c:\go\pkg\windows_amd64\net.a
packagefile os=c:\go\pkg\windows_amd64\os.a
packagefile os/exec=c:\go\pkg\windows_amd64\os\exec.a
packagefile path=c:\go\pkg\windows_amd64\path.a
packagefile runtime=c:\go\pkg\windows_amd64\runtime.a
packagefile strconv=c:\go\pkg\windows_amd64\strconv.a
packagefile strings=c:\go\pkg\windows_amd64\strings.a
packagefile syscall=c:\go\pkg\windows_amd64\syscall.a
packagefile time=c:\go\pkg\windows_amd64\time.a
EOF
cd C:\Users\Worker\Desktop\govmm\qemu
"c:\go\pkg\tool\windows_amd64\compile.exe" -o "$WORK\b001\pkg.a" -trimpath "$WORK\b001=>" -p /C/Users/Worker/Desktop/govmm/qemu -complete -buildid dOlQOtmVqMsEWPaTfptx/dOlQOtmVqMsEWPaTfptx -goversion go1.15.2 -D /C/Users/Worker/Desktop/govmm/qemu -importcfg "$WORK\b001\importcfg" -pack -c=4 "C:\Users\Worker\Desktop\govmm\qemu\image.go" "C:\Users\Worker\Desktop\govmm\qemu\qemu.go" "C:\Users\Worker\Desktop\govmm\qemu\qmp.go"

/C/Users/Worker/Desktop/govmm/qemu

.\qmp.go:1517:9: undefined: syscall.UnixRights

So my environment is like follows:

$ go version
go version go1.15.2 windows/amd64

$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Worker\AppData\Local\go-build
set GOENV=C:\Users\Worker\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\Worker\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Worker\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Worker\AppData\Local\Temp\go-build111068182=/tmp/go-build -gno-record-gcc-switches

$ bash --version
bash --version
GNU bash, version 4.4.23(1)-release (x86_64-pc-msys)

Govmm provides some proper knobs to use disable-modern on virtio-pci devices that are coldplugged, but didn't do the same for QMP.
We need to make sure that hotplug works great when being run in nested environment that requires virtio 0.9 instead of 1.0, hence we need to provide the caller to enable or disable disable-modern.

Description of problem

ExecuteBlockdevAdd and ExecuteBlockdevAddWithCache both appear to be intended to create block devices in the guest which backend onto a block device in the host. That seems to be the way that Kata always uses it. However blockdevAddBaseArgs, used by both those functions always uses the file driver, which is only intended for use with regular file backends.

Use of the file driver for host block devices was deprecated in qemu-3.0, and has been removed entirely in qemu-6.0 (commit 8d17adf34f5). We should be using the host_device driver instead.

Quite a few open source projects have govmm as a dependency. It makes sense to include support for mainframes here to allow the users to have cross platform support

Secure Execution, also known as Protected Virtualization in QEMU, is a confidential computing technology for s390x (IBM Z & LinuxONE). Let's enable it in GoVMM! PR is prepared.

I wonder if vfio device can add x-pci-vendor-id and x-pci-device-id parameter pass to qemu.
Since some vendor id like 1ded can not be identified by virtio-pci driver, so I have to determine a specified vendor id pass to qemu

Can you please provide usage with simple example in golang.

Reference: kata-containers/runtime#1155

Problem description see kata-containers/runtime#1363

The easy to spot actions that should be changed are:

• govmm should require 2 reviews for merging
• Kata Containers CI must be integrated to this repo
  • This one includes:
    • travis
    • jenkins
    • labels checks
    • DCO check
• labels should be added to the repo

In order to support "sandbox" feature on qemu, I need to update the govmm code.

Param "-sandbox on" in qemu can introduce another protect layer on the host, to make the secure container more secure.

The default option is disable because this feature may introduce some performance cost, even though user can enable /proc/sys/net/core/bpf_jit_enable to reduce the impact.

Please refer to kata-containers/kata-containers#2266 for more info.

Some programs using this vendor may have their own children reapers which has reaped the
qemu's process before cmd.Wait to reap it. For this case, the cmd.Run/cmd.Wait
will get an error contains substring of "no child processes", thus it's needed
to ignore this error and deal it as success.

https://godoc.org/github.com/intel/govmm/qemu#LaunchCustomQemu says:

The function will block until the launched qemu process exits.

This is a bit misleading, because it sounds like one has to call the function in goroutine to get continue working with the QEMU instance while it is running. But that's not true, because the function will return once QEMU is done with the virtual machine startup thanks to --daemonize.

Which feature do you think can be improved?
virtio-mem-pci devices can be hotplugged only on the root bus.

How can it be improved?
Devices can't be hotplugged into the root bus of PCIe machines, eg Q35.
That means the feature can't be used on q35 machines.

Additional Information
Extend the API to optionally allow hotplugging memory devices to PCI bridges.

I'm working on the s390x support for kata and govmm. This issue tracks the modifications of the govmm code in order to be able to gradually add s390x support. These limitations or differences need to be taken into account:

• qemu on s390 uses ccw devices and not pci
• no vhost-user device support. See kata-containers/runtime#659
• the mq flag has no vectors flag. See commet in libvirt
• disable-modern option is not present for ccw devices
• romfile option not present for ccw devices

There are two projects that make use of govmm:

• https://github.com/kata-containers
• https://github.com/ciao-project/ciao

However, Ciao is no longer being maintained (it's archived), hence we can now move govmm into the Kata Containers organisation!

Why?

The Kata Containers GitHub organisation is the logical "parent" for govmm now that there is only a single user of govmm; it makes the code and PRs more discoverable. It also makes contributions simpler.

Notes

• govmm is used by both the 1.x Kata runtime and the new 2.0 runtime.

• govmm and Kata Containers both use the same Apache-2.0 licence.

• Moving this repo from https://github.com/intel/govmm to https://github.com/kata-containers/govmm can be done immediately by changing a setting in the govmm repo.

• After the move, https://github.com/intel/govmm will automatically redirect to https://github.com/kata-containers/govmm.

  This means nothing will "break" - URLs can be updated in a "lazy" fashion.

in dimm support check func, arm64 is excluded. but arm64 does support memory dimm in qemu.

isDimmSupported(config *Config) bool { switch runtime.GOARCH { case "amd64", "386", "ppc64le": if config != nil && config.Machine.Type == MachineTypeMicrovm { // microvm does not support NUMA return false }

For many usecases such as using userspace drivers (e.g: DPDK) to drive devices through VFIO, the presence of a virtual IOMMU device is needed.

Adding this issue to track the work needed to add such support.

check socket, thread and die, see #101 (comment)

travis CI is failing with the following error:

Bad response status from coveralls: 422
{"message":"service_job_id (717167073) must be unique for Travis Jobs not supplying a Coveralls Repo Token","error":true}
The command "$GOPATH/bin/goveralls -v -service=travis-ci" exited with 1.

Here I need some help to enable "-pflash" in govmm. thanks!

In some arches like arm64, "-pflash" is needed as UEFI volume. Specially, two pflash volumes are used in arm64, of which one is static for the UEFI firmware and eother one to store variables. So it's better to define pflash object as []string in qemu.go. Accordingly, a helper to append pflash to "QemuParams" is also needed.

@jodh-intel @devimc @amshinde

In order to support more device configurations it would make sense to expose all the available device driver options either by adding them to the existing device structs or by providing a generic way to pass arbitrary device driver options.

Description of problem

MacVTap seems not set up properly.

When I create a NET device of type MacVtap in this way:

	return qemu.NetDevice{
		Type:       qemu.MACVTAP,
		ID:         myobj.TAPIface.ID,
		Driver:     qemu.VirtioNetPCI,
		IFName:     myobj.TAPIface.Name,
		MACAddress: myobj.MacAddr,
		FDs:        []*os.File{myobj.TAPIface.VMFds},

		Script:     "no",
		DownScript: "no",
	}

I get the following output:

-netdev tap,id=macvtap4,fds=3 -device driver=virtio-net-pci,netdev=macvtap4,mac=ce:ca:44:23:96:5f,disable-modern=false,mq=on,vectors=4,romfile=

Expected result

Shouldn't I get also in the line the location of the tap device like 3<>/dev/tap5? in this way seems the macvtap does not work

Actual result

(replace this text with an explanation of what actually happened)

Further information

(replace this text with any extra information you think might be useful)

should clarify https://github.com/intel/govmm/pull/24/files#diff-3f8ad3b51588f02283eda2d19c5ffd06R741

I assume that the default queues will be one if an argument is not specified to qemu. We should verify this and add a clarifying comment to the code, as it isn't clear in the qemu documentation.

@caoruidong can you please drive this?

iommu_platform is required for Secure Execution, but for some devices, GoVMM will omit it (my bad) or put an extra comma. PR is prepared.

If I would like to pass the flag -serial stdio, would be possible to do that using the Config object or would I need to use LaunchCustomQemu func?

I have a distro Linux image that in order to be run with QEMU requires -nographic flag. If I use it together with daemonize, QEMU does not start and errors out saying that both flags cannot be used.

The problem is that by removing daemonize, the cmd.Run() hangs. I know that actually it does not hangs and it runs QEMU but it does not display anything because by design there is nothing attached to the cmd standard output and standard error.

I've tried to run it only with daemonize but I got the error:

ERRO[0098] Unable to launch /usr/bin/qemu-system-x86_64: exit status 1
ERRO[0098] Unable to init server: Could not connect: Connection refused
gtk initialization failed

I've also tried to pass -vga none but it did not work.

When using an architecture/setup that does not have DIMM support, memory backends are ignored. Such setups can actually use memory backends provided the memory backend is specified as a -machine parameter (instead of a -numa option). This is required for enabling virtio-fs support for s390x in Kata. I have prepared a PR.

To hotplug IBM Adjunct Processor (AP) devices into a VM, functionality besides the existing ExecutePCIVFIODeviceAdd is required, since the device_add command is a little different. See also Issue #491 in the Kata Containers project.

I have prepared a patch for this which I intend to create a PR for.

/cc @alicefr

https://travis-ci.org/kata-containers/runtime/jobs/461613820

• architecture:
	Host: ppc64le
	golang: ppc64le
	Build: ppc64le
• golang:
	go version go1.10.5 linux/ppc64le
• Summary:
	destination install path (DESTDIR)    : /
	binary installation path (BINDIR)     : /usr/local/bin
	binaries to install                   :
	 - /usr/local/bin/kata-runtime
	 - /usr/local/bin/containerd-shim-kata-v2
	 - /usr/libexec/kata-containers/kata-netmon
	 - /usr/local/bin/data/kata-collect-data.sh
	config to install (CONFIG)            : cli/config/configuration.toml
	install path (CONFIG_PATH)            : /usr/share/defaults/kata-containers/configuration.toml
	alternate config path (SYSCONFIG)     : /etc/kata-containers/configuration.toml
	hypervisor path (QEMUPATH)            : /usr/bin/qemu-system-ppc64le
	assets path (PKGDATADIR)              : /usr/share/kata-containers
	proxy+shim path (PKGLIBEXECDIR)       : /usr/libexec/kata-containers
     BUILD   /home/travis/gopath/src/github.com/kata-containers/runtime/kata-runtime
# github.com/kata-containers/runtime/vendor/github.com/intel/govmm/qemu
../vendor/github.com/intel/govmm/qemu/qemu.go:103:6: undefined: isVirtioPCI
../vendor/github.com/intel/govmm/qemu/qemu.go:266:5: undefined: isVirtioPCI
../vendor/github.com/intel/govmm/qemu/qemu.go:358:5: undefined: isVirtioPCI
../vendor/github.com/intel/govmm/qemu/qemu.go:469:5: undefined: isVirtioPCI
../vendor/github.com/intel/govmm/qemu/qemu.go:489:16: netdev.Type.QemuDeviceParam undefined (type NetDeviceType has no field or method QemuDeviceParam)
../vendor/github.com/intel/govmm/qemu/qemu.go:493:74: netdev.Type.QemuDeviceParam undefined (type NetDeviceType has no field or method QemuDeviceParam)
../vendor/github.com/intel/govmm/qemu/qemu.go:516:5: undefined: isVirtioPCI
../vendor/github.com/intel/govmm/qemu/qemu.go:527:16: netdev.Type.QemuNetdevParam undefined (type NetDeviceType has no field or method QemuNetdevParam)
../vendor/github.com/intel/govmm/qemu/qemu.go:531:49: netdev.Type.QemuNetdevParam undefined (type NetDeviceType has no field or method QemuNetdevParam)
../vendor/github.com/intel/govmm/qemu/qemu.go:579:16: netdev.Type.QemuNetdevParam undefined (type NetDeviceType has no field or method QemuNetdevParam)
../vendor/github.com/intel/govmm/qemu/qemu.go:579:16: too many errors
Makefile:342: recipe for target '/home/travis/gopath/src/github.com/kata-containers/runtime/kata-runtime' failed
make: *** [/home/travis/gopath/src/github.com/kata-containers/runtime/kata-runtime] Error 2
The command "make" failed and exited with 2 during .
Your build has been stopped.

Looking over this file (qemu/qemu.go, editorial note), there are a lot of functions (88, 18 of which are QemuConfig() ones ;) and each seems to handle building up these params differently (either appending with a comma, then joining with a null string; or appending without a separator, then joining with a comma).

Originally posted by @jodh-intel in #179 (review)

There is a max_ports option for virtio serial device in qemu. Qemu enables ioeventfd by default for virtio-serial-pci, if max_ports is 31 by default, virtio-serial-bus will add 2+2+30*2=64 virtqueues.
It will cost a lot of time overhead during boot time. It will be better that govmm provides max_ports for user to tun their qemu usecase

Add support for the share-rw flag for virtio-blk.

Currently only supports TAP. Need to make it configurable so we can also support macvtap.

In order to be able to hotplug and use the virtio-blk-ccw for kata. Each device needs to be identified by a device number. In this way, it's possible to determine which device number can be assigned to the hotplugged virtio-blk-ccw. This PR extends govmm to ba able to assign a devno to each device and it adds the virtio-blk-ccw device.

Main track issue for kata is kata-containers/runtime#1153

The output govmm when utilized with Kata is pretty verbose in some often-called-functions.

Let's remove the info print @ https://github.com/kata-containers/govmm/blob/master/qemu/qmp.go#L270-L272

Current versions that travis tests are 1.8, 1.9 and tip. Should consider adding more recent versions (and dropping old ones?)

Travis CI is ridiculously slow, it takes too much time to run and sometimes it doesn't report the status, it gets stuck, see comment #134 (comment)
kata containers uses github actions to run some tests, they look stable and the CI run pretty fast, unfortunately only x86 is supported:
I propose to move x86 CI jobs to github actions and keep s390 jobs in travis

This PR broke s390x because they insert numa and memory-backend-ram as default behvior. However, this combination is not supported on s390x.

The ContextID of VSOCK should be a uint64 not a uint32. That provokes an endianess issue in the kata-runtime. See kata-containers/runtime#947. Qemu requests an uint64:

$ qemu-system-x86_64 -device vhost-vsock-pci,help
[...]
vhost-vsock-pci.guest-cid=uint64

VirtioBlock device driver should be "virtio-blk-pci" instead of "virtio-blk".

VhostUserDevices have no CCW device numbers

QEMU device structs (e.g. FSDevice) carry a CCW device number. (CCW is a bus used on s390x/IBM Z.)
This is missing for VhostUserDevices. PR is prepared.

Enabling the support for Protected Execution Facility(PEF) which is he confidential computing technology on ppc64le.

NUMA support is available for Power (ppc64le) architecture. Since this is disabled in govmm, it prevents using virtiofs with Kata for ppc64le architecture.

I'm raising this issue to track the changes for enabling NUMA for ppc64le architecture

Which feature do you think can be improved?

We should be able to Hotplug a block device to QEMU as read-only. The existing ExecuteBlockdevAdd doesn't allow for specifying this option.

How can it be improved?

Add ability to specify the access permissions associated with a block device which is being added.

I'm implementing a https://godoc.org/github.com/intel/govmm/qemu#QMPLog logger. The API description mentions a log level, but it remains unclear what the values are that QMP uses internally.

Phrased differently, if I have 5 as my programs log level and compare against that, will QMP be silent, verbose, or log in excruciating detail everything that it does?

govmm is now part of the kata-containers GitHub organisation, so update to reflect this.

See also: #141.

Add pvpanic can let QEMU issue an GUEST_PANICKED event to outside. And "dump-guest-memory" can get the guest memory dump if wanted.

All these two will help to debug/manage QEMU VMs.

QEMU removed the -realtime option, instead, -overcommit mem-lock=on|off should be used. This is a trivial change to make in GoVMM, but the virtcontainers API 1.0 has the Realtime bool.
Does this mean we need to update the virtcontainers API?