digitalocean / go-libvirt Goto Github PK

Hi,

I have some goroutines:

• One for listening and processing domain lifecycle events. Call DomainGetState for each domain from event;
• Multiple for handling start/stop VM's and wait until state become to required sequentially call DomainGetState;
• One for collecting and processing metrics for every domains. Use DomainBlockStats, DomainInterfaceStats and etc. function for collecting metrics.

I faced with an next issue: sometimes, when I request to start or stop multiple domain at once, all those goroutines is stuck. I try to dig into this issue and find out that all goroutines is blocked on digitalocean/go-libvirt.Libvirt.getResponse() because cannot read message from channel. Unfortunately this is a floating bug and I cannot find how to reproduce it.

For now I use this workaround:

const getResponseTimeout = 5 * time.Second

func (l *Libvirt) getResponse(c chan response) (response, error) {
	timer := time.NewTimer(getResponseTimeout)
	defer timer.Stop()

	var resp response
	select {
	case <-timer.C:
		return response{}, fmt.Errorf("cannot get response from libvirt within %s", getResponseTimeout)
	case resp = <-c:
	}
	if resp.Status == StatusError {
		return resp, decodeError(resp.Payload)
	}

	return resp, nil
}

Anyone have some thoughts why this happens?

Will anyone be allowed to do PR to this repo? We have used libvirt with rpc for a while (better than creating and using binding) but in a different language (Rust) but thought about bringing it to Go and then I saw this and thought it would be cool to just contribute instead of doing the same thing

The documentation for the Connect() method says:

// Connect establishes communication with the libvirt server.
// The underlying libvirt socket connection will be created via the dialer.
// Since the connection can be lost, the Disconnected function can be used
// to monitor for a lost connection.

There is however no such method on libvirt.Libvirt. There is a method on socket.Socket, but the socket used is not exposed by libvirt.Libvirt.

I see that in the rpc package, there is a private libvirtError struct, which contains the Code of the error. I would want to access it, to distinguish connection problems from errors like "domain does not exist". I find it in combination with helper functions like e.g. IsNotFound(err error) bool pretty helpful.

Are there some ideas floating around on how proper error handling is done? Would it make sense to make LibvirtError public and add the error code constants to the project?

How to calculate cpu usage by the following two interfaces:
DomainGetCPUStats()
DomainGetInfo()

It is found that the problematic virtual machine monitoring data is stuck, and the response is not obtained for a long time.

log: Guest agent is not responding.

Like command:
virsh domstats domainName

From remote_protocol.x
REMOTE_PROC_CONNECT_GET_ALL_DOMAIN_STATS = 344

Currently there is no way to use these bindings with a libvirtd that is configured to use the polkit authentication method.

To fix this issue, a simple call to AuthPolkit() before opening the connection should be enough

Since #140 was implemented and New() deprecated, how is one supposed to do a qemu+ssh connection?

I was under the impression this should work:

package main

import (
	"fmt"

	"github.com/digitalocean/go-libvirt"
	`github.com/digitalocean/go-libvirt/socket`
	`github.com/digitalocean/go-libvirt/socket/dialers`
)

func main() {

	var conn *libvirt.Libvirt
	var dialer socket.Dialer
	var err error

	dialer = dialers.NewRemote("<IP_ADDRESS_HERE>", dialers.UsePort("22"))

	conn = libvirt.NewWithDialer(dialer)
	if err = conn.ConnectToURI("qemu+ssh://user@<IP_ADDRESS_HERE>/system"); err != nil {
		fmt.Println(err)
		return
	}
}

But it just hangs.

I find it hard to believe SSH tunneling was broken by oversight considering it's treated as a first-class connection URI upstream.

I want to add LookupDomainByUUID as i see payload.UUID needs to be 16 bytes long.
But when i'm convert string uuid to payload.UUID and try to get domain i have error

2017/04/03 14:58:44 failed to retrieve domain info: Domain not found: no domain with matching uuid '37653332-3532-3634-3135-623931383038'
exit status 1

But i'm provide uuid 7e32526415b918081a6b000008f85d7f

i want to have event in my ap when conection to libvirt daemon closed (client disconnected), as i see libvirt sources and this package i can use ConnectRegisterCloseCallback
But how can i work with this function?

Does you plan provide support from digitalocean to this package or after some times move to dbus based? (i think you know that libvirt devs write dbus interface to libvirt and in this case this generated wire binding can be relaced with dbus based)

DomainGetXMLDesc: write unix @->/var/run/libvirt/libvirt-sock: use of closed network connection.

code:
xmlDesc, err := l.DomainGetXMLDesc(*domain, 0)
if err != nil {
log.Printf("failed to DomainGetXMLDesc: %v", err)
return err
}

KB link has been updated and the old one now 404s.

https://libvirt.org/internals/rpc.html --> https://libvirt.org/kbase/internals/rpc.html

PR Link: #152

xdr:decodeInterface: can't decode to nil interface

Hello,

Awesome work on this new project so far! I was wondering if it would be possible to add the the current domain state in the info that can be accessed? I'm willing to work on this but just need to know where to start poking.

It seems like that we could use the REMOTE_PROC_DOMAIN_GET_STATE = 212 remote event as highlighted in this doc.

This issue is a followup from: #56

We are evaluating to use go-libvirt library in the upstream dmacvicar/terraform-provider-libvirt#813

Right now, not having acess to specific error is problematic, since we leveraged these errors with the other libvirt golang
library. see

Is any chance to expose this errors? What is the valid rationale? we would be able to help to improve the library if needed.

I still agree also with @rmohr #56 (comment)

see for example:

https://github.com/dmacvicar/terraform-provider-libvirt/blob/master/libvirt/volume.go#L22

Uncertain exactly why this occurs, but between the 6/26 commit and 7/15 commit (PR #86) the TLS connectivity no longer works. Commit hashes are shown in the go.mod file below if that assists.

Please let me know if I need to alter behavior of the code. For now, I'll lock into the 6/26 code.

My environment:

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
$ virsh version
Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Using API: QEMU 4.0.0
Running hypervisor: QEMU 2.11.1

Bad run sample output (the hang lasts for hours - we left town today and it was still hanging when I returned):

$ go run main.go
TCP connection to 'odin:16514' established
Libvirt client created
^Csignal: interrupt

Good run sample output:

$ go run main.go
TCP connection to 'odin:16514' established
Libvirt client created
Connected to libvirt
Version: 4.0.0
ID	Name		UUID
--------------------------------------------------------
17	vm-852ea340-3a46-471e-7b33-16b30df9bcba	852ea3403a46471e7b3316b30df9bcba

go.mod file:

module tls

go 1.12

// GOOD
require github.com/digitalocean/go-libvirt v0.0.0-20190626172931-4d226dd6c437

// BAD
//require github.com/digitalocean/go-libvirt v0.0.0-20190715144809-7b622097a793

main.go sample code (sorry, a bit long and hard-coded to locations on my filesystem):

package main

import (
	"crypto/tls"
	"crypto/x509"
	"fmt"
	"io/ioutil"
	"log"

	libvirt "github.com/digitalocean/go-libvirt"
)

const connectstring = "odin:16514"
const keyfile = "/home/rob/libvirt-cert/clientkey.pem"
const certfile = "/home/rob/libvirt-cert/clientcert.pem"
const cafile = "/home/rob/libvirt-cert/cacert.pem"

func main() {
	keyfilexml, err := ioutil.ReadFile(keyfile)
	if err != nil {
		panic(err)
	}

	certfilexml, err := ioutil.ReadFile(certfile)
	if err != nil {
		panic(err)
	}

	cafilexml, err := ioutil.ReadFile(cafile)
	if err != nil {
		panic(err)
	}

	cert, err := tls.X509KeyPair([]byte(certfilexml), []byte(keyfilexml))
	if err != nil {
		panic(err)
	}

	roots := x509.NewCertPool()
	roots.AppendCertsFromPEM([]byte(cafilexml))

	cfg := &tls.Config{
		Certificates: []tls.Certificate{cert},
		RootCAs:      roots,
	}

	conn, err := tls.Dial("tcp", connectstring, cfg)
	defer conn.Close()
	if err != nil {
		panic(err)
	}
	fmt.Printf("TCP connection to '%s' established\n", connectstring)

	client := libvirt.New(conn)
	fmt.Printf("Libvirt client created\n")
	err = client.Connect()
	defer client.Disconnect()
	if err != nil {
		panic(err)
	}
	fmt.Printf("Connected to libvirt\n")

	v, err := client.Version()
	if err != nil {
		log.Fatalf("failed to retrieve libvirt version: %v", err)
	}
	fmt.Println("Version:", v)

	domains, err := client.Domains()
	if err != nil {
		log.Fatalf("failed to retrieve domains: %v", err)
	}

	fmt.Println("ID\tName\t\tUUID")
	fmt.Printf("--------------------------------------------------------\n")
	for _, d := range domains {
		fmt.Printf("%d\t%s\t%x\n", d.ID, d.Name, d.UUID)
	}

	if err := client.Disconnect(); err != nil {
		log.Fatalf("failed to disconnect: %v", err)
	}
}

golint has a flag that causes it to exit with non-zero status, rendering our CI build script for this purpose obsolete.

Replace with golint -set_exit_status ./... in CI.

This is an easy Hacktoberfest contribution, so I'll leave this unfixed for a few days. Feel free to submit a PR to fix this and work toward that t-shirt!

In official api document, there are DomainMigrate operations.

However, in this repository they are missing. There are only some basic operations such as DomainMigratePrepare3, DomainMigratePerform3...

Is there anyone can help me finish the migrate operations ?

When calling a RPC method of libvirt, the returned error contains an error code used to indicate the error types, which is helpful.

I found there's no way to extract the error code from method call, cause the error returned in this package only contains the error message, which is not friendly for error cause tracing.

Error handling in this package(https://github.com/digitalocean/go-libvirt/blob/master/rpc.go#L355):

// decodeError extracts an error message from the provider buffer.
func decodeError(buf []byte) error {
    var e libvirtError

    dec := xdr.NewDecoder(bytes.NewReader(buf))
    _, err := dec.Decode(&e)
    if err != nil {
        return err
    }

    if strings.Contains(e.Message, "unknown procedure") {
        return ErrUnsupported
    }

    return errors.New(e.Message)
}

As the code above shows, the actual error returned was constructed by only the e.Message, e.Code was dropped.

I want to write a code to lookup domain first, then compare the error code to determine whether to define a new domain, something like this:

dom, err := l.DomainLookupByName("test")
if err != nil {
    if libvirtError, ok := err.(libvirt.Error); ok {
        if libvirtError.Code == libvirt.ERR_NO_DOMAIN {
            ...
        }
    }
}

Is there any plan to support error code extraction?

How plans define the domain?
Can I implement methods DomainDefineXML(xml []byte) error and DomainDefineXMLFlags(xml []byte, flags DomainDefineFlags) error like libvirt documentation

log: 2020-05-09T00:57:19Z E! failed to dial libvirt: dial unix /var/run/libvirt/libvirt-sock: connect: resource temporarily unavailable
2020-05-09T00:57:19Z E! failed to retry dial libvirt: dial unix /var/run/libvirt/libvirt-sock: connect: resource temporarily unavailable

Will we be able to receive events when states change? Is this possible either now or with the RPC?

Thanks

The package github.com/digitalocean/go-qemu/qmp is no longer compilable and gives the error:

# github.com/digitalocean/go-qemu/qmp
../../digitalocean/go-qemu/qmp/rpc.go:62:22: rpc.l.Events undefined (type *libvirt.Libvirt has no field or method Events)

I suspect its since this commit e2a69bc

We could really use integration testing to validate request parameters and multiple versions of libvirt.

I'm not entirely certain if this is a bug in my code or something is up with the NetworkUpdate call from this library.

I'm writing some code that will ultimately be installed into a VM in order to manage multiple VM's. The base OS layer does not have any libvirt dependencies, and this is the best solution for my needs. (So thanks for that!)

When I do a NetworkUpdate to add an IP>DHCP>Host entry, an error is generated: Operation not supported: can't update 'ip' section of network 'default'.

What I've done:

• verified the project is on the latest code.
• extracted the specific code, and it fails.
• used TLS connections (thinking there may be a different socket file to utilize)
• implemented with the libvirt/libvirt-go library, and it works.
• verified this also works from the virsh command-line.

Therefore, I think I've got a valid setup but haven't been able to identify what may be wrong (in either code base).

Running on Ubuntu 18.04.2.

$ virsh version
Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Using API: QEMU 4.0.0
Running hypervisor: QEMU 2.11.1

Using this library (and it fails):

const networkDhcpXML = "<host name='vm-8029d8c8-ee6c-45b9-6476-3ba7381c3cb5' ip='192.168.123.7'/>"
const networkName = "default"
const socket = "/var/run/libvirt/libvirt-sock"

func main() {
	conn, err := net.DialTimeout("unix", socket, 2*time.Second)
	defer conn.Close()
	if err != nil {
		panic(err)
	}
	fmt.Printf("Connected to '%s'\n", socket)

	client := libvirt.New(conn)
	err = client.Connect()
	if err != nil {
		panic(err)
	}
	fmt.Printf("Connected to libvirt\n")

	net, err := client.NetworkLookupByName(networkName)
	if err != nil {
		panic(err)
	}
	fmt.Printf("net = %v\n", net)

	xmlDesc, err := client.NetworkGetXMLDesc(net, 0)
	if err != nil {
		panic(err)
	}
	fmt.Printf("networkXMLDesc=%s\n", xmlDesc)

	cmd := uint32(libvirt.NetworkUpdateCommandAddLast)
	section := uint32(libvirt.NetworkSectionIPDhcpHost)
	//netflags := libvirt.NetworkUpdateAffectLive | libvirt.NetworkUpdateAffectConfig
	fmt.Printf("NetworkUpdate(%v, %d, %d, -1, %s, %d)\n", net, cmd, section, networkDhcpXML, 0)
	err = client.NetworkUpdate(net, cmd, section, -1, networkDhcpXML, 0)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Setup network DHCP\n")
}

Using the libvirt/libvirt-go library (and it works):

const networkDhcpXML = "<host name='vm-8029d8c8-ee6c-45b9-6476-3ba7381c3cb5' ip='192.168.123.7'/>"
const networkName = "default"

func main() {
	conn, err := libvirt.NewConnect("qemu:///system")
	if err != nil {
		panic(err)
	}
	defer conn.Close()

	net, err := conn.LookupNetworkByName(networkName)
	defer net.Free()
	if err != nil {
		panic(err)
	}
	fmt.Printf("net = %v\n", net)

	xmlDesc, err := net.GetXMLDesc(0)
	if err != nil {
		panic(err)
	}
	fmt.Printf("networkXMLDesc=%s\n", xmlDesc)

	cmd := libvirt.NETWORK_UPDATE_COMMAND_ADD_LAST
	section := libvirt.NETWORK_SECTION_IP_DHCP_HOST
	//netflags := libvirt.NetworkUpdateAffectLive | libvirt.NetworkUpdateAffectConfig
	fmt.Printf("NetworkUpdate(%v, %d, %d, -1, %s, %d)\n", net, cmd, section, networkDhcpXML, 0)
	err = net.Update(cmd, section, -1, networkDhcpXML, 0)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Setup network DHCP\n")
}

Using virsh:

$ virsh net-update default add --section ip-dhcp-host --xml "<host name='vm-8029d8c8-ee6c-45b9-6476-3ba7381c3cb5' ip='192.168.123.7'/>"
Updated network default live state

If it's useful, the specific usage is in the libvirtManager here. It's been rejiggered into the samples to work out whatever issue is occurring.

Thanks for any information you may provide!

I'm interesting in this package (i have about 400 servers with 100-150 vms on each host, so i'm not small libvirt user).
As i see now api needs more rewrite, because for example all methods recieve domain via string. But this is not very usable, because all the time we need to send additional lookup for domain resource. Also this design has some drawbacks - for example i can start domain, get it resource and use it to attach devices, get stats, reboot and watch events. But in you case this all need to do additional lookup, but why? I'm already have domain.
Also for function naming - events, define, undefine start, stop all needs prefix, because pool, volume, domain all have the same actions. Yes, we can decide via passed params, but i think this is not human frendly.

This is part of #31

@mdlayher @benlemasurier

libvirt version: 4.0.0
go-libvrit version: v0.0.0

code:
` // Report block device statistics. storage
for _, disk := range libvirtSchema.Devices.Disks {
if disk.Device == "cdrom" || disk.Device == "fd" {
continue
}

	isActive, err := l.DomainIsActive(*domain)
	if err != nil {
		log.Printf("E! [domain:%s] failed to get domain active state for block: %s", domain.Name, err.Error())
		return err
	}
	var rRdReq, rRdBytes, rWrReq, rWrBytes int64
	if isActive == 1 {
		rRdReq, rRdBytes, rWrReq, rWrBytes, _, err = l.DomainBlockStats(*domain, disk.Target.Device)
		if err != nil {
			log.Printf("E! [domain:%s] failed to get DomainBlockStats: %v", domain.Name, err.Error())
			return err
		}`

Hi! I found there is QEMUProcDomainAgentCommand = 3 have been defined but have no methods implement, is there any plans? thanks here!

libvirt log :
<30>2020-06-11T08:42:47.577912+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.489+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.581094+08:00 cmp001 libvirtd[18541]: message repeated 95 times: [ 2020-06-11 00:42:47.489+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.581120+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.490+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.584284+08:00 cmp001 libvirtd[18541]: message repeated 98 times: [ 2020-06-11 00:42:47.490+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.584310+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.491+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.586997+08:00 cmp001 libvirtd[18541]: message repeated 91 times: [ 2020-06-11 00:42:47.491+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.587023+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.492+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.590406+08:00 cmp001 libvirtd[18541]: message repeated 98 times: [ 2020-06-11 00:42:47.492+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.590467+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.493+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.593430+08:00 cmp001 libvirtd[18541]: message repeated 94 times: [ 2020-06-11 00:42:47.493+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.593456+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.494+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.596393+08:00 cmp001 libvirtd[18541]: message repeated 87 times: [ 2020-06-11 00:42:47.494+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.596419+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.495+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.598657+08:00 cmp001 libvirtd[18541]: message repeated 71 times: [ 2020-06-11 00:42:47.495+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.598683+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.496+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.600824+08:00 cmp001 libvirtd[18541]: message repeated 71 times: [ 2020-06-11 00:42:47.496+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.600885+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.497+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.603485+08:00 cmp001 libvirtd[18541]: message repeated 83 times: [ 2020-06-11 00:42:47.497+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.603511+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.498+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.606121+08:00 cmp001 libvirtd[18541]: message repeated 87 times: [ 2020-06-11 00:42:47.498+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.606142+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.499+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.608683+08:00 cmp001 libvirtd[18541]: message repeated 106 times: [ 2020-06-11 00:42:47.499+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.608710+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.500+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.610636+08:00 cmp001 libvirtd[18541]: message repeated 74 times: [ 2020-06-11 00:42:47.500+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.610669+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.501+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.613128+08:00 cmp001 libvirtd[18541]: message repeated 100 times: [ 2020-06-11 00:42:47.501+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.613154+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.502+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.615599+08:00 cmp001 libvirtd[18541]: message repeated 98 times: [ 2020-06-11 00:42:47.502+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.615626+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.503+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.617977+08:00 cmp001 libvirtd[18541]: message repeated 91 times: [ 2020-06-11 00:42:47.503+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.617996+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.504+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.620380+08:00 cmp001 libvirtd[18541]: message repeated 93 times: [ 2020-06-11 00:42:47.504+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.620400+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.505+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.622342+08:00 cmp001 libvirtd[18541]: message repeated 76 times: [ 2020-06-11 00:42:47.505+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.622369+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.506+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.624397+08:00 cmp001 libvirtd[18541]: message repeated 82 times: [ 2020-06-11 00:42:47.506+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.624425+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.507+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.626356+08:00 cmp001 libvirtd[18541]: message repeated 85 times: [ 2020-06-11 00:42:47.507+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.626376+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.508+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.628627+08:00 cmp001 libvirtd[18541]: message repeated 88 times: [ 2020-06-11 00:42:47.508+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.628647+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.509+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.630766+08:00 cmp001 libvirtd[18541]: message repeated 99 times: [ 2020-06-11 00:42:47.509+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]
<30>2020-06-11T08:42:47.630785+08:00 cmp001 libvirtd[18541]: 2020-06-11 00:42:47.510+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error
<30>2020-06-11T08:42:47.632915+08:00 cmp001 libvirtd[18541]: message repeated 78 times: [ 2020-06-11 00:42:47.510+0000: 18541: error : virNetSocketReadWire:1811 : End of file while reading data: Input/output error]

Bugs caused:
In my monitoring collector program, there will be stuck, when encountering the above situation in libvirt.

I am trying to subscribe to these events (in the title) like following:

events, err := l.SubscribeEvents(ctx, libvirt.DomainEventIDDeviceAdded, []libvirt.Domain{})
if err != nil {
	log.Fatal(err)
}

and then reading from that channel, but there no event comes no matter what I do..

I can see all these in virsh event --all --loop:

event 'lifecycle' for domain instance-000005ad: Defined Added
event 'lifecycle' for domain instance-000005ad: Started Booted
event 'lifecycle' for domain instance-000005ad: Suspended Paused
event 'lifecycle' for domain instance-000005ad: Resumed Unpaused
event 'device-added' for domain instance-000005ad: net1
event 'metdata-change' for domain instance-000005ad: element http://openstack.org/xmlns/libvirt/nova/1.1

but only lifecycle events work fine and can be subscribe/read from:

events, err := l.SubscribeEvents(ctx, libvirt.DomainEventIDLifecycle, []libvirt.Domain{})
if err != nil {
	log.Fatal(err)
}

any chance I am just missing something? Btw trying to use this on a openstack host with libvirt 5:

# virsh version
Compiled against library: libvirt 5.0.0
Using library: libvirt 5.0.0
Using API: QEMU 5.0.0
Running hypervisor: QEMU 2.11.1

rState, rmaxmem, rmemory, rvirCpu, rcputime, err := l.DomainGetInfo(*domain)

rmaxmem and rmemory are always equal.

Hi, I've been playing around with this library all day and I can't seem to get a correct read on the disk usage of a VM.

From ConnectGetAllDomainStats I get the following for a particular VM:

block.0.allocation: 1302527488
block.0.capacity: 8589934592
block.0.physical: 1285488640

And with virsh domstats I get the same info (since the source might be the same)

But inside the VM, with df, I get the following:

Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/vda1        8377344 2203900   6173444  27% /

When the VM is created, the data from ConnectGetAllDomainStats seems to be close to correct, or correct, but it seems to never update, since after creating a 1G file with fallocate in the VM, it still shows the old values.

I'm using libvirt v4.5.0

Is there a way of getting the real disk usage of a VM with this library? I've been hunting around the documentation but no function seems to provide this.

Newer versions of goyacc try to optimize the representation of certain things. If using the latest goyacc, one of the generated files will be updated to use thinner ints which breaks the build like so:

diff --git a/internal/lvgen/y.go b/internal/lvgen/y.go
index 1ea031b..389c36a 100644
--- a/internal/lvgen/y.go
+++ b/internal/lvgen/y.go
@@ -118,7 +118,7 @@ const yyInitialStackSize = 16
 //line sunrpc.y:279
 
 //line yacctab:1
-var yyExca = [...]int{
+var yyExca = [...]int8{
        -1, 1,
        1, -1,
        -2, 0,
@@ -128,7 +128,7 @@ const yyPrivate = 57344

$ go generate ./...
  processing libvirt.yml done.
# github.com/digitalocean/go-libvirt/internal/lvgen
./lvlexer.go:75:12: cannot use yyTok2[tokType - yyPrivate] (type int8) as type int in assignment
internal/lvgen/lv-gen.go:29: running "go": exit status 2

For now, I've installed goyacc@c6776771dde7a49828feb54fd8be11695a32b558 for local development.

goyacc seems to perform this optimization since commit 59f1f2c5a8fd844a6cc55e9a773a7003bd8e5480.

edit: the following components could be addressed:

• goyacc
• go/go-get/go-install (the CI seems to be pinned to 1.16.2 and uses go-get, as does ./scripts/gen-consts.sh; go-get is deprecated in Go 1.17)

I am trying to migrate via qemu+tls. For virsh it's virsh migrate --live testvm qemu+tls://$destination/system where the client will read from /etc/pki/libvirt/clientcert.pem /etc/pki/libvirt/private/clientkey.pem and /etc/pki/CA/cacert.pem

I see no fields where I can put the key/crt/ca in []byte in the this library. is it implemented?

https://godoc.org/github.com/digitalocean/go-libvirt

As part of integrating go-libvirt into terraform-provider-libvirt, I have implemented support for connection URIs, including the tcp, tls, unix and ssh transports.

I think this could make go-libvirt easier to use for those using the original bindings, and probably belongs outside of the terraform provider.

Is there interest in having this functionality in go-libvirt?. If that is the case, I could merge the type with go-libvirt and work on a patch.

Looking forward for your feedback. //cc @trapgate

Trying to build github.com/digitalocean/go-qemu with the current snapshot of this library results in the following compilation error:

github.com/digitalocean/go-qemu/qmp/raw
# github.com/digitalocean/go-qemu/qmp [github.com/digitalocean/go-qemu/qmp.test]
src/github.com/digitalocean/go-qemu/qmp/rpc_test.go:76:38: cannot use conn (variable of type *libvirttest.MockLibvirt) as type net.Conn in argument to NewLibvirtRPCMonitor:
	*libvirttest.MockLibvirt does not implement net.Conn (missing Close method)
src/github.com/digitalocean/go-qemu/qmp/rpc_test.go:123:38: cannot use conn (variable of type *libvirttest.MockLibvirt) as type net.Conn in argument to NewLibvirtRPCMonitor:
	*libvirttest.MockLibvirt does not implement net.Conn (missing Close method)
src/github.com/digitalocean/go-qemu/qmp/rpc_test.go:133:38: cannot use conn (variable of type *libvirttest.MockLibvirt) as type net.Conn in argument to NewLibvirtRPCMonitor:
	*libvirttest.MockLibvirt does not implement net.Conn (missing Close method)
src/github.com/digitalocean/go-qemu/qmp/rpc_test.go:143:38: cannot use conn (variable of type *libvirttest.MockLibvirt) as type net.Conn in argument to NewLibvirtRPCMonitor:
	*libvirttest.MockLibvirt does not implement net.Conn (missing Close method)
src/github.com/digitalocean/go-qemu/qmp/rpc_test.go:176:38: cannot use conn (variable of type *libvirttest.MockLibvirt) as type net.Conn in argument to NewLibvirtRPCMonitor:
	*libvirttest.MockLibvirt does not implement net.Conn (missing Close method)
# github.com/digitalocean/go-qemu/hypervisor [github.com/digitalocean/go-qemu/hypervisor.test]
src/github.com/digitalocean/go-qemu/hypervisor/rpc_test.go:27:9: cannot use m (variable of type *libvirttest.MockLibvirt) as type net.Conn in return statement:
	*libvirttest.MockLibvirt does not implement net.Conn (missing Close method)

I think it's correct to file the issue here, rather than against go-qemu. (If not, please feel free to move it.)

// DomainGetCPUStats is the go wrapper for REMOTE_PROC_DOMAIN_GET_CPU_STATS.
func (l *Libvirt) DomainGetCPUStats(Dom Domain, Nparams uint32, StartCPU int32, Ncpus uint32, Flags TypedParameterFlags) (rParams []TypedParam, rNparams int32, err error) {
}
Nparams ，StartCPU，TypedParameterFlags
How to finish the above three parameters?
And What can I get back? rParams ，rNparams

Here is the back trace:

Jul 6 09:35:00 CVIM-compute-1 telegraf: panic: runtime error: makeslice: len out of range
Jul 6 09:35:00 CVIM-compute-1 telegraf: goroutine 189 [running]:
Jul 6 09:35:00 CVIM-compute-1 telegraf: github.com/influxdata/telegraf/vendor/github.com/digitalocean/go-libvirt.(*Libvirt).listen(0xc42073c230)
Jul 6 09:35:00 CVIM-compute-1 telegraf: /builddir/build/src/github.com/influxdata/telegraf/vendor/github.com/digitalocean/go-libvirt/rpc.go:179 +0xb9
Jul 6 09:35:00 CVIM-compute-1 telegraf: created by github.com/influxdata/telegraf/vendor/github.com/digitalocean/go-libvirt.New
Jul 6 09:35:00 CVIM-compute-1 telegraf: /builddir/build/src/github.com/influxdata/telegraf/vendor/github.com/digitalocean/go-libvirt/libvirt.go:549 +0x25d
Jul 6 09:35:00 CVIM-compute-1 systemd: telegraf.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Jul 6 09:35:00 CVIM-compute-1 systemd: Unit telegraf.service entered failed state.

This only happens for some VMs and not other VMs.

I don’t understand the meaning of these three parameters.

Originally posted by @fcddk in #92 (comment)

libvirt version: 4.0.0
go-libvrit version: v0.0.0

code:
`
// Report network interface statistics. 网络

for _, iface := range libvirtSchema.Devices.Interfaces {
	if iface.Target.Device == "" {
		continue
	}



	isActive, err := l.DomainIsActive(*domain)
	if err != nil {
		log.Printf("E! [domain:%s] failed to get domain active state for interface: %s", domain.Name, err.Error())
		return err
	}



	var rRxBytes, rRxPackets, rRxErrs, rRxDrop, rTxBytes, rTxPackets, rTxErrs, rTxDrop int64


	if isActive == 1 {
		rRxBytes, rRxPackets, rRxErrs, rRxDrop, rTxBytes, rTxPackets, rTxErrs, rTxDrop, err = l.DomainInterfaceStats(*domain, iface.Target.Device)


		if err != nil {
			log.Printf("E! [domain:%s] failed to get DomainInterfaceStats: %s", domain.Name, err.Error())
			return err
		}`	

Does you have any plans to implement libvirt streams ? They are useful for volume upload and volume upload.

Cause libvirtd error：
logs: virNetSocketReadWire:1811 : End of file while reading data: Input/output error

Hi,

I think it will be nice to have the ability to close the domain events stream from Libvirt.Events() when it necessary to the caller.

I would like to obtain the amount of memory currently used by the domain like here

I'll try to implement REMOTE_PROC_DOMAIN_MEMORY_STATS, ok? What do you think? :-)

failed to get DomainBlockStats: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainBlockStats)

my code:

`

isActive, err := l.DomainIsActive(*domain)
var rRdReq, rRdBytes, rWrReq, rWrBytes int64
if isActive == 1 {
rRdReq, rRdBytes, rWrReq, rWrBytes, _, err = l.DomainBlockStats(*domain, disk.Target.Device) }`

	if err != nil {
		log.Fatalf("failed to get DomainBlockStats: %v", err)
		return err
	}

`

First of all: I really appreciate all your efforts to get a libvirt integration without C bindings!

Currently the Connect() error method uses the hardcoded libvirt driver URI qemu:///system which is kind of a showstopper for projects where I'd like to use this library.

After playing around a bit with the code I could not notice any problem with adding an overload like

ConnectToURI(uri string) error that allows for instance to use the mock driver test:///default or other libvirt drivers.

I'll open a PR and would be happy about any feedback!

Kind regards
Peter

I try to upload a Ubuntu image to libvirt via StorageVolUpload. If I upload the image from a file it does work but if I upload it from a http stream the image gets corrupted.
Strangely it does work from http stream as well if I use a big buffered reader in between.
Does anyone have an idea whats going on here?

Here are the steps to reproduce the behaviour:

• download the image

curl -O https://cloud-images.ubuntu.com/minimal/releases/disco/release-20190605/ubuntu-19.04-minimal-cloudimg-amd64.img

• download test code main.txt

curl -o main.go https://github.com/digitalocean/go-libvirt/files/3311212/main.txt

• run test

go run main.go

• compare downloaded files in /var/lib/libvirt/images (or where ever your default pool stores the files)

sha256sum /var/lib/libvirt/images/image_from_*

Hi.
I have been waiting for VPC functional for several weeks. https://github.com/terraform-providers/terraform-provider-digitalocean/issues/415
When it was added, I immediately used it in my project.
And Digitalocean blocked my account without explaining the reasons and did not tell me anything.
I lost work on which I worked for several weeks.
I urge everyone to use anything AWS, GCP, Azure, but not this garbage ocean.
Digitalocean doesn't care about its customers!