Giter Club home page Giter Club logo

fog-libvirt's Introduction

fog

fog is the Ruby cloud services library, top to bottom:

  • Collections provide a simplified interface, making clouds easier to work with and switch between.
  • Requests allow power users to get the most out of the features of each individual cloud.
  • Mocks make testing and integrating a breeze.

Build Status Code Climate Gem Version SemVer

Dependency Notice

Currently all fog providers are getting separated into metagems to lower the load time and dependency count.

If there's a metagem available for your cloud provider, e.g. fog-aws, you should be using it instead of requiring the full fog collection to avoid unnecessary dependencies.

'fog' should be required explicitly only if the provider you use doesn't yet have a metagem available.

Getting Started

The easiest way to learn fog is to install the gem and use the interactive console. Here is an example of wading through server creation for Amazon Elastic Compute Cloud:

$ sudo gem install fog
[...]

$ fog

  Welcome to fog interactive!
  :default provides [...]

>> server = Compute[:aws].servers.create
ArgumentError: image_id is required for this operation

>> server = Compute[:aws].servers.create(:image_id => 'ami-5ee70037')
<Fog::AWS::EC2::Server [...]>

>> server.destroy # cleanup after yourself or regret it, trust me
true

Ruby version

Fog requires Ruby 2.0.0 or later.

Ruby 1.8 and 1.9 support was dropped in fog-v2.0.0 as a backwards incompatible change. Please use the later fog 1.x versions if you require 1.8.7 or 1.9.x support.

Collections

A high level interface to each cloud is provided through collections, such as images and servers. You can see a list of available collections by calling collections on the connection object. You can try it out using the fog command:

>> Compute[:aws].collections
[:addresses, :directories, ..., :volumes, :zones]

Some collections are available across multiple providers:

  • compute providers have flavors, images and servers
  • dns providers have zones and records
  • storage providers have directories and files

Collections share basic CRUD type operations, such as:

  • all - fetch every object of that type from the provider.
  • create - initialize a new record locally and a remote resource with the provider.
  • get - fetch a single object by its identity from the provider.
  • new - initialize a new record locally, but do not create a remote resource with the provider.

As an example, we'll try initializing and persisting a Rackspace Cloud server:

require 'fog'

compute = Fog::Compute.new(
  :provider           => 'Rackspace',
  :rackspace_api_key  => key,
  :rackspace_username => username
)

# boot a gentoo server (flavor 1 = 256, image 3 = gentoo 2008.0)
server = compute.servers.create(:flavor_id => 1, :image_id => 3, :name => 'my_server')
server.wait_for { ready? } # give server time to boot

# DO STUFF

server.destroy # cleanup after yourself or regret it, trust me

Models

Many of the collection methods return individual objects, which also provide common methods:

  • destroy - will destroy the persisted object from the provider
  • save - persist the object to the provider
  • wait_for - takes a block and waits for either the block to return true for the object or for a timeout (defaults to 10 minutes)

Mocks

As you might imagine, testing code using Fog can be slow and expensive, constantly turning on and shutting down instances. Mocking allows skipping this overhead by providing an in memory representation of resources as you make requests. Enabling mocking is easy to use: before you run other commands, simply run:

Fog.mock!

Then proceed as usual, if you run into unimplemented mocks, fog will raise an error and as always contributions are welcome!

Requests

Requests allow you to dive deeper when the models just can't cut it. You can see a list of available requests by calling #requests on the connection object.

For instance, ec2 provides methods related to reserved instances that don't have any models (yet). Here is how you can lookup your reserved instances:

$ fog
>> Compute[:aws].describe_reserved_instances
#<Excon::Response [...]>

It will return an excon response, which has body, headers and status. Both return nice hashes.

Go forth and conquer

Play around and use the console to explore or check out fog.io and the provider documentation for more details and examples. Once you are ready to start scripting fog, here is a quick hint on how to make connections without the command line thing to help you.

# create a compute connection
compute = Fog::Compute.new(:provider => 'AWS', :aws_access_key_id => ACCESS_KEY_ID, :aws_secret_access_key => SECRET_ACCESS_KEY)
# compute operations go here

# create a storage connection
storage = Fog::Storage.new(:provider => 'AWS', :aws_access_key_id => ACCESS_KEY_ID, :aws_secret_access_key => SECRET_ACCESS_KEY)
# storage operations go here

geemus says: "That should give you everything you need to get started, but let me know if there is anything I can do to help!"

Versioning

Fog library aims to adhere to Semantic Versioning 2.0.0, although it does not address challenges of multi-provider libraries. Semantic versioning is only guaranteed for the common API, not any provider-specific extensions. You may also need to update your configuration from time to time (even between Fog releases) as providers update or deprecate services.

However, we still aim for forwards compatibility within Fog major versions. As a result of this policy, you can (and should) specify a dependency on this gem using the Pessimistic Version Constraint with two digits of precision. For example:

spec.add_dependency 'fog', '~> 1.0'

This means your project is compatible with Fog 1.0 up until 2.0. You can also set a higher minimum version:

spec.add_dependency 'fog', '~> 1.16'

Getting Help

Contributing

Please refer to CONTRIBUTING.md.

License

Please refer to LICENSE.md.

fog-libvirt's People

Contributors

abbbi avatar agx avatar anonym avatar apuntamb avatar ares avatar bluewind avatar byonoy-dne avatar dcbw avatar dependabot[bot] avatar dmitri-d avatar domcleal avatar edwardbetts avatar ekohl avatar electrofelix avatar geemus avatar germang avatar lzap avatar mmoll avatar ofedoren avatar olifre avatar plribeiro3000 avatar radeksimko avatar rwf14f avatar sford avatar strzibny avatar tbrisker avatar td512 avatar teancom avatar voxik avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

fog-libvirt's Issues

No ability to create libvirt KVM virtual machine instances with uefi support

Currently fog-libvirt only supports ability to create virtual machines via libvirt that are BIOS based. BIOS is slowly being deprecated on with Linux distributions. While Windows 11 no longer supports it all together.

Foreman currently relies on fog-libvirt to create virtual machines from a libvirt compute resource. Currently it can only support creation of BIOS based KVM virtual machines.

Enabling a way to specify creation of libvirt KVM instances with UEFI will resolve this issue.

https://community.theforeman.org/t/libvirt-and-uefi/22756

vagrant up fails with fog-libvirt 0.0.4, works with 0.0.3

Running Fedora 23 (fully updated) with vagrant-libvirt 0.0.32 and fog-libvirt 0.0.4. My vagrant ups are failing with:

==> foo: Creating shared folders metadata...
==> foo: Starting domain.
==> foo: Waiting for domain to get an IP address...
==> foo: Removing domain...
/home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/service.rb:244:in `validate_options': Missing required arguments: libvirt_uri (ArgumentError)
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/service.rb:268:in `handle_settings'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/service.rb:98:in `new'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/services_mixin.rb:16:in `new'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/compute.rb:65:in `new'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/services_mixin.rb:4:in `[]'
    from /home/albert/.vagrant.d/gems/gems/fog-libvirt-0.0.4/lib/fog/libvirt/models/compute/server.rb:269:in `addresses'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:39:in `block (3 levels) in call'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/model.rb:72:in `instance_eval'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/model.rb:72:in `block in wait_for'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/wait_for.rb:7:in `block in wait_for'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/wait_for.rb:6:in `loop'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/wait_for.rb:6:in `wait_for'
    from /home/albert/.vagrant.d/gems/gems/fog-core-1.36.0/lib/fog/core/model.rb:69:in `wait_for'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:38:in `block (2 levels) in call'
    from /usr/share/vagrant/lib/vagrant/util/retryable.rb:17:in `retryable'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:33:in `block in call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/util/timer.rb:9:in `time'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:31:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/start_domain.rb:188:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_boot_order.rb:60:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_network_interfaces.rb:138:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_networks.rb:79:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/share_folders.rb:20:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/prepare_nfs_settings.rb:18:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/synced_folders.rb:87:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/synced_folder_cleanup.rb:28:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/plugins/synced_folders/nfs/action_cleanup.rb:25:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/prepare_nfs_valid_ids.rb:12:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-hostmanager-1.8.1/lib/vagrant-hostmanager/action/update_all.rb:27:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/provision.rb:80:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_domain.rb:201:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_domain_volume.rb:51:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:109:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_storage_pool.rb:50:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /home/albert/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_name_of_domain.rb:35:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builder.rb:116:in `call'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `block in run'
    from /usr/share/vagrant/lib/vagrant/util/busy.rb:19:in `busy'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `run'
    from /usr/share/vagrant/lib/vagrant/action/builtin/call.rb:53:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /usr/share/vagrant/lib/vagrant/action/warden.rb:34:in `call'
    from /usr/share/vagrant/lib/vagrant/action/builder.rb:116:in `call'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `block in run'
    from /usr/share/vagrant/lib/vagrant/util/busy.rb:19:in `busy'
    from /usr/share/vagrant/lib/vagrant/action/runner.rb:66:in `run'
    from /usr/share/vagrant/lib/vagrant/machine.rb:224:in `action_raw'
    from /usr/share/vagrant/lib/vagrant/machine.rb:199:in `block in action'
    from /usr/share/vagrant/lib/vagrant/environment.rb:561:in `lock'
    from /usr/share/vagrant/lib/vagrant/machine.rb:185:in `call'
    from /usr/share/vagrant/lib/vagrant/machine.rb:185:in `action'
    from /usr/share/vagrant/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'

I happened to have another vagrant setup with fog-libvirt 0.0.3 and that works fine.

add missing vol-delete feature

The following irb session shows the missing feature:

1.9.2-p320 :019 > client.volumes
 =>   <Fog::Compute::Libvirt::Volumes
    [
      <Fog::Compute::Libvirt::Volume
        id="/var/lib/libvirt/images/mytest.img",
        pool_name="default",
        key="/var/lib/libvirt/images/mytest.img",
        name="mytest.img",
        path="/var/lib/libvirt/images/mytest.img",
        capacity=0,
        allocation=0,
        format_type="qcow2"
      >,
      <Fog::Compute::Libvirt::Volume
        id="/var/lib/libvirt/images/test-vol-foo.qcow2",
        pool_name="default",
        key="/var/lib/libvirt/images/test-vol-foo.qcow2",
        name="test-vol-foo.qcow2",
        path="/var/lib/libvirt/images/test-vol-foo.qcow2",
        capacity=5,
        allocation=0,
        format_type="qcow2"
      >,
      <Fog::Compute::Libvirt::Volume
        id="/var/lib/libvirt/images/fog-test.qcow2",
        pool_name="default",
        key="/var/lib/libvirt/images/fog-test.qcow2",
        name="fog-test.qcow2",
        path="/var/lib/libvirt/images/fog-test.qcow2",
        capacity=10,
        allocation=0,
        format_type="qcow2"
      >
    ]
  >
1.9.2-p320 :024 > client.volumes.delete :name => "fog-test.qcow2"
 => nil 
1.9.2-p320 :028 > client.volumes
 =>   <Fog::Compute::Libvirt::Volumes
    [
      <Fog::Compute::Libvirt::Volume
        id="/var/lib/libvirt/images/mytest.img",
        pool_name="default",
        key="/var/lib/libvirt/images/mytest.img",
        name="mytest.img",
        path="/var/lib/libvirt/images/mytest.img",
        capacity=0,
        allocation=0,
        format_type="qcow2"
      >,
      <Fog::Compute::Libvirt::Volume
        id="/var/lib/libvirt/images/test-vol-foo.qcow2",
        pool_name="default",
        key="/var/lib/libvirt/images/test-vol-foo.qcow2",
        name="test-vol-foo.qcow2",
        path="/var/lib/libvirt/images/test-vol-foo.qcow2",
        capacity=5,
        allocation=0,
        format_type="qcow2"
      >,
      <Fog::Compute::Libvirt::Volume
        id="/var/lib/libvirt/images/fog-test.qcow2",
        pool_name="default",
        key="/var/lib/libvirt/images/fog-test.qcow2",
        name="fog-test.qcow2",
        path="/var/lib/libvirt/images/fog-test.qcow2",
        capacity=10,
        allocation=0,
        format_type="qcow2"
      >
    ]
  > 

Running virsh directly on the same setup, works fine:

$ virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit
virsh # vol-list  default
Name                 Path
-----------------------------------------
fog-test.qcow2       /var/lib/libvirt/images/fog-test.qcow2
mytest.img           /var/lib/libvirt/images/mytest.img
test-vol-foo.qcow2   /var/lib/libvirt/images/test-vol-foo.qcow2

virsh # vol-delete --pool default "fog-test.qcow2"
Vol fog-test.qcow2 deleted

virsh # vol-list  default
Name                 Path
-----------------------------------------
mytest.img           /var/lib/libvirt/images/mytest.img                            
test-vol-foo.qcow2   /var/lib/libvirt/images/test-vol-foo.qcow2

Original issue reported by @noushi at fog/fog#2859

cc/ @noushi @geemus

Adding Xen support.

fog-libvirt is currently tuned to work with KVM.
With a few changes to the templates and domain I have been able to use my foreman install to manage my Xen/libvirt based VMs.
What are the chances of altering the templates to support Xen or allowing an alternate source of templates?

Ceph RBD volume creation

This has already been proposed in:
fog/fog#3285 (comment)
a few years ago, but sadly was closed without further comment / reasoning ๐Ÿ˜ข .

I hope this proposal here will be more successful.

I am right now patching server.xml.erb as follows:

  <devices>
<% args = {}
    File.readlines('/etc/foreman/ceph.conf').each do |line|
      pair = line.strip.split("=")
      key = pair[0]
      value = pair[1]
      args[key] = value
    end
%>
<% volumes.each do |vol| -%>
  <% if vol.pool_name.include? args["libvirt_ceph_pool"]  %>
    <disk type='network' device='disk'>
      <driver name='qemu' type='<%= vol.format_type %>' cache='writeback' discard='unmap'/>
      <source protocol='rbd' name='<%= vol.path %>'>
        <% args["monitor"].split(",").each do |mon| %>
        <host name='<%= mon %>' port='<%= args["port"] %>'/>
        <% end %>
      </source>
      <auth username='<%= args["auth_username"] %>'>
        <secret type='ceph' uuid='<%= args["auth_uuid"] %>'/>
      </auth>
      <target dev='sd<%= ('a'..'z').to_a[volumes.index(vol)] %>' bus='scsi'/>
    </disk>
  <% else %>
    <disk type='file' device='disk'>
      <driver name='qemu' type='<%= vol.format_type %>'/>
      <source file='<%= vol.path %>'/>
      <%# we need to ensure a unique target dev -%>
      <target dev='vd<%= ('a'..'z').to_a[volumes.index(vol)] %>' bus='virtio'/>
    </disk>
  <% end %>
<% end -%>
<% if iso_file -%>

This works perfectly fine, of course, configuration should be added for those choices (virtio-scsi, writeback, discard).
Also, such volumes can not be deleted by fog anymore:
https://projects.theforeman.org/issues/12063

Cannot list inactive pools

https://github.com/fog/fog-libvirt/blob/master/lib/fog/libvirt/requests/compute/list_pools.rb#L30 makes a call to get the num_of_volumes, however for inactive pools this will throw an exception in pool_to_attrbiutes that gets swallowed by either find_pool_by_name or find_pool_by_uuid.

I installed the gem, created a new pool in libvirt using virt-manager or virsh. Next I deactived the pool and ran the following code (similar to what vagrant-libvirt would do):

require 'fog/libvirt'
conn_attr = {
    :provider => "libvirt",
    :libvirt_uri => "qemu:///system",
    :libvirt_ip_command => " awk \"/$mac/ {print \\$1}\" /proc/net/arp"
}
conn = Fog::Compute.new(conn_attr)
pool = conn.client.lookup_storage_pool_by_name("test")
pool.num_of_volumes

The result was be similar to:

Libvirt::RetrieveError: Call to virStoragePoolNumOfVolumes failed: Requested operation is not valid: storage pool 'test' is not active
    from (irb):63:in `num_of_volumes'
    from (irb):63
    from /home/baileybd/.rvm/rubies/ruby-2.0.0-p643/bin/irb:12:in `<main>'

Looking at the states supported by list_pools, it suggests it can handle inactive pools.

This caused vagrant-libvirt/vagrant-libvirt#536

qemu-guest-agent support

server.xml.erb is not very configurable right now.
Adding the lines:

    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
    </channel>

allows qemu-guest-agent connectivity to the VM, which is crucial for fsfreeze, trim and other commands.
If this is present on VM creation, e.g. RHEL 7.5 / CentOS 7.5 systems install the guest agent with anaconda, and set up the Guest Agent Service automatically.

Question: Current use of filters in requests can result in test code being unable to exercise behaviour

A number of the requests support filters that change how libvirt is queried for the various resources:

class Real
def list_domains(filter = { })
data=[]
if filter.key?(:uuid)
data << client.lookup_domain_by_uuid(filter[:uuid])
elsif filter.key?(:name)
data << client.lookup_domain_by_name(filter[:name])
else
client.list_defined_domains.each { |name| data << catchLibvirtExceptions { client.lookup_domain_by_name(name) } } unless filter[:defined] == false
client.list_domains.each { |id| data << catchLibvirtExceptions { client.lookup_domain_by_id(id) } } unless filter[:active] == false
end
data.compact.map { |d| domain_to_attributes d }.compact
end
# Catch Libvirt exceptions to avoid race conditions involving
# concurrent libvirt operations from other processes. For example,
# domains being undefined while fog-libvirt is trying to work with
# domain lists.
def catchLibvirtExceptions
yield
rescue ::Libvirt::RetrieveError, ::Libvirt::Error
nil
end
end

Corresponding Mock:

class Mock
def list_domains(filter = { })
dom1 = mock_domain 'fog-dom1'
dom2 = mock_domain 'fog-dom2'
dom3 = mock_domain 'a-fog-dom3'
[dom1, dom2, dom3]
end

however because the filter is fully processed in the Real object, and the Mock just ignores it, it means that exercising any of the logic to ensure information is being retrieved or handled correctly around the filters, can be quite tricky.

Consequently this appears to result in the need to duplicate the filtering behaviour in the Mock instance that will subsequently need to be kept aligned with the Real instance. e.g.

class Mock
def list_networks(filter={ })
networks = [ {
:uuid => 'a29146ea-39b2-412d-8f53-239eef117a32',
:name => 'net1',
:bridge_name => 'virbr0'
},
{
:uuid => 'fbd4ac68-cbea-4f95-86ed-22953fd92384',
:name => 'net2',
:bridge_name => 'virbr1'
}
]
return networks if filter.empty?
case filter.keys.first
when :uuid
[networks.find(:uuid => filter[:uuid]).first]
when :name
[networks.find(:name => filter[:name]).first]
else
networks
end
end

needs to filter on the same attributes as:

class Real
def list_networks(filter = { })
data=[]
if filter.keys.empty?
(client.list_networks + client.list_defined_networks).each do |network_name|
data << network_to_attributes(client.lookup_network_by_name(network_name))
end
else
data = [network_to_attributes(get_network_by_filter(filter))]
end
data
end

Is this the intended usage for Mocks vs Real? Or should the specific requests be wrapped with private functions for both the Mock and Real with the filter code being common for both and subsequently calling the private methods?
e.g.:

module Fog
  module Libvirt
    class Compute
        def self.list_networks(filter = { })
          data=[]
          if filter.keys.empty?
            (list_networks + list_defined_networks).each do |network_name|
              data << network_to_attributes()
            end
          else
            data = [network_to_attributes(get_network_by_filter(filter))]
          end
          data
        end

        def self.get_network_by_filter(filter = { })
          case filter.keys.first
            when :uuid
              lookup_by_uuid(filter[:uuid])
            when :name
              lookup_by_name(filter[:name])
          end
        end
      end

      class Real
        private
        def list_networks
          client.list_networks
        end

        def list_networks_defined
          client.list_networks_defined
        end

        def lookup_by_name(name)
          client.lookup_network_by_name(network_name)
        end

        def lookup_by_uuid(uuid)
          client.lookup_network_by_uuid(uuid)
        end
      end

...

That way the Mock can implement the four calls list_networks, list_networks_defined, lookup_by_name and lookup_by_uuid, retrieving information from an internal stored array, but it still ensures the main logic is exercised and it is only the client request part itself that ends up needing to be mocked and kept in sync.

Note that I'm not expecting the above to be usable as is, just a way of trying to outline how it's a bit difficult to ensure the filtering code in list_networks in the existing code is correct given that any tests won't exercise it, and it's unclear if this is the intended approach or something that has be caused by the fact that for libvirt the filtering results in different API calls being made.

Failure during ansible provisioning

I have posted this on the gentoo forums (the OS in use) and the vagrant issue tracker with no responses, so I thought I would come straight to the source of the issue.

As I mentioned, I am using the gentoo OS along with vagrant 2.2.2 using KVM/qemu/libvirt for virtualization. The vagrant box gets spun up well enough, but I seem to have a small issue, and a large issue. The small issue just seems to affect CentOS boxes (6 and 7) which is if the ssh daemon is started with systemd I can not ssh directly to the box but if I start the service manually (/usr/sbin/sshd) everything works fine, but the "vagrant ssh" does work. The bigger issue that doesn't seem to be OS specific is that when vagrant runs the provisioning process (only tested with ansible), it blows up with the following error.

I realize that this may be and OS (i.e. kernel config likely) or even qemu/libvirt/KVM level fix (config or missing dependency through missing USE flag), but if I can understand why the error is occurring maybe I could get this working.

/home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/fog-libvirt-0.5.0/lib/fog/libvirt/requests/compute/list_domains.rb:9:in lookup_domain_by_uuid': Call to virDomainLookupByUUID failed: internal error: client socket is closed (Libvirt::RetrieveError) from /home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/fog-libvirt-0.5.0/lib/fog/libvirt/requests/compute/list_domains.rb:9:in list_domains'
from /home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/fog-libvirt-0.5.0/lib/fog/libvirt/models/compute/servers.rb:15:in get' from /home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/driver.rb:65:in get_domain'
from /home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/driver.rb:79:in created?' from /home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/provider.rb:101:in state'
from /home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/action/wait_till_up.rb:105:in terminate' from /home/sirtuxalot/.vagrant.d/gems/2.4.5/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/action/wait_till_up.rb:101:in recover'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:67:in block in recover' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:64:in each'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:64:in recover' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/builtin/call.rb:61:in recover'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:67:in block in recover' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:64:in each'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:64:in recover' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:53:in rescue in call'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/warden.rb:28:in call' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/builder.rb:116:in call'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/runner.rb:66:in block in run' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/util/busy.rb:19:in busy'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/action/runner.rb:66:in run' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/machine.rb:239:in action_raw'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/machine.rb:208:in block in action' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/environment.rb:614:in lock'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/machine.rb:194:in call' from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/machine.rb:194:in action'
from /usr/lib64/ruby/gems/2.4.0/gems/vagrant-2.2.2/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'

Thank you. If you need any more info, please let me know.

cloud-init iso using mkisofs

Annoyingly MacOS does not support genisoimage. Given the behaviour between mkisofs and genisoimage is identical (on CentOS 8 it's mkisofs is even symlinked to genisoimage), it will really be useful to use mkisofs as a fallback of genisoimage.

I created a pull request on #69. Happy to discuss about it!

Call to virStorageVolGetInfo failed: possible race condition

Hi!

I just run into this problem:

-----> Starting Kitchen (v1.4.2)
-----> Cleaning up any prior instances of <default-debian-8>
-----> Destroying <default-debian-8>...
       Finished destroying <default-debian-8> (0m0.00s).
-----> Testing <default-debian-8>
-----> Creating <default-debian-8>...
       Bringing machine 'default' up with 'libvirt' provider...
       /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `info': Call to virStorageVolGetInfo failed: cannot stat file '/var/lib/libvirt/images/vurbia-VAGRANTSLASH-debian-7_vagrant_box_image_20160213024500.img': No such file or directory (Libvirt::RetrieveError)
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `volume_to_attributes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:10:in `block (2 levels) in list_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `each'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `block in list_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:44:in `block in raw_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `each'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `raw_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:8:in `list_volumes'
        from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/models/compute/volumes.rb:11:in `all'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:63:in `block in call'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `synchronize'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_storage_pool.rb:50:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_name_of_domain.rb:35:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/call.rb:53:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:224:in `action_raw'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:199:in `block in action'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/environment.rb:561:in `lock'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `call'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `action'
        from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
-----> Destroying <default-debian-8>...
       Finished destroying <default-debian-8> (0m0.00s).
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #create action: [Expected process to exit with [0], but received '1'
---- Begin output of vagrant up --no-provision --provider libvirt ----
STDOUT: Bringing machine 'default' up with 'libvirt' provider...
STDERR: /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `info': Call to virStorageVolGetInfo failed: cannot stat file '/var/lib/libvirt/images/vurbia-VAGRANTSLASH-debian-7_vagrant_box_image_20160213024500.img': No such file or directory (Libvirt::RetrieveError)
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:32:in `volume_to_attributes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:10:in `block (2 levels) in list_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `each'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:9:in `block in list_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:44:in `block in raw_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `each'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:42:in `raw_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/requests/compute/list_volumes.rb:8:in `list_volumes'
    from /var/lib/jenkins/.vagrant.d/gems/gems/fog-libvirt-0.0.3/lib/fog/libvirt/models/compute/volumes.rb:11:in `all'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:63:in `block in call'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `synchronize'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:60:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_storage_pool.rb:50:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /var/lib/jenkins/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_name_of_domain.rb:35:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/call.rb:53:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/warden.rb:34:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/builder.rb:116:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `block in run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/util/busy.rb:19:in `busy'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/action/runner.rb:66:in `run'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:224:in `action_raw'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:199:in `block in action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/environment.rb:561:in `lock'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `call'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/machine.rb:185:in `action'
    from /opt/vagrant/embedded/gems/gems/vagrant-1.8.1/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
---- End output of vagrant up --no-provision --provider libvirt ----
Ran vagrant up --no-provision --provider libvirt returned 1]

If you notice that the error is regarding a volume for a completely different instance you may arrive at the same conclusion I did.
I believe this is a race condition caused by bringing up two different instances at the same time that causes lib/fog/libvirt/requests/compute/list_volumes.rb:32 to try to stat the non-existant image file that the other instance is bringing up.

I don't know enough ruby to fix this myself, but I'll help in any way I can.

Thanks!
--Ariel

new gem release?

It would be nice to have a new release, so projects using this gem can benefit from the fixes and enhancements.

Support for hostdev mode network

Hi,
I have libvirt network with hostdev mode (pool of virtual NICs on PCI-E) and server is managed by Foreman.
This type of network seems to be unsupported by fog-libvirt.

<network>
<name>pf-eth0</name>
<uuid>0ae0eb24-d0d9-4a02-942e-d245c77e12ba</uuid>
<forward mode='hostdev' managed='yes'>
<driver name='vfio'/>
<pf dev='eth0'/>
</forward>
</network>

Backtace from foreman is attached as trace.txt

Is someone willing to add such support please?

Q35 machine type does not support IDE

When trying to use a cloud-init image based deployment with Foreman, the VM provisioning fails as the Q35 machine type does not support the IDE bus. This was broken by #127 a change similar to this fix was mentioned as part of #76 but does not look like it made it into the final set. I haven't been able to figure out the conditional check yet so do not have a pr but the beloow changes have been confirmed to work under q35. I have not looked to see if i440 supports sata allowing the logic to be skipped/deferred for now. If you have suggestions to query the machine, I can try to finish turning this into a pr.

xml.target(:dev => "hdc", :bus => "ide") --> xml.target(:dev => "sr0", :bus => "sata")
xml.address(:type => "drive", :controller => 0, :bus => 1, :unit => 0) --> xml.address(:type => "drive", :controller => 0, :bus => 0, :unit => 0)

This issue is mentioned in serval places

Q35 chipset support?

Hi, I'm using this gem by way of Foreman's libvirt provider. I'm having trouble figuring out a non-hacky way to create a machine based on the newer Q35 chipset. Currently, I'm simply editing server.xml.erb and adding machine='q35' into the <os><type> key and switching the IDE cdrom for a SATA one.

I believe that changes to server.xml.erb are necessary insofar as replacing the Q35-incompatable IDE bus, but I'm having trouble finding a way to specify a machine's chipset from libvirt's Ruby API.

Any suggestions so that I could possibly get a real PR going?

Lack of macvlan/macvtap support

Hello,

At the moment fog-libvirt knows nothing about macvlan/macvtap interfaces:
https://libvirt.org/formatnetwork.html#examplesDirect

It assumes, that the defined network should have a bridge_name:

def network_to_attributes(net)
          return if net.nil?
          {
            :uuid        => net.uuid,
            :name        => net.name,
            :bridge_name => net.bridge_name
          }
        end

macvtap doesn't have this parameter. So, even when macvtap is not actually used by running VMs, the fact that it's defined in libvirt on its own breaks fog-libvirt.

It would be very nice to have different networking models support. As an example, we are currently unable to create VMs in Foreman on supervisors with macvlan because of this issue.

Thank you for your time.

Use libvirt DHCP leases API

Fog::Compute::Libvirt::Server#addresses runs an external shell command to determine the IP address of a VM that has a given MAC address. The implementation assumes libvirtd runs on localhost or can be SSHed to; it ignores other cases, such as a TCP transport. Please use the ruby-libvirt bindings' dhcp_leases method, to also support the case when libvirtd runs on a remote host, using a TCP transport.

The code should be something along these lines:

libvirt_conn = @client
libvirt_network = libvirt_conn.lookup_network_by_name('default')
libvirt_network.dhcp_leases('52:54:00:54:74:88')

minitest/unit.rb:28:in `const_missing': uninitialized constant MiniTest::Test (NameError)

Trying to build fog-libvirt on CentOS 7, when I try to run the tests with:

ruby -Iminitests -e "Dir.glob './minitests/**/*_test.rb', &method(:require)"

I'm getting an error:

  Fog::Compute[:libvirt] | servers collection (libvirt)
    The servers collection
      + should not be empty
      + should be a kind of Fog::Compute::Libvirt::Servers
      should be able to reload itself + succeeds
      should be able to get a model
        by instance uuid + succeeds
  
  196 succeeded in 0.252094842 seconds
  
+ ruby -Iminitests -e 'Dir.glob '\''./minitests/**/*_test.rb'\'', &method(:require)'
/usr/share/gems/gems/minitest-4.3.2/lib/minitest/unit.rb:28:in `const_missing': uninitialized constant MiniTest::Test (NameError)
	from /builddir/build/BUILD/fog-libvirt-0.3.0/usr/share/gems/gems/fog-libvirt-0.3.0/minitests/server/user_data_iso_test.rb:3:in `<top (required)>'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
	from -e:1:in `glob'
	from -e:1:in `<main>'

Any idea what that might be caused by?

dhcp_leases.rb:8:in `dhcp_leases': undefined method `dhcp_leases' for #<Libvirt::Network:0x00000003ea12f0> (NoMethodError)

virNetworkGetDHCPLeases isn't supported in libvirt < 1.2.6 so on distros carrying older libvirt, everything compiles but then using this call fails with above error.

This was introduced in PR #11

I think a few things could be done:

  1. Start depending on a specific libvirt version (depending on just ruby-libvirt doesn't seem to be enough)
  2. Test if the method exists before calling it and then fail with a descriptive error if it doesn't
  3. Support old and new behaviour (ugh)

Thoughts?

Support for cpu mode

Hi,

I'd like to be able to set the cpu mode on my VMs on creation, but fog-libvirt doesn't seem to currently support this. This would require adding a line like that can render to <cpu mode='something'/> to fog-libvirt/lib/fog/libvirt/models/compute/templates/server.xml.erb, along with other things.

Is this the right place to look for a feature like this? If yes, would it be ok if I hacked on this and submitted a PR to add this functionality?

Ask for merge and release permissions

A set of issues was reported on Foreman project with recent version of libvirt. Some have even fixing PRs here, e.g. #116

I know this gem was maintained more actively, but I suppose it falls under similar category as other adapters, @geemus if you wouldn't object, we'd like to get merge permissions and release ownership on rubygems so we could restore the compatibility with the new version of libvirt. That means, we would not actively develop the gem, but we'd try to maintain it, since it's Foreman project dependency.

Thanks for considering.

dhcp_leases.rb:8:in `dhcp_leases': Call to virNetworkGetDHCPLeases failed: internal error: no lease with matching MAC address: 52:54:00:32:36:b7 (Libvirt::Error)

When calling dhcp_leases with a mac that does not exist, libvirt throws an error.

/root/.vagrant.d/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/requests/compute/dhcp_leases.rb:8:in `dhcp_leases': Call to virNetworkGetDHCPLeases failed: internal error: no lease with matching MAC address: 52:54:00:32:36:b7 (Libvirt::Error)
    from /root/.vagrant.d/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/requests/compute/dhcp_leases.rb:8:in `dhcp_leases'
    from /root/.vagrant.d/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/models/compute/network.rb:20:in `dhcp_leases'
    from /root/.vagrant.d/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/models/compute/server.rb:272:in `block in addresses'
    from /root/.vagrant.d/gems/gems/fog-core-1.43.0/lib/fog/core/collection.rb:19:in `each'
    from /root/.vagrant.d/gems/gems/fog-core-1.43.0/lib/fog/core/collection.rb:19:in `each'
    from /root/.vagrant.d/gems/gems/fog-libvirt-0.3.0/lib/fog/libvirt/models/compute/server.rb:270:in `addresses'
    from /root/.vagrant.d/gems/gems/vagrant-libvirt-0.0.36/lib/vagrant-libvirt/action/wait_till_up.rb:44:in `block (3 levels) in call'
...

vagrant-libvirt calls def addresses in a while loop, waiting for a VM to boot and retrying until an IP is returned (with an eventual timeout). Since PR #11 this fails due to a functional change.

I believe this case should be handled correctly and return nil for def addresses(service_arg=service, options={}) as it was before so that this doesn't break existing functionality. Or is the failure here intentional?

volume filters always returns volume object when no match found

In trying to understand why the following piece of code https://github.com/hpe-hcss/authorization/blob/master/api/openapi-spec/authorization-broker-v1alpha2.yml#L120 always works, as in if there is no match found when using the filter argument to volumes.all() at https://github.com/fog/fog-libvirt/blob/master/lib/fog/libvirt/models/compute/volumes.rb#L10-L12, why would we be able to check the returned volume to see if it's id is not nil?

Doing some testing with the following where "someimage.img" does not exist

require 'fog/libvirt'

conn_attr = {:provider => 'libvirt', :libvirt_uri => 'qemu:///system'}
conn = Fog::Compute.new(conn_attr)
vols = conn.volumes.all(name: "someimage.img")
puts "vols => #{vols}"

and I'd get the outpu

vols => [  <Fog::Libvirt::Compute::Volume
    id=nil,
    pool_name="default",
    key=nil,
    name="fog-567594173461216",
    path=nil,
    capacity="10G",
    allocation="1G",
    owner="0",
    group="0",
    format_type="raw",
    backing_volume=nil
  >]

While digging into the code I spotted that that when a filter is used, https://github.com/fog/fog-libvirt/blob/master/lib/fog/libvirt/requests/compute/list_volumes.rb#L20 calls get_volume, which is defined at https://github.com/fog/fog-libvirt/blob/master/lib/fog/libvirt/requests/compute/list_volumes.rb#L72 and will return an empty hash in the case there is no volume.

Looking at https://github.com/fog/fog-libvirt/blob/master/lib/fog/libvirt/requests/compute/list_volumes.rb#L20 and https://github.com/fog/fog-libvirt/blob/master/lib/fog/libvirt/models/compute/volumes.rb#L11 it appears this results in [ {} ] being passed to https://github.com/fog/fog-core/blob/master/lib/fog/core/collection.rb#L75-L78 which will result in a collection containing a single volume created from an empty hash getting returned.

This seems like a bug, as if I do the same check with pools:

require 'fog/libvirt'

conn_attr = {:provider => 'libvirt', :libvirt_uri => 'qemu:///system'}
conn = Fog::Compute.new(conn_attr)

pools = conn.pools.all(name: "does not exist")
puts "pools => #{pools}"

the output is:

pools => []

A quick experiment suggests that the following is sufficient to resolve:

diff --git a/lib/fog/libvirt/requests/compute/list_volumes.rb b/lib/fog/libvirt/requests/compute/list_volumes.rb
index 9670f0a..3b837f7 100644
--- a/lib/fog/libvirt/requests/compute/list_volumes.rb
+++ b/lib/fog/libvirt/requests/compute/list_volumes.rb
@@ -17,7 +17,7 @@ module Fog
               end
             end
           else
-            return [get_volume(filter)]
+            data << get_volume(filter)
           end
           data.compact
         end
@@ -69,7 +69,8 @@ module Fog
               return raw ? vol : volume_to_attributes(vol)
             end
           end
-          { }
+
+          nil
         end
       end

Will send a PR up shortly assuming this seems reasonable

Cloning volumes from templates when creating new VMs

Heyo,
I was wondering if there's a way to do this. I saw there's a volume_template_name attr, but when I try to use it, I get Fog::Errors::Error (Error saving the server: Error creating the volume : identity is required for this operation)

Cheers

lxc support

Hello, I've had a quick look at getting lxc support in the libvirt driver. I'm no programmer, and have no experience with ruby. With the following changes I was able to create a lxc container using the libvirt driver.

I would like to see lxc support in some fashion in the libvirt driver, hope this can help that happen.

# diff libvirt/models/compute/templates/server.xml.erb.lxc  libvirt/models/compute/templates/server.xml.erb
9d8
<     <init>'<%= os_init %>'</init>
11a11,15
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>   </features>
13,15d16
<   <on_poweroff>destroy</on_poweroff>
<   <on_reboot>restart</on_reboot>
<   <on_crash>destroy</on_crash>
39a41,43
>     <serial type='pty'>
>       <target port='0'/>
>     </serial>
41c45
<       <target type='lxc' port='0'/>

---
>       <target port='0'/>
42a47,52
>     <input type='tablet' bus='usb'/>
>     <input type='mouse' bus='ps2'/>
>     <graphics type='<%= display[:type] %>' port='<%= display[:port] %>' autoport='yes' <% if display[:listen] and !(display[:listen].empty?)  %> listen='<%= display[:listen] %>'<% end %> <% if display[:password] and !(display[:password].empty?) %>passwd='<%=display[:password] %>'<% end %> />
>     <video>
>       <model type='cirrus' vram='9216' heads='1'/>
>     </video>
#diff libvirt/models/compute/server.rb libvirt/models/compute/server.rb.lxc 
15a16
>         attribute :os_init
361c362,363
<             :os_type                => "hvm",

---
>             :os_type                => "exe",
>             :os_init                => "/bin/init",
363c365
<             :domain_type            => "kvm",

---
>             :domain_type            => "lxc",

Original issue opened by @selund at fog/fog#3221

cc/ @selund @geemus

Operation not supported: cannot change keymap setting on vnc graphics

Looks like libvirt doesnt support to set keymap when using vnc.

 | Libvirt::Error: Call to virDomainUpdateDeviceFlags failed: Operation not supported: cannot change keymap setting on vnc graphics
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/fog-libvirt-0.4.0/lib/fog/libvirt/requests/compute/update_display.rb:21:in `update_device'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/fog-libvirt-0.4.0/lib/fog/libvirt/requests/compute/update_display.rb:21:in `update_display'
 | /usr/share/foreman/vendor/ruby/2.3.0/gems/fog-libvirt-0.4.0/lib/fog/libvirt/models/compute/server.rb:218:in `update_display'

Port defaults to -1

Hey all,

In libvirt/models/compute/templates/server.xml.erb there is an error on line 49 that's causing the port to default to -1

Here's the fix:

<% if display[:port] and !(display[:port].empty?) %>port='<%= display[:port] %>' autoport='no' <% else %>port='-1' autoport='yes' <%end%> instead of port='<%= display[:port] %>' autoport='yes'

I've already tested it on my local cluster, and that stops the port defaulting to -1

autostart support for servers

Hi,

There is an autostart attribute in the server model for libvirt. However, I don't see where this attribute could be enabled when creating a VM. I don't know libvirt's API, but it seems there's a separate method to call when enabling autostart.

Is it currently possible to enable autostart when creating a VM with fog? If so, how? If not, would it be possible to add it?

Thank you.

Original issue opened by @eolamey at fog/fog#3343

cc/ @eolamey @geemus

list_domains:domains_volumes only parses <source file=...>

Hi,

vols_by_file = xml_elements(xml, "domain/devices/disk/source", "file")

that parses only definitions like:

    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source file='/dev/mapper/s093043'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>

However, there's also "dev" (type block) which seams to be valid as well?

    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/mapper/s093250'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>

The latter isn't covered by the above mentioned request.

volumes/volume_path would be nil then.

hugepages support

Hi,

Would it be possible to optionally enable hugepages support for the libvirt driver? This could default to false, but having an option to enable it when creating the VM would be great.

When enabled, the server.xml template would have:

<domain>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  ...
</domain>

Enabling hugepages support on a Linux system requires a few steps, but I'm guessing documenting them would be outside of fog's scope.

Thank you.

Original issue opened by @eolamey at fog/fog#3342

cc/ @eolamey @geemus

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.