Giter Club home page Giter Club logo

vmware_exporter's Introduction

Needs new home. Since Broadcom is no longer catering to its opensource/free base, I have decided to move on from maintaining this project. Please create a discussion if you would like to take over managing this project.

vmware_exporter

VMware vCenter Exporter for Prometheus.

Get VMware vCenter information:

  • Basic VM and Host metrics
  • Current number of active snapshots
  • Datastore size and other stuff
  • Snapshot Unix timestamp creation date

Badges

Docker Stars Docker Pulls Docker Automated

Travis Build Status Docker Build Join the chat at https://gitter.im/vmware_exporter/community

Usage

Requires Python >= 3.6

  • Install with $ python setup.py install or via pip $ pip install vmware_exporter. The docker command below is preferred.
  • Create config.yml based on the configuration section. Some variables can be passed as environment variables
  • Run $ vmware_exporter -c /path/to/your/config
  • Go to http://localhost:9272/metrics?vsphere_host=vcenter.company.com to see metrics

Alternatively, if you don't wish to install the package, run it using $ vmware_exporter/vmware_exporter.py or use the following docker command:

docker run -it --rm  -p 9272:9272 -e VSPHERE_USER=${VSPHERE_USERNAME} -e VSPHERE_PASSWORD=${VSPHERE_PASSWORD} -e VSPHERE_HOST=${VSPHERE_HOST} -e VSPHERE_IGNORE_SSL=True -e VSPHERE_SPECS_SIZE=2000 --name vmware_exporter pryorda/vmware_exporter

When using containers combined with --env-file flag, please use capital letters to set bolleans, for example:

$ podman run -it --rm -p 9272:9272 --name vmware_exporter --env-file config.env pryorda/vmware_exporter
$ cat config.env
[email protected]
VSPHERE_PASSWORD=Secure-Pass
VSPHERE_HOST=192.168.0.1
VSPHERE_IGNORE_SSL=TRUE
VSPHERE_SPECS_SIZE=2000

Configuration and limiting data collection

Only provide a configuration file if enviroment variables are not used. If you do plan to use a configuration file, be sure to override the container entrypoint or add -c config.yml to the command arguments.

If you want to limit the scope of the metrics gathered, you can update the subsystem under collect_only in the config section, e.g. under default, or by using the environment variables:

collect_only:
    vms: False
    vmguests: True
    datastores: True
    hosts: True
    snapshots: True

This would only connect datastores and hosts.

You can have multiple sections for different hosts and the configuration would look like:

default:
    vsphere_host: "vcenter"
    vsphere_user: "user"
    vsphere_password: "password"
    ignore_ssl: False
    specs_size: 5000
    fetch_custom_attributes: True
    fetch_tags: True
    fetch_alarms: True
    collect_only:
        vms: True
        vmguests: True
        datastores: True
        hosts: True
        snapshots: True

esx:
    vsphere_host: vc.example2.com
    vsphere_user: 'root'
    vsphere_password: 'password'
    ignore_ssl: True
    specs_size: 5000
    fetch_custom_attributes: True
    fetch_tags: True
    fetch_alarms: True
    collect_only:
        vms: False
        vmguests: True
        datastores: False
        hosts: True
        snapshots: True

limited:
    vsphere_host: slowvc.example.com
    vsphere_user: '[email protected]'
    vsphere_password: 'password'
    ignore_ssl: True
    specs_size: 5000
    fetch_custom_attributes: True
    fetch_tags: True
    fetch_alarms: False
    collect_only:
        vms: False
        vmguests: False
        datastores: True
        hosts: False
        snapshots: False

Switching sections can be done by adding ?section=limited to the URL.

Environment Variables

Variable Precedence Defaults Description
VSPHERE_HOST config, env, get_param n/a vsphere server to connect to
VSPHERE_USER config, env n/a User for connecting to vsphere
VSPHERE_PASSWORD config, env n/a Password for connecting to vsphere
VSPHERE_SPECS_SIZE config, env 5000 Size of specs list for query stats function
VSPHERE_IGNORE_SSL config, env False Ignore the ssl cert on the connection to vsphere host
VSPHERE_FETCH_CUSTOM_ATTRIBUTES config, env False Set to true to collect objects custom attributes as metric labels
VSPHERE_FETCH_TAGS config, env False Set to true to collect objects tags as metric labels
VSPHERE_FETCH_ALARMS config, env False Fetch objects triggered alarms, and in case of hosts hdw alarms as well
VSPHERE_COLLECT_HOSTS config, env True Set to false to disable collection of host metrics
VSPHERE_COLLECT_DATASTORES config, env True Set to false to disable collection of datastore metrics
VSPHERE_COLLECT_VMS config, env True Set to false to disable collection of virtual machine metrics
VSPHERE_COLLECT_VMGUESTS config, env True Set to false to disable collection of virtual machine guest metrics
VSPHERE_COLLECT_SNAPSHOTS config, env True Set to false to disable collection of snapshot metrics

You can create new sections as well, with very similiar variables. For example, to create a limited section you can set:

Variable Precedence Defaults Description
VSPHERE_LIMITED_HOST config, env, get_param n/a vsphere server to connect to
VSPHERE_LIMITED_USER config, env n/a User for connecting to vsphere
VSPHERE_LIMITED_PASSWORD config, env n/a Password for connecting to vsphere
VSPHERE_LIMITED_SPECS_SIZE config, env 5000 Size of specs list for query stats function
VSPHERE_LIMITED_IGNORE_SSL config, env False Ignore the ssl cert on the connection to vsphere host
VSPHERE_LIMITED_FETCH_CUSTOM_ATTRIBUTES config, env False Set to true to collect objects custom attributes as metric labels
VSPHERE_LIMITED_FETCH_TAGS config, env False Set to true to collect objects tags as metric labels
VSPHERE_LIMITED_FETCH_ALARMS config, env False Fetch objects triggered alarms, and in case of hosts hdw alarms as well
VSPHERE_LIMITED_COLLECT_HOSTS config, env True Set to false to disable collection of host metrics
VSPHERE_LIMITED_COLLECT_DATASTORES config, env True Set to false to disable collection of datastore metrics
VSPHERE_LIMITED_COLLECT_VMS config, env True Set to false to disable collection of virtual machine metrics
VSPHERE_LIMITED_COLLECT_VMGUESTS config, env True Set to false to disable collection of virtual machine guest metrics
VSPHERE_LIMITED_COLLECT_SNAPSHOTS config, env True Set to false to disable collection of snapshot metrics

You need to set at least VSPHERE_SECTIONNAME_USER for the section to be detected.

Prometheus configuration

You can use the following parameters in the Prometheus configuration file. The params section is used to manage multiple login/passwords.

  - job_name: 'vmware_vcenter'
    metrics_path: '/metrics'
    static_configs:
      - targets:
        - 'vcenter.company.com'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: localhost:9272

  - job_name: 'vmware_esx'
    metrics_path: '/metrics'
    file_sd_configs:
      - files:
        - /etc/prometheus/esx.yml
    params:
      section: [esx]
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: localhost:9272

# Example of Multiple vCenter usage per #23

- job_name: vmware_export
    metrics_path: /metrics
    static_configs:
    - targets:
      - vcenter01
      - vcenter02
      - vcenter03
    relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: exporter_ip:9272

Current Status

  • vCenter and vSphere 6.0/6.5 have been tested.
  • VM information, Snapshot, Host and Datastore basic information is exported, i.e:
# HELP vmware_snapshots VMware current number of existing snapshots
# TYPE vmware_snapshot_count gauge
vmware_snapshot_timestamp_seconds{vm_name="My Super Virtual Machine"} 2.0
# HELP vmware_snapshot_timestamp_seconds VMware Snapshot creation time in seconds
# TYPE vmware_snapshot_timestamp_seconds gauge
vmware_snapshot_age{vm_name="My Super Virtual Machine",vm_snapshot_name="Very old snaphot"} 1478146956.96092
vmware_snapshot_age{vm_name="My Super Virtual Machine",vm_snapshot_name="Old snapshot"} 1478470046.975632

# HELP vmware_datastore_capacity_size VMware Datastore capacity in bytes
# TYPE vmware_datastore_capacity_size gauge
vmware_datastore_capacity_size{ds_name="ESX1-LOCAL"} 67377299456.0
# HELP vmware_datastore_freespace_size VMware Datastore freespace in bytes
# TYPE vmware_datastore_freespace_size gauge
vmware_datastore_freespace_size{ds_name="ESX1-LOCAL"} 66349694976.0
# HELP vmware_datastore_uncommited_size VMware Datastore uncommitted in bytes
# TYPE vmware_datastore_uncommited_size gauge
vmware_datastore_uncommited_size{ds_name="ESX1-LOCAL"} 0.0
# HELP vmware_datastore_provisoned_size VMware Datastore provisoned in bytes
# TYPE vmware_datastore_provisoned_size gauge
vmware_datastore_provisoned_size{ds_name="ESX1-LOCAL"} 1027604480.0
# HELP vmware_datastore_hosts VMware Hosts number using this datastore
# TYPE vmware_datastore_hosts gauge
vmware_datastore_hosts{ds_name="ESX1-LOCAL"} 1.0
# HELP vmware_datastore_vms VMware Virtual Machines number using this datastore
# TYPE vmware_datastore_vms gauge
vmware_datastore_vms{ds_name="ESX1-LOCAL"} 0.0

# HELP vmware_host_power_state VMware Host Power state (On / Off)
# TYPE vmware_host_power_state gauge
vmware_host_power_state{host_name="esx1.company.com"} 1.0
# HELP vmware_host_cpu_usage VMware Host CPU usage in MHz
# TYPE vmware_host_cpu_usage gauge
vmware_host_cpu_usage{host_name="esx1.company.com"} 2959.0
# HELP vmware_host_cpu_max VMware Host CPU max availability in MHz
# TYPE vmware_host_cpu_max gauge
vmware_host_cpu_max{host_name="esx1.company.com"} 28728.0
# HELP vmware_host_memory_usage VMware Host Memory usage in Mbytes
# TYPE vmware_host_memory_usage gauge
vmware_host_memory_usage{host_name="esx1.company.com"} 107164.0
# HELP vmware_host_memory_max VMware Host Memory Max availability in Mbytes
# TYPE vmware_host_memory_max gauge
vmware_host_memory_max{host_name="esx1.company.com"} 131059.01953125

References

The VMware exporter uses theses libraries:

The initial code is mainly inspired by:

Forked from https://github.com/rverchere/vmware_exporter. I removed the fork so that I could do searching and everything.

Maintainer

Daniel Pryor pryorda

License

See LICENSE file

Known Vulnerabilities

vmware_exporter's People

Contributors

akurach avatar alecorps avatar amcintosh avatar blemmenes avatar dzen avatar erraa avatar finkr avatar gitter-badger avatar gmaurice avatar griesbacher avatar hmmxp avatar hpedrorodrigues avatar jc2k avatar jcpowermac avatar karlism avatar kremers avatar krishofmans avatar m0nsterrr avatar mifrost avatar pryorda avatar rmontenegroo avatar rverchere avatar systemcrash avatar teriand avatar thuvh avatar treydock avatar v-zhuravlev avatar wejdross avatar wtip avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vmware_exporter's Issues

UTF8 Error

Hi,
We have a VM called 'Serveur Modรจle'.
During scraping, the following error occurs:

[2018-09-19 08:48:17.842438+00:00] Start collecting vcenter metrics for vcenter
[2018-09-19 08:48:18.163349+00:00] Starting VM performance metric collection
[2018-09-19 08:48:18.172564+00:00] Total Virtual Machines: 79
[2018-09-19 08:48:24.360088+00:00] Finish starting vm performance vm collection
[2018-09-19 08:48:24.360130+00:00] Starting VM snapshot metric collection
[2018-09-19 08:48:25.045333+00:00] Finished VM snapshot metric collection
[2018-09-19 08:48:25.045381+00:00] Stop collecting vcenter metrics for vcenter
[2018-09-19 08:48:25.053014+00:00] [Failure instance: Traceback: <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\xe8' in position 11: ordinal not in range(128)
/usr/lib/python2.7/dist-packages/twisted/internet/base.py:1208:mainLoop
/usr/lib/python2.7/dist-packages/twisted/internet/base.py:828:runUntilCurrent
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:457:callback
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:565:_startRunCallbacks
--- <exception caught here> ---
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:651:_runCallbacks
/usr/local/bin/vmware_exporter.py:129:generate_latest_metrics
]
Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1208, in mainLoop
    self.runUntilCurrent()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 828, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 457, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 565, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 651, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/bin/vmware_exporter.py", line 129, in generate_latest_metrics
    for k, v in sorted(labels.items())]))
exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xe8' in position 11: ordinal not in range(128)

To fix this issue, I did the following modifications in vmware_exporter.py:
line 128:   k, v.replace('\\', r'\\').replace('\n', r'\n').replace('"', r'\"').encode('utf-8'))
line 139:   request.write(''.join(output))

I don't know if it's the cleanest way to fix the problem, but it works.

UnicodeEncodeError when collecting data

I've got this error messege when collecting data from one vcenter. An other vcenter works fine with the same config.
The Unicode code for u'\xe4' means an "รค" - but there no such character in any name ...

web.Server Traceback (most recent call last):
exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 44: ordinal not in range(128)
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:577 in _runCallbacks

576                    try:
577                        current.result = callback(current.result, *args, **kw)
578                        if current.result is current:

/opt/prometheus/vmware_exporter/vmware_exporter.py:129 in generate_latest_metrics

128                            k, v.replace('\\', r'\\').replace('\n', r'\n').replace('"', r'\"')).decode('ascii')
129                         for k, v in sorted(labels.items())]))
130                else:

exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 44: ordinal not in range(128)

[Feature Request] Add Host Hardware Information

From @hmmxp on February 1, 2018 0:4

Dear All,

Some Suggestion:

Is it possible to include the VMware Host Information Like:

  • ESxi Version, Build
  • Server Manufacturer
  • Server Model
  • Server Service Tag
  • CPU Cores
  • Processor Type
  • VMware Host Physical Network Adapter Transmit Traffic
  • VMware Host Physical Network Adapter Receive Traffic
  • VMware Host Physical Network Adapter Errors
  • VMware Host Physical Network Adapter Dropped Packets

Copied from original issue: rverchere/vmware_exporter#24

Develop true CI/CD Pipeline.

I feel that we we should just be merging to master and tagging the commits with the version. I'm not sure if we need to go release branch -> master.

Feature request: make the vm perf list configurable

Currently the list of metrics collected per each VM is hard-coded, this presents two issues:

  1. additional metrics that may be needed are difficult to add (requires a new version)
  2. (as in our case) we would like to skip some metrics that we don't need and make the scrapes shorter

I think a similar approach to collect_only would work here as well.

WDYT @pryorda?

error while run vmware_exporter command

On Debian GNU/Linux 9.8 (stretch)
When i run command
vmware_exporter -c config.yml
I het this error

Traceback (most recent call last):
  File "/usr/local/bin/vmware_exporter", line 11, in <module>
    load_entry_point('vmware-exporter==0.7.3', 'console_scripts', 'vmware_exporter')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python2.7/dist-packages/vmware_exporter-0.7.3-py2.7.egg/vmware_exporter/vmware_exporter.py", line 218
SyntaxError: 'return' with argument inside generator

Prometheus erro

From @smallfish01 on August 9, 2017 9:36

Hi, When I compled the configuration and restart prometheus service, I got the error as below:
time="2017-08-09T17:34:37+08:00" level=info msg="Loading configuration file prometheus.yml" source="main.go:252"
time="2017-08-09T17:34:37+08:00" level=error msg="Error reading file "/opt/vmware_exporter/config.yml": yaml: unmarshal errors:
line 1: cannot unmarshal !!map into []*config.TargetGroup" source="file.go:199"
time="2017-08-09T17:34:38+08:00" level=error msg="Error reading file "/opt/vmware_exporter/config.yml": yaml: unmarshal errors:
line 1: cannot unmarshal !!map into []*config.TargetGroup" source="file.go:199"

My config.yml configuration is:
default:
vmware_user: '[email protected]'
vmware_password: 'Er4545'
ignore_ssl: True

esx:
vmware_user: 'root'
vmware_password: 'Er4545'
ignore_ssl: True

Do you know why?

Thank you!

Copied from original issue: rverchere/vmware_exporter#13

Test that mocks match requested values

In #81 the values passed to batch_fetch_properties didn't match the mocks (e.g. here).

We could probably modify the tests to make sure we don't update the mocks and not the call to batch_fetch_properties. E.g. something like:

def test_collect_vms():
   ... snip ...

   properties = {
            'vm-1': {
                'name': 'vm-1',
                'runtime.host': vim.ManagedObject('host-1'),
                'runtime.powerState': 'poweredOn',
                'summary.config.numCpu': 1,
                'summary.config.memorySizeMB': 1024,
                'runtime.bootTime': boot_time,
                'snapshot': snapshot,
                'guest.disk': [disk],
            }
   }

   with mock.patch.object(collector, 'batch_fetch_properties') as batch_fetch_properties:
        batch_fetch_properties.return_value = _succeed(properties)
        yield collector._vmware_get_vms(metrics)
        assert _check_properties(batch_fetch_properties.call_args[0][1])

        # Every key in properties['vm-1'] must be in the call to batch_fetch_properties
        assert len(set(properties['vm-1'].keys()) - set(batch_fetch_properties.call_args[0][1])) == 0

getting errors while trying to collect data and scaling

Hi,
I'm getting this errors while running the vmware_exporter container:

Unhandled error in Deferred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
self._startRunCallbacks(fail)
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
self._runCallbacks()
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
_inlineCallbacks(r, g, status)
--- ---
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 421, in host_labels
result = yield threads.deferToThread(lambda: _collect(dc, dc.hostFolder))
File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 421, in
result = yield threads.deferToThread(lambda: _collect(dc, dc.hostFolder))
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 414, in _collect
host_inventory.extend(_collect(dc, folder))
builtins.AttributeError: 'dict' object has no attribute 'extend'

This is the relevant part in the docker-compose file:

vmware_exporter:
image: pryorda/vmware_exporter:latest
restart: always
links:
- prometheus
ports:
- "9272:9272"
environment:
VSPHERE_HOST: 'VCenter.domain.com'
VSPHERE_USER: 'user'
VSPHERE_PASSWORD: 'password'
VSPHERE_IGNORE_SSL: 'True'
VSPHERE_COLLECT_VMS: 'False'
VSPHERE_COLLECT_VMGUESTS: 'False'
VSPHERE_COLLECT_DATASTORES: 'False'
VSPHERE_COLLECT_SNAPSHOTS: 'False'
VSPHERE_COLLECT_HOSTS: 'True'

I'm using VSphere 6.

what am I doing wrong? (as you can see I'm using the latest version).

thank you

'dict' object has no attribute 'extend'

Now getting different error, with different vcenter

`[2019-02-09 20:04:59.231355+00:00] Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 806, in _async_render_GET
yield self.generate_latest_metrics(request)
twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <class 'AttributeError'>: 'dict' object has no attribute 'extend'
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:501:errback
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:1475:gotResult
--- ---
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/python/failure.py:491:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:421:host_labels
/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py:266:
/usr/local/lib64/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib64/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:421:
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:414:_collect
]]
--- ---
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:429:vm_labels
/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py:99:parallelize
]]
--- ---
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:594:_vmware_get_vms
/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py:99:parallelize
]]

Unhandled error in Deferred:

Traceback (most recent call last):
--- ---
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 429, in vm_labels
virtual_machines, host_labels = yield parallelize(self.vm_inventory, self.host_labels)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py", line 99, in parallelize
results = yield defer.DeferredList(args, fireOnOneErrback=True)
twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback: <class 'AttributeError'>: 'dict' object has no attribute 'extend'
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:501:errback
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:1475:gotResult
--- ---
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/python/failure.py:491:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:421:host_labels
/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py:266:
/usr/local/lib64/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib64/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:421:
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:414:_collect
]]

Unhandled error in Deferred:

Traceback (most recent call last):
--- ---
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 429, in vm_labels
virtual_machines, host_labels = yield parallelize(self.vm_inventory, self.host_labels)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py", line 99, in parallelize
results = yield defer.DeferredList(args, fireOnOneErrback=True)
twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback: <class 'AttributeError'>: 'dict' object has no attribute 'extend'
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:501:errback
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:1475:gotResult
--- ---
/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib64/python3.6/site-packages/twisted/python/failure.py:491:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:421:host_labels
/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py:266:
/usr/local/lib64/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib64/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:421:
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:414:_collect
]]

Unhandled error in Deferred:

Traceback (most recent call last):
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
self._startRunCallbacks(fail)
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
self._runCallbacks()
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
_inlineCallbacks(r, g, status)
--- ---
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/usr/local/lib64/python3.6/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 421, in host_labels
result = yield threads.deferToThread(lambda: _collect(dc, dc.hostFolder))
File "/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py", line 266, in
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/local/lib64/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib64/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 421, in
result = yield threads.deferToThread(lambda: _collect(dc, dc.hostFolder))
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 414, in _collect
host_inventory.extend(_collect(dc, folder))
builtins.AttributeError: 'dict' object has no attribute 'extend'

Unhandled error in Deferred:

Traceback (most recent call last):
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
self._startRunCallbacks(fail)
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
self._runCallbacks()
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
_inlineCallbacks(r, g, status)
--- ---
File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/usr/local/lib64/python3.6/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 421, in host_labels
result = yield threads.deferToThread(lambda: _collect(dc, dc.hostFolder))
File "/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/usr/local/lib64/python3.6/site-packages/twisted/python/threadpool.py", line 266, in
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/local/lib64/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib64/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 421, in
result = yield threads.deferToThread(lambda: _collect(dc, dc.hostFolder))
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 414, in _collect
host_inventory.extend(_collect(dc, folder))
builtins.AttributeError: 'dict' object has no attribute 'extend'

`

Release v0.4.2 : Docker Image : Twisted bug

The v0.4.2 Docker image is using Twisted 18.9.0, which apparently shipped with bug:

https://twistedmatrix.com/trac/ticket/9410

This bug causes failed pulls, while spitting out the following traceback:

[2019-01-16 20:45:18.843973+00:00] Starting web server on port 9272
[2019-01-16 20:45:21.586008+00:00] Start collecting metrics from <redacted>
[2019-01-16 20:45:21.697253+00:00] Starting inventory collection
[2019-01-16 20:45:23.220044+00:00] Finished inventory collection
[2019-01-16 20:45:23.220248+00:00] Starting vm metrics collection
[2019-01-16 20:45:23.222406+00:00] Starting host metrics collection
[2019-01-16 20:45:23.352528+00:00] Finished host metrics collection
[2019-01-16 20:45:25.548729+00:00] START: _vmware_get_vm_perf_manager_metrics
[2019-01-16 20:45:26.007193+00:00] START: _vmware_get_vm_perf_manager_metrics: QUERY
[2019-01-16 20:45:30.705149+00:00] FIN: _vmware_get_vm_perf_manager_metrics: QUERY
[2019-01-16 20:45:30.800046+00:00] FIN: _vmware_get_vm_perf_manager_metrics
[2019-01-16 20:45:30.801162+00:00] Finished vm metrics collection
[2019-01-16 20:45:30.834423+00:00] Finished collecting metrics from <redacted>
[2019-01-16 20:45:31.272678+00:00] Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
StopIteration: [<prometheus_client.core.GaugeMetricFamily object at 0x7f69b4d37f98>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4d37e80>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4d37f28>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4d371d0>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69bb6816a0>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e2b0>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e2e8>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e320>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e358>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e390>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e3c8>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e400>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b4e1e438>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd0b8>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd2b0>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd2e8>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd278>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd4a8>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd1d0>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd198>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cd518>, <prometheus_client.core.GaugeMetricFamily object at 0x7f69b48cdb38>]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 686, in _async_render_GET
    yield self.generate_latest_metrics(request)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 733, in generate_latest_metrics
    request.write(output)
  File "/usr/local/lib/python3.6/site-packages/twisted/web/server.py", line 238, in write
    http.Request.write(self, data)
  File "/usr/local/lib/python3.6/site-packages/twisted/web/http.py", line 1122, in write
    self.channel.writeHeaders(version, code, reason, headers)
AttributeError: 'NoneType' object has no attribute 'writeHeaders'


[2019-01-16 20:46:14.382899+00:00] Start collecting metrics from <redacted>
[2019-01-16 20:46:14.493398+00:00] Starting inventory collection
[2019-01-16 20:46:16.002297+00:00] Finished inventory collection
[2019-01-16 20:46:16.002345+00:00] Starting vm metrics collection
[2019-01-16 20:46:16.002777+00:00] Starting host metrics collection
[2019-01-16 20:46:16.122627+00:00] Finished host metrics collection
Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 690, in _async_render_GET
    request.write(b'# Collection failed')
  File "/usr/local/lib/python3.6/site-packages/twisted/web/server.py", line 238, in write
    http.Request.write(self, data)
  File "/usr/local/lib/python3.6/site-packages/twisted/web/http.py", line 1137, in write
    self.channel.writeSequence(toChunk(data))
builtins.AttributeError: 'NoneType' object has no attribute 'writeSequence'

In order to get around the issue, I dropped Twisted's version down a major release to 17.9.0:

$ cat Dockerfile
FROM pryorda/vmware_exporter:v0.4.2

USER root
RUN apk add musl-dev gcc py3-setuptools
RUN pip uninstall -y twisted && \
    pip install twisted==17.9.0

Now everything is running happily.

Below is my config:

$ cat config/vmware-exporter.yml
default:
    vsphere_host: "<redacted>"  # vcenter endpoint
    vsphere_user: "<redacted>"
    vsphere_password: "<redacted>"
    ignore_ssl: True
    collect_only:
        vms: True
        vmguests: True
        datastores: False
        hosts: True
        snapshots: False

vsphere_host query string incompatible with prometheus

The commit linked below, which changed the query string for the vcenter host from target= to vsphere_host=, broke compatibility with Prometheus.

7d79dde#diff-53c0cb48872ead8bf7a02d1c967386c0L75

Prometheus will pass, for example, http://123.123.123.123:9272/metrics?target=myvc.example.com

Using the most recent docker container as of writing, you'll get "No vsphere_host defined!" back.

Otherwise, this works well :) Until that's fixed, we're going to use different sections for different vcenters.

Thanks for your hard work!

Create Chat Place

We need to have a place where we can have instant chat without jumping between tickets.

Cloning VM results in error in exporter

If I clone a Virtual Machine I get no results for the Server.

Error:

[2019-01-11 11:14:11.438130+00:00] Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 692, in _async_render_GET
    yield self.generate_latest_metrics(request)
twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: <class 'KeyError'>: 'runtime.host'
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:460:callback
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1475:gotResult
--- <exception caught here> ---
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1418:_inlineCallbacks
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:438:_vmware_get_vms
]]

Problem is, that during cloning the Field "runtime.host" did not exists.

Unclear values for vmware_vm_cpu_usage_average metric

From @dannyk81 on February 10, 2018 1:53

I can't quite figure out the values of vmware_vm_cpu_usage_average metric, for example:

vmware_vm_cpu_usage_average{instance="<vcenter>",job="vmware-exporter",vm_name="xyz1"} | 202
vmware_vm_cpu_usage_average{instance="<vcenter>",job="vmware-exporter",vm_name="xyz2"} | 225
vmware_vm_cpu_usage_average{instance="<vcenter>",job="vmware-exporter",vm_name="xyz3"} | 4015
vmware_vm_cpu_usage_average{instance="<vcenter>",job="vmware-exporter",vm_name="xyz4"} | 207
vmware_vm_cpu_usage_average{instance="<vcenter>",job="vmware-exporter",vm_name="xyz5"} | 209

according to this https://www.vmware.com/support/developer/converter-sdk/conv61_apireference/cpu_counters.html

The description of this counter is Amount of actively used virtual CPU, as a percentage of total available CPU, but the values I'm seeing do not seem like percentages.

Any clues?

Copied from original issue: rverchere/vmware_exporter#29

object has no attribute 'writeHeaders'

Hi,

on version 0.7.0, centos 7, getting random fail to export, like every 12h 1-3 times with this error below.

Thanks, great software ๐Ÿ‘

During handling of the above exception, another exception occurred:

[2019-02-08 12:14:33.523685+00:00] Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
StopIteration: ((vim.PerformanceManager.EntityMetricBase) [
   (vim.PerformanceManager.EntityMetric) {
      dynamicType = <unset>,
      dynamicProperty = (vmodl.DynamicProperty) [],
      entity = 'vim.VirtualMachine:vm-10212',
      sampleInfo = (vim.PerformanceManager.SampleInfo) [
         (vim.PerformanceManager.SampleInfo) {
            dynamicType = <unset>,
            dynamicProperty = (vmodl.DynamicProperty) [],
            timestamp = 2019-02-08T12:14:20Z,
            interval = 20
         }
      ],
      value = (vim.PerformanceManager.MetricSeries) [
         (vim.PerformanceManager.IntSeries) {
            dynamicType = <unset>,
            dynamicProperty = (vmodl.DynamicProperty) [],
            id = (vim.PerformanceManager.MetricId) {
               dynamicType = <unset>,
               dynamicProperty = (vmodl.DynamicProperty) [],
               counterId = 12,
               instance = ''
            },
            value = (long) [
               22
            ]
         },


Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
StopIteration: [<prometheus_client.core.GaugeMetricFamily object at 0x7fd6519135f8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6519137b8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6519135c0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651913518>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651913748>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651913550>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651913a58>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651913be0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651913630>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529588>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515290b8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515292b0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515292e8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529358>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529390>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515293c8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529400>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529438>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515294a8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515290f0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515294e0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529470>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529748>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6515297f0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529940>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529978>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd651529a20>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d5550>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d5588>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d55c0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d55f8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d5630>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d56a0>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d56d8>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d5780>, <prometheus_client.core.GaugeMetricFamily object at 0x7fd6514d5668>]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 806, in _async_render_GET
    yield self.generate_latest_metrics(request)
  File "/usr/local/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 853, in generate_latest_metrics
    request.write(output)
  File "/usr/local/lib64/python3.6/site-packages/twisted/web/server.py", line 238, in write
    http.Request.write(self, data)
  File "/usr/local/lib64/python3.6/site-packages/twisted/web/http.py", line 1122, in write
    self.channel.writeHeaders(version, code, reason, headers)
AttributeError: 'NoneType' object has no attribute 'writeHeaders'

Issues passing config file using docker

Or I'm just not understanding it....

Running a docker install of Prometheus.
Relevant parts of my docker-compose file:

  prometheus:
    image: prom/prometheus:v2.6.1
    container_name: prometheus
#    deploy:
#      mode: replicated
#      replicas: 2
    volumes:
      - ./prometheus/:/etc/prometheus/
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--web.console.libraries=/etc/prometheus/console_libraries'
      - '--web.console.templates=/etc/prometheus/consoles'
      - '--storage.tsdb.retention=30d'
      - '--web.enable-lifecycle'
      - '--web.enable-admin-api'
    restart: unless-stopped
    ports:
     - "9090:9090"

  vmware-exporter:
    image: pryorda/vmware_exporter
    container_name: vmware-exporter
    restart: unless-stopped
    ports:
      - '9272:9272'
    volumes:
      - ./vmware-exporter/config.yml:/opt/vmware_exporter/config.yml
    command:
      - '-c /opt/vmware_exporter/config.yml'

nano ./vmware-exporter/config.yml

default:
    vsphere_host: '1.2.3.2'
    vsphere_user: 'root'
    vsphere_password: 'password'
    ignore_ssl: True

ESXi-1.2.3.3:
    vsphere_host: '1.2.3.3'
    vsphere_user: 'root'
    vsphere_password: 'password'
    ignore_ssl: True

ESXi-1.2.3.4:
    vsphere_host: '1.2.3.4'
    vsphere_user: 'root'
    vsphere_password: 'password'
    ignore_ssl: True

nano ./prometheus/prometheus.yml

# VSphere ESXi
  - job_name: 'ESXi'
    metrics_path: '/metrics'
    file_sd_configs:
      - files:
        - /etc/prometheus/esx.yml
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: localhost:9272

nano ./prometheus/esx.yml

- targets:
    - 1.2.3.3
  labels:
    name: 'ESXi-1.2.3.3'
    group: 'ESXi Servers'

- targets:
    - 1.2.3.4
  labels:
    name: 'ESXi-1.2.3.4'
    group: 'ESXi Servers'

When running docker-compose up -d, I get the following error.

FileNotFoundError: [Errno 2] No such file or directory: ' /opt/vmware_exporter/config.yml'

So, I comment out:

    command:
      - '-c /opt/vmware_exporter/config.yml'

and add:

environment:
       - VSPHERE_USER=root
       - VSPHERE_HOST=1.2.3.2
       - VSPHERE_PASSWORD=password
       - VSPHERE_IGNORE_SSL=True

just to get things running.

So now, I can exec into the vmware-exporter container and run ls:

sudo docker exec -it vmware-exporter /bin/sh
/opt/vmware_exporter # ls
CHANGELOG.md            MANIFEST.in             config.yml              requirements-tests.txt  setup.py                vmware_exporter
Dockerfile              README.md               dashboards              requirements.txt        systemd
LICENSE                 alerts                  kubernetes              setup.cfg               tests

The config file is there. A cat of the config file confirms it's mapped properly from the bind mount.
So I'm now at a loss as to why it can't find it a start up when I define the complete path in the environment variable.

Everything essentially works. The only issue is that I can only pull stats from one host as defined in the environment variables. I assumed it would walk though all of the targets defined in esx.yml but it's not.
I also assumed that I should have used:

params:
      section: [esx]

in the prometheus.yml file but in doing that, I would only be able to point it at one host which defeats the object of being able to point it at a file using file_sd_configs. I also tried using the section parameter more than once but that failed. Also adding the hosts as an array in the [] failed.

I'm clearly missing something so if anyone would like to enlighten me, I'd appreciate it. :-)

Failure instance: Traceback: <class 'KeyError'>: 'summary.config.numCpu'

Started hitting this in one of my environments:

Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.129410+00:00] Start collecting metrics from <vcenter>
Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.213902+00:00] Starting inventory collection
Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.832281+00:00] Finished inventory collection
Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.832866+00:00] Starting datastore metrics collection
Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.833736+00:00] Starting host metrics collection
Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.834292+00:00] Starting VM metrics collection
Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.875157+00:00] Finished datastore metrics collection
Dec 27 15:01:58 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:58.884837+00:00] Finished host metrics collection
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: [2018-12-27 15:01:59.011146+00:00] [Failure instance: Traceback: <class 'KeyError'>: 'summary.config.numCpu'
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/twisted/internet/base.py:1276:mainLoop
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/twisted/internet/base.py:902:runUntilCurrent
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:460:callback
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: --- <exception caught here> ---
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:687:generate_latest_metrics
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/prometheus_client/exposition.py:63:generate_latest
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/prometheus_client/core.py:97:collect
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:216:collect
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: /usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:442:_vmware_get_vms
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: ]
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: Traceback (most recent call last):
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/twisted/internet/base.py", line 1276, in mainLoop
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: self.runUntilCurrent()
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/twisted/internet/base.py", line 902, in runUntilCurrent
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: call.func(*call.args, **call.kw)
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 460, in callback
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: self._startRunCallbacks(result)
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: self._runCallbacks()
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: --- <exception caught here> ---
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: current.result = callback(current.result, *args, **kw)
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 687, in generate_latest_metrics
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: output = generate_latest(registry)
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/prometheus_client/exposition.py", line 63, in generate_latest
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: for metric in registry.collect():
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/prometheus_client/core.py", line 97, in collect
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: for metric in collector.collect():
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 216, in collect
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: virtual_machines = self._vmware_get_vms(content, metrics, host_inventory)
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 442, in _vmware_get_vms
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: metrics['vmware_vm_num_cpu'].add_metric(labels, row['summary.config.numCpu'])
Dec 27 15:01:59 prometheus-01 docker-vmware-exporter[6744]: builtins.KeyError: 'summary.config.numCpu'

There were no changes in the environment, we are pretty much in a freeze during the holidays, so no new VMs/etc...

Getting metrics from more than one VCenter

Hi,

I'm trying to configure the vmware_exporter to pull metrics from diferent vcenters, but I'm not getting the hang of the configuration on the prometheus server file. Is there any example for me to look?

Thanks,
Bruno

Metrics gathering blocks if connection is timing out

From @ipsecguy on September 5, 2017 11:31

Hi,

I have multiple ESXi Hosts im am scraping. Some of them are switched off most of the time.

It seems that if the vmware_exporter is in the process trying to connect to a currently switched off system it cannot process a second connection.

This can easily be reproduced using a web browser by connecting to a currenlty "off" target and opening a second tab trying to connect to an existent machine. Both fail.

Can the exporter process a single request only or is the problem mybe in the python vware library?

Andreas

Copied from original issue: rverchere/vmware_exporter#15

add exporter runtime metrics

From @wtip on September 12, 2017 21:17

It would be nice to add some exporter runtime metrics.
For example at the moment it would be really useful if I knew how long the exporter is taking to scrape vmware for metrics.

Copied from original issue: rverchere/vmware_exporter#17

Datastore collector

On ESXi 6.7 when enabling Datastore collector it throws:

[2018-10-20 12:04:50.253239+00:00] Start collecting metrics from 192.168.200.234
[2018-10-20 12:04:50.297318+00:00] Starting inventory collection
[2018-10-20 12:04:50.340661+00:00] Finished inventory collection
[2018-10-20 12:04:50.340700+00:00] Starting VM performance metrics collection
[2018-10-20 12:04:50.469131+00:00] Total Virtual Machines: 4
[2018-10-20 12:04:50.471253+00:00] Finished VM performance metrics collection
[2018-10-20 12:04:50.471350+00:00] Starting VM Guests metrics collection
[2018-10-20 12:04:50.564639+00:00] Total Virtual Machines: 4
[2018-10-20 12:04:50.693015+00:00] Finished VM Guests metrics collection
[2018-10-20 12:04:50.693258+00:00] Starting datastore metrics collection
[2018-10-20 12:04:50.740913+00:00] Finished datastore metrics collection
[2018-10-20 12:04:50.741204+00:00] Starting host metrics collection
[2018-10-20 12:04:50.810167+00:00] Finished host metrics collection
[2018-10-20 12:04:50.810702+00:00] Finished collecting metrics from 192.168.200.234
[2018-10-20 12:04:50.828556+00:00] [Failure instance: Traceback: <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'replace'
/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/base.py:1276:mainLoop
/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/base.py:902:runUntilCurrent
/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py:460:callback
/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py:568:_startRunCallbacks
--- <exception caught here> ---
/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py:654:_runCallbacks
build/bdist.linux-x86_64/egg/vmware_exporter/vmware_exporter.py:133:generate_latest_metrics
]

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/base.py", line 1276, in mainLoop
    self.runUntilCurrent()
  File "/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/base.py", line 902, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 460, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.7/site-packages/Twisted-18.9.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "build/bdist.linux-x86_64/egg/vmware_exporter/vmware_exporter.py", line 133, in generate_latest_metrics

exceptions.AttributeError: 'NoneType' object has no attribute 'replace'

SSL: CERTIFICATE_VERIFY_FAILED

When i run vmware_exporter -c config.yml
I see error

ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)

What I need to setup? Maybe i must install certificate of vcenter in prometheus server?

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.5/dist-packages/vmware_exporter-0.7.3-py3.5.egg/vmware_exporter/vmware_exporter.py", line 390, in datastore_labels
    dcs = yield self.datacenter_inventory
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.5/dist-packages/vmware_exporter-0.7.3-py3.5.egg/vmware_exporter/vmware_exporter.py", line 361, in datacenter_inventory
    content = yield self.content
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.5/dist-packages/vmware_exporter-0.7.3-py3.5.egg/vmware_exporter/vmware_exporter.py", line 252, in content
    connection = yield self.connection
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.5/dist-packages/vmware_exporter-0.7.3-py3.5.egg/vmware_exporter/vmware_exporter.py", line 240, in connection
    sslContext=context,
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.5/dist-packages/Twisted-19.2.0rc1-py3.5-linux-x86_64.egg/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.5/dist-packages/pyvmomi-6.7.1.2018.12-py3.5.egg/pyVim/connect.py", line 836, in SmartConnect

  File "/usr/local/lib/python3.5/dist-packages/pyvmomi-6.7.1.2018.12-py3.5.egg/pyVim/connect.py", line 718, in __FindSupportedVersion

  File "/usr/local/lib/python3.5/dist-packages/pyvmomi-6.7.1.2018.12-py3.5.egg/pyVim/connect.py", line 638, in __GetServiceVersionDescription

  File "/usr/local/lib/python3.5/dist-packages/pyvmomi-6.7.1.2018.12-py3.5.egg/pyVim/connect.py", line 604, in __GetElementTree

  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/lib/python3.5/http/client.py", line 1261, in connect
    server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 760, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)

python2 support

vmware-exporter 0.6.5 gets a syntax error whenever I am trying to use python2:

# vmware_exporter -c /etc/default/vmware_exporter.yml 
Traceback (most recent call last):
  File "/usr/local/bin/vmware_exporter", line 7, in <module>
    from vmware_exporter.vmware_exporter import main
  File "/usr/local/lib/python2.7/dist-packages/vmware_exporter/vmware_exporter.py", line 206
    return list(metrics.values())   # noqa: F705
SyntaxError: 'return' with argument inside generator

https://github.com/pryorda/vmware_exporter/blob/ccdf5e9/vmware_exporter/vmware_exporter.py#L206

I found here https://www.pythondoeswhat.com/2017/04/a-return-to-yield_29.html they suggest using python3.
Nevertheless, I am wondering whether python2 will be supported or dropped?

Thank you.

vm metrics error

Error, cannot get vm metrics vmware_vm_mem_usage_average for kr01-maxpatrol-Consolidator-01
Sep  4 12:14:08 exporter vmware_exporter.py[27283]: [2018-09-04 09:13:17.730548+00:00] Error, cannot get vm metrics vmware_vm_net_received_average for kr01-maxpatrol-Consolidator-01
Sep  4 12:14:08 exporter vmware_exporter.py[27283]: [2018-09-04 09:13:17.731369+00:00] Error, cannot get vm metrics vmware_vm_net_transmitted_average for kr01-maxpatrol-Consolidator-01
Sep  4 12:14:08 exporter vmware_exporter.py[27283]: [2018-09-04 09:13:21.162032+00:00] Error, cannot get vm metrics vmware_vm_net_received_average for mingos-fs-01
Sep  4 12:14:08 exporter vmware_exporter.py[27283]: [2018-09-04 09:13:21.172551+00:00] Error, cannot get vm metrics vmware_vm_net_transmitted_average for mingos-fs-01
Sep  4 12:14:08 exporter vmware_exporter.py[27283]: [2018-09-04 09:13:25.448982+00:00] Error, cannot get vm metrics vmware_vm_net_received_average for EISGMU-TEST-BSBA-APP1
Sep  4 12:14:08 exporter vmware_exporter.py[27283]: [2018-09-04 09:13:25.457902+00:00] Error, cannot get vm metrics vmware_vm_net_transmitted_average for EISGMU-TEST-BSBA-APP1

latest pull from git
pyvmomi 6.7.0
prometheus-client 0.0.19
Python 2.7.12

Failed to collect vcenter data

I tried to use this exporter using the docker image 'pryorda/vmware_exporter'.

I want to test the exporter locally with curl:

 curl localhost:9272/metrics?vsphere_host=vcenter02.company.com

I only get a collection error. Viewing the logs I get

[2019-01-03 10:26:23.940622+00:00] Starting web server on port 9272
[2019-01-03 10:26:26.638387+00:00] Start collecting metrics from vcenter02.company.com
[2019-01-03 10:26:26.819656+00:00] Starting inventory collection
[2019-01-03 10:26:26.853162+00:00] Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 686, in _async_render_GET
    yield self.generate_latest_metrics(request)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 725, in generate_latest_metrics
    metrics = yield collector.collect()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 185, in collect
    content,
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 586, in _vmware_get_inventory
    for host in folder.host:
AttributeError: 'vim.Folder' object has no attribute 'host'

My docker-compose file

version: "3"
services:
  vcenter_exporter:
    container_name: vcenter-exporter
    image: pryorda/vmware_exporter:v0.4.2
    ports:
    - 9272:9272
    environment:
      VSPHERE_USER: "monitor"
      VSPHERE_PASSWORD: "p@$$w0rd"
      VSPHERE_IGNORE_SSL: "True"
      VSPHERE_COLLECT_VMS: "False"
      VSPHERE_COLLECT_DATASTORES: "False"
      VSPHERE_COLLECT_VMGUESTS: "False"
      VSPHERE_COLLECT_SNAPSHOTS: "False"

Add "cluster" label for hosts, vms and datastores

From @dannyk81 on February 9, 2018 22:35

It would be great to add "cluster" label to indicate to which HA cluster the host/vm belongs to.

Same, for datastores that belong to a Datastore cluster.

We manage multiple clusters with the same vCenter, and would like to filter metrics per "PROD" and "STAGING" clusters, for example.

Copied from original issue: rverchere/vmware_exporter#28

Metrics for only one host or datastore instance are reported

Helllo,

VMware exporter returns metrics only for one host or datastore instance.

Environment:
vmware_exporter.py v0.3.0
Python 3.5.2 (Ubuntu) & Python 3.6.6 (OpenBSD)
I've checked that all permissions are correct on vSphere side and also tried connecting by using administrator account.

Steps to reproduce:

$ cat config.yml
default:
    vsphere_host: "hostname"   # this is vCenter host
    vsphere_user: 'username'
    vsphere_password: 'password'
    ignore_ssl: True
    collect_only:
        vms: False
        vmguests: False
        datastores: True
        hosts: True
        snapshots: False
$ ./vmware_exporter.py -c config.yml
$ curl -s localhost:9272/metrics | grep -v "#"
vmware_host_power_state{cluster_name="XXX cluster",dc_name="DC_NAME",host_name="vm1.example.com"} 1.0

Applying following patch shows that only first host or datastore is processed in for loop:

--- vmware_exporter.py.orig     2018-12-20 14:11:48.535447407 +0100
+++ vmware_exporter.py  2018-12-20 14:15:59.207702036 +0100
@@ -299,7 +299,9 @@
         """
         log("Starting datastore metrics collection")
         datastores = self._vmware_get_obj(content, [vim.Datastore])
+        print(datastores)
         for datastore in datastores:
+            print(datastore)
             # ds.RefreshDatastoreStorageInfo()
             summary = datastore.summary
             ds_name = summary.name
@@ -461,7 +463,9 @@
         """
         log("Starting host metrics collection")
         hosts = self._vmware_get_obj(content, [vim.HostSystem])
+        print(hosts)
         for host in hosts:
+            print(host)
             summary = host.summary
             host_name, host_dc_name, host_cluster_name = self._vmware_host_metadata(inventory, host)
             host_metadata = [host_name, host_dc_name, host_cluster_name]

Output after applying patch:

$ ./vmware_exporter.py -c config.yml 
[2018-12-20 13:21:36.743208+00:00] Starting web server on port 9272
[2018-12-20 13:21:41.860451+00:00] Start collecting metrics from hostname
[2018-12-20 13:21:42.041749+00:00] Starting inventory collection
[2018-12-20 13:21:43.712368+00:00] Finished inventory collection
[2018-12-20 13:21:43.712878+00:00] Starting datastore metrics collection
[2018-12-20 13:21:43.713506+00:00] Starting host metrics collection
(ManagedObject) [
   'vim.Datastore:datastore-1263',
   'vim.Datastore:datastore-26',
   'vim.Datastore:datastore-29',
   'vim.Datastore:datastore-25',
   'vim.Datastore:datastore-28',
   'vim.Datastore:datastore-6324',
   'vim.Datastore:datastore-1261',
   'vim.Datastore:datastore-369',
   'vim.Datastore:datastore-3412',
   'vim.Datastore:datastore-6169',
   'vim.Datastore:datastore-6167',
   'vim.Datastore:datastore-2064',
   'vim.Datastore:datastore-2065',
   'vim.Datastore:datastore-12841',
   'vim.Datastore:datastore-12847',
   'vim.Datastore:datastore-27354',
   'vim.Datastore:datastore-27349'
]
'vim.Datastore:datastore-1263'
(ManagedObject) [
   'vim.HostSystem:host-1487',
   'vim.HostSystem:host-1315',
   'vim.HostSystem:host-1390',
   'vim.HostSystem:host-1260',
   'vim.HostSystem:host-1460',
   'vim.HostSystem:host-1462',
   'vim.HostSystem:host-24',
   'vim.HostSystem:host-27',
   'vim.HostSystem:host-6322',
   'vim.HostSystem:host-3411',
   'vim.HostSystem:host-367',
   'vim.HostSystem:host-6246',
   'vim.HostSystem:host-6276',
   'vim.HostSystem:host-6255',
   'vim.HostSystem:host-6194',
   'vim.HostSystem:host-6166',
   'vim.HostSystem:host-6205',
   'vim.HostSystem:host-2063',
   'vim.HostSystem:host-12840',
   'vim.HostSystem:host-12846',
   'vim.HostSystem:host-27348',
   'vim.HostSystem:host-27353'
]
'vim.HostSystem:host-1487'
[2018-12-20 13:21:43.828089+00:00] Finished collecting metrics from hostname

add host network metrics

From @wtip on July 21, 2017 18:21

I need some host network metrics so that I can keep an eye on host level network bottlenecks.
I probably would like to see throughput for each nic and not just an aggregate host throughput which seems useless.
I was trying to figure out what the net.throughput.contention.summation metric actually reports as this might be an easier alternative.

Copied from original issue: rverchere/vmware_exporter#12

Failure to run directly, and wont build

[root@c383pc-pmetheus ~]# git clone https://github.com/pryorda/vmware_exporter.git
Cloning into 'vmware_exporter'...
remote: Enumerating objects: 66, done.
remote: Counting objects: 100% (66/66), done.
remote: Compressing objects: 100% (55/55), done.
remote: Total 535 (delta 26), reused 40 (delta 11), pack-reused 469
Receiving objects: 100% (535/535), 154.51 KiB | 0 bytes/s, done.
Resolving deltas: 100% (281/281), done.
[root@c383pc-pmetheus ~]# cd vmware_exporter/
[root@c383pc-pmetheus vmware_exporter]# python vmware_exporter/vmware_exporter.py
  File "vmware_exporter/vmware_exporter.py", line 218
    return list(metrics.values())   # noqa: F705
SyntaxError: 'return' with argument inside generator
[root@c383pc-pmetheus vmware_exporter]# python setup.py install
running install
running bdist_egg
running egg_info
creating vmware_exporter.egg-info
writing requirements to vmware_exporter.egg-info/requires.txt
writing vmware_exporter.egg-info/PKG-INFO
writing top-level names to vmware_exporter.egg-info/top_level.txt
writing dependency_links to vmware_exporter.egg-info/dependency_links.txt
writing entry points to vmware_exporter.egg-info/entry_points.txt
writing manifest file 'vmware_exporter.egg-info/SOURCES.txt'
reading manifest file 'vmware_exporter.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'vmware_exporter.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/vmware_exporter
copying vmware_exporter/__init__.py -> build/lib/vmware_exporter
copying vmware_exporter/defer.py -> build/lib/vmware_exporter
copying vmware_exporter/helpers.py -> build/lib/vmware_exporter
copying vmware_exporter/vmware_exporter.py -> build/lib/vmware_exporter
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/vmware_exporter
copying build/lib/vmware_exporter/__init__.py -> build/bdist.linux-x86_64/egg/vmware_exporter
copying build/lib/vmware_exporter/defer.py -> build/bdist.linux-x86_64/egg/vmware_exporter
copying build/lib/vmware_exporter/helpers.py -> build/bdist.linux-x86_64/egg/vmware_exporter
copying build/lib/vmware_exporter/vmware_exporter.py -> build/bdist.linux-x86_64/egg/vmware_exporter
byte-compiling build/bdist.linux-x86_64/egg/vmware_exporter/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/vmware_exporter/defer.py to defer.pyc
SyntaxError: 'return' with argument inside generator (defer.py, line 100)

byte-compiling build/bdist.linux-x86_64/egg/vmware_exporter/helpers.py to helpers.pyc
byte-compiling build/bdist.linux-x86_64/egg/vmware_exporter/vmware_exporter.py to vmware_exporter.pyc
SyntaxError: 'return' with argument inside generator (vmware_exporter.py, line 218)

creating build/bdist.linux-x86_64/egg/EGG-INFO
copying vmware_exporter.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying vmware_exporter.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying vmware_exporter.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying vmware_exporter.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying vmware_exporter.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying vmware_exporter.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/vmware_exporter-0.7.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing vmware_exporter-0.7.0-py2.7.egg
Removing /usr/lib/python2.7/site-packages/vmware_exporter-0.7.0-py2.7.egg
Copying vmware_exporter-0.7.0-py2.7.egg to /usr/lib/python2.7/site-packages
vmware-exporter 0.7.0 is already the active version in easy-install.pth
Installing vmware_exporter script to /usr/bin

Installed /usr/lib/python2.7/site-packages/vmware_exporter-0.7.0-py2.7.egg
Processing dependencies for vmware-exporter==0.7.0
Traceback (most recent call last):
  File "setup.py", line 20, in <module>
    'vmware_exporter=vmware_exporter.vmware_exporter:main'
  File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/site-packages/setuptools/command/install.py", line 73, in run
    self.do_egg_install()
  File "/usr/lib/python2.7/site-packages/setuptools/command/install.py", line 101, in do_egg_install
    cmd.run()
  File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 380, in run
    self.easy_install(spec, not self.no_deps)
  File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 604, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 655, in install_item
    self.process_distribution(spec, dist, deps)
  File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 706, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 631, in resolve
    requirements.extend(dist.requires(req.extras)[::-1])
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2489, in requires
    dm = self._dep_map
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2474, in _dep_map
    for extra,reqs in split_sections(self._get_metadata(name)):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2964, in split_sections
    for line in yield_lines(s):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2161, in yield_lines
    for ss in strs:
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2503, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1429, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1421, in get_metadata
    return self._get(self._fn(self.egg_info,name))
  File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1486, in _get
    return self.loader.get_data(path)
zipimport.ZipImportError: bad local file header in /usr/lib/python2.7/site-packages/vmware_exporter-0.7.0-py2.7.egg

Develop Contribution Guidelines

As a contributor, we need to have a process and template to follow to make changes flow smoother. This will allow us to approve changes faster and have all required information up front.

Collection failed on vCenter 6.5

Heay

i just wanted to try the exporter with Docker. I created a Vcenter user and added him to the Administrator Group and i see a successful login in the logs.

The vCenter is version 6.5.0 build 9451637 and Manages 2 ESXI which are clustered.
image

docker run -it --rm -p 9272:9272 -e VSPHERE_USER=[email protected] -e VSPHERE_PASSWORD=XXXXX -e VSPHERE_HOST=vcenter -e VSPHERE_IGNORE_SSL=True --name vmware_exporter pryorda/vmware_exporter

[2019-04-29 12:09:28.594283+00:00] Starting web server on port 9272
[2019-04-29 12:09:43.043876+00:00] Start collecting metrics from vcenter
[2019-04-29 12:09:43.043963+00:00] Starting vm metrics collection
[2019-04-29 12:09:43.044022+00:00] Fetching vim.VirtualMachine inventory
[2019-04-29 12:09:43.046407+00:00] Retrieving service instance content
[2019-04-29 12:09:43.049132+00:00] START: _vmware_get_vm_perf_manager_metrics
[2019-04-29 12:09:43.049266+00:00] Starting datastore metrics collection
[2019-04-29 12:09:43.049326+00:00] Fetching vim.Datastore inventory
[2019-04-29 12:09:43.049991+00:00] Starting host metrics collection
[2019-04-29 12:09:43.050515+00:00] Fetching vim.HostSystem inventory
[2019-04-29 12:09:43.135560+00:00] Retrieved service instance content
[2019-04-29 12:09:43.660540+00:00] Fetched vim.VirtualMachine inventory (0:00:00.616470)
[2019-04-29 12:09:43.730291+00:00] Fetched vim.HostSystem inventory (0:00:00.679720)
[2019-04-29 12:09:43.739050+00:00] Fetched vim.Datastore inventory (0:00:00.689666)
[2019-04-29 12:09:43.860414+00:00] Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 818, in _async_render_GET
yield self.generate_latest_metrics(request)
twisted.internet.defer.FirstError: FirstError[#2, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <class 'AttributeError'>: 'vim.Folder' object has no attribute 'datastoreFolder'
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:501:errback
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1475:gotResult
--- ---
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib/python3.6/site-packages/twisted/python/failure.py:512:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:392:datastore_labels
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:266:
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:392:
]]
--- ---
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:493:_vmware_get_datastores
/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py:99:parallelize
]]

Unhandled error in Deferred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
self._startRunCallbacks(fail)
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
self._runCallbacks()
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
_inlineCallbacks(r, g, status)
--- ---
File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 392, in datastore_labels
result = yield threads.deferToThread(lambda: _collect(dc, dc.datastoreFolder))
File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 392, in
result = yield threads.deferToThread(lambda: _collect(dc, dc.datastoreFolder))
builtins.AttributeError: 'vim.Folder' object has no attribute 'datastoreFolder'

Add VM metric to indicate vm tools status

This will be very helpful to ensure vmtools is running on all VMs as required, perhaps we can even include the version.

We're actually hitting an issue form time to time where our terraform plan/apply stalls because some vmtools crashed on some VMs and it's a bit tricky to hunt down which one(s), this would make our lives much easier.

an example is available here

I'll try to prepare a PR for this (if no objections), once we move forward with the more heavier changes in the pipe.

Add unit tests

As a vmware_exporter dev, I believe that it would be best to have unit tests for all of the code. This will allow us to test changes rapidly, enable true CI, and not be as worried when pushing new changes.

This should go in after #44 and will freeze this project to changes until they are done.

@dannyk81 @Jc2k Thoughts?

Docker image - multiple vsphere environments

Hi.

Using the docker image and using the example command line in the from the readme. This is working great however I have 2 platforms that Iโ€™d wish to collect metrics from.

What is the best practice for this requirement using this exporter ?

Cheers.

Pete

Problems starting exporter

Hi All,

Having a few issues starting the exporter;

ubuntu@ukasprom02:~/vmware_exporter-0.6.4/vmware_exporter$ vmware_exporter -c /home/ubuntu/vmware_exporter-0.6.4/vmware_exporter/config.yml
Traceback (most recent call last):
File "/usr/local/bin/vmware_exporter", line 11, in
load_entry_point('vmware-exporter==0.6.4', 'console_scripts', 'vmware_exporter')()
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 480, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2693, in load_entry_point
return ep.load()
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2324, in load
return self.resolve()
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 2330, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/usr/local/lib/python2.7/dist-packages/vmware_exporter-0.6.4-py2.7.egg/vmware_exporter/vmware_exporter.py", line 206

config.yml
default:
vsphere_host: "xxxxxx"
vsphere_user: "xxxxx"
vsphere_password: "xxxxxx"
ignore_ssl: True
collect_only:
vms: False
vmguests: False
datastores: True
hosts: True
snapshots: False

any idea what could be causing this?

Cheers

Pete

Getting Alarm Info

From @l11h on March 1, 2018 13:24

It would be great to get Alarm info for VM's / Hosts
For VM there is an overallStatus option which works but it would be great to write to a label which alarm was executed.
This way we can easily check if we need to deal with it or not / how many alarms there are for the VM

Copied from original issue: rverchere/vmware_exporter#35

Limit metrics to be collected

From @dannyk81 on February 9, 2018 18:7

Hey! first of, thanks for this work ๐Ÿ‘

We are now testing this exporter in our environment, one thing we noticed that it takes ~1m32.703s to scrape the vcenter metrics, probably as our environment is quite big.

I wonder if it's possible to limit which metrics should be collected? for example if we only want datastore metrics?

Copied from original issue: rverchere/vmware_exporter#27

VCSA 6.7.0.2 no metrics

When using VMware-vCenter-Server-Appliance-6.7.0.20000-10244745-patch-FP.iso (latest version) the docker container is unable to produce metrics for export.

`
/usr/bin/docker run -it -p 9272:9272 -e VSPHERE_USER=[email protected] -e VSPHERE_PASSWORD='mypass' -e VSPHERE_HOST='10.0.0.xx' -e VSPHERE_IGNORE_SSL=True --name vmware_exporter pryorda/vmware_exporter
[2019-02-14 19:44:00.540658+00:00] Starting web server on port 9272
[2019-02-14 19:44:01.513733+00:00] Start collecting metrics from 10.0.0.99
[2019-02-14 19:44:01.513832+00:00] Starting vm metrics collection
[2019-02-14 19:44:01.513909+00:00] Fetching vim.VirtualMachine inventory
[2019-02-14 19:44:01.517189+00:00] Retrieving service instance content
[2019-02-14 19:44:01.519459+00:00] START: _vmware_get_vm_perf_manager_metrics
[2019-02-14 19:44:01.520242+00:00] Starting datastore metrics collection
[2019-02-14 19:44:01.520908+00:00] Fetching vim.Datastore inventory
[2019-02-14 19:44:01.522654+00:00] Starting host metrics collection
[2019-02-14 19:44:01.523283+00:00] Fetching vim.HostSystem inventory
[2019-02-14 19:44:01.633741+00:00] Retrieved service instance content
[2019-02-14 19:44:02.089670+00:00] Fetched vim.VirtualMachine inventory (0:00:00.575705)
[2019-02-14 19:44:02.100404+00:00] Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 766, in _async_render_GET
yield self.generate_latest_metrics(request)
twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#0, [Failure instance: Traceback: <class 'TypeError'>: Required field "querySpec" not provided (not @optional)
/usr/local/lib/python3.6/threading.py:916:_bootstrap_inner
/usr/local/lib/python3.6/threading.py:864:run
/usr/local/lib/python3.6/site-packages/twisted/_threads/_threadworker.py:46:work
/usr/local/lib/python3.6/site-packages/twisted/_threads/_team.py:190:doWork
--- ---
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:266:
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py:706:
/usr/local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py:511:_InvokeMethod
/usr/local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py:1041:CheckField
]]
--- ---
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:554:_vmware_get_vm_perf_manager_metrics
/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py:99:parallelize
]]

[2019-02-14 19:44:02.131425+00:00] Fetched vim.Datastore inventory (0:00:00.609866)
[2019-02-14 19:44:02.138566+00:00] Fetched vim.HostSystem inventory (0:00:00.614648)
[2019-02-14 19:44:02.208135+00:00] Finished datastore metrics collection
[2019-02-14 19:44:02.262205+00:00] Finished vm metrics collection
[2019-02-14 19:44:02.263507+00:00] Finished host metrics collection
[2019-02-14 19:44:16.512584+00:00] Start collecting metrics from 10.0.0.99
[2019-02-14 19:44:16.512666+00:00] Starting vm metrics collection
[2019-02-14 19:44:16.512727+00:00] Fetching vim.VirtualMachine inventory
[2019-02-14 19:44:16.512795+00:00] Retrieving service instance content
[2019-02-14 19:44:16.513403+00:00] START: _vmware_get_vm_perf_manager_metrics
[2019-02-14 19:44:16.513966+00:00] Starting datastore metrics collection
[2019-02-14 19:44:16.514041+00:00] Fetching vim.Datastore inventory
[2019-02-14 19:44:16.514330+00:00] Starting host metrics collection
[2019-02-14 19:44:16.514741+00:00] Fetching vim.HostSystem inventory
[2019-02-14 19:44:16.585087+00:00] Retrieved service instance content
[2019-02-14 19:44:17.031539+00:00] Fetched vim.VirtualMachine inventory (0:00:00.518771)
[2019-02-14 19:44:17.057751+00:00] Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 766, in _async_render_GET
yield self.generate_latest_metrics(request)
twisted.internet.defer.FirstError: FirstError[#1, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#0, [Failure instance: Traceback: <class 'TypeError'>: Required field "querySpec" not provided (not @optional)
/usr/local/lib/python3.6/threading.py:916:_bootstrap_inner
/usr/local/lib/python3.6/threading.py:864:run
/usr/local/lib/python3.6/site-packages/twisted/_threads/_threadworker.py:46:work
/usr/local/lib/python3.6/site-packages/twisted/_threads/_team.py:190:doWork
--- ---
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:266:
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py:706:
/usr/local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py:511:_InvokeMethod
/usr/local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py:1041:CheckField
]]
--- ---
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:554:_vmware_get_vm_perf_manager_metrics
/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py:99:parallelize
]]
`

Performance impact of refreshing each datastore info at every scrape

From @wtip on September 12, 2017 21:47

I've been running into an issue where one of my esxi hosts appears to be unreachable by the exporter.
I think what's actually happening is that the scrape is timing out.

vmware-exporter-scrape

The jump in scrape duration coincides with about the time that I deployed this code change #16
and it looks like I might be bumping up against my default 10 second timeout.
I've increased my timeout but I'm not sure if this is a great solution for everyone. Maybe the datastore refresh should be a configurable option.

Copied from original issue: rverchere/vmware_exporter#18

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.