perfwatcher / collectm Goto Github PK

Hi,

I have collectm setup and sending to a collectd concentrator, which mostly works fine, but recently I've started experiencing high CPU utilisation on the collectd concentrator, and the collectm data stops being received. I only have collectm clients sending to the collectd network plugin.

Syslog is filling up with:

Aug 12 12:21:57 app1 collectd[1371]: network plugin: parse_packet: Received truncated packet, try increasing `MaxPacketSize'
Aug 12 12:21:57 app1 collectd[1371]: network plugin: parse_packet: Received truncated packet, try increasing `MaxPacketSize'
Aug 12 12:21:57 app1 collectd[1371]: network plugin: parse_part_string: Output buffer too small.
Aug 12 12:21:57 app1 collectd[1371]: plugin_dispatch_values: Dataset not found: nbcpu (from "LMSSYDCW1/cpu-total/nbcpu"), check your types.db!

I have increased MaxPacketSize in the collectd config to 65535 but that hasn't made any difference.

Does anyone have any thoughts on this?

Hello,

I am currently using collectm in a project for my company, and I had some issues when using performance counters to gather data. If you put 2 or more counters that does not exist, collectm will scramble data, and some counters that does not exist will be monitored while some that exist and should be monitored have no data or get wrong data. After reading the code of the TypePerf module, I found two causes for this issue:

• The first one is the behaviour of the TypePerf command you're using to gather data from performance counters: if you put multi-instance counters that exist but for an instance that does not exist, TypePerf will not tell you that it is sending data for this counter. Here is an example:

• The second is the fact that the TypePerf module is using indexes to remove counters when they are not available. When removing the counters, it is not taking into account the fact that it is removing counters, and thus changing the indexes of the counters after the one that has been removed. Here is a schema to illustrate that:

I already found how to fix those issues, and if you are ok with that, I can try to push my fix into your repo.

Using influxdb as backend to push perfmon metrics and the series name is separated with "/" which is less than ideal . is it possible to change that to say "."?

I'm trying out collectm on a windows 2008 server. In collectdCompat.js I see that it collects cpu, mem, df, disk, interface, load, uptime, process and swap. However in the collectd server receiving the data sent from collectm, I only get disk, df, cpu and interface. Am I missing something here?

Collectd supports plugins that write to the network in different protocols. The protocol I need for my use case is write_http, but it would make sense for collectm to support plugins for writing stuff. I already have some ideas of how to add this in, but I'm just creating this issue to let everybody know that I'm planning to work on this.

When I go on http://localhost:25826, when I click on "Stop/Restart", the service does not restart.

I noticed that there are 2 node.exe processes for the service : wrapper.js (father) and collectw (child) itself. When I click on restart, the process collectw dies. But wrapper.js does not detect that collectw died and nothing happen.

The problem can be in node_modules\node-windows\lib\wrapper.js or in the way collectw dies.

When running CollectM for extended periods, memory consumed by the node.exe application increases continuously over time. In one case of a server that's been running for about 2 weeks, it is consuming nearly 2GB of RAM. What could possibly be causing this?

I am attempting to install this on a Win Svr 2008 R2 SP1 Server and when you run the installer it does not install the service.

If I attempt to run node on service.js with the arg of "install" it reports that the service is already installed. If I do anything other than "install" or "installandstart" it reports nothing and the service is not installed.

Is there a way to get this sending data to StatsD?

Hello, I can't get CollectM 1.6.0 to work on either Server 2008 or 2012. Crashes like this:

2016-11-04T07:50:13.741Z - info: Collectm version 1.6.0
2016-11-04T07:50:13.741Z - info: Collectm is starting
2016-11-04T07:50:13.756Z - info: Using configuration files in C:\Program Files\CollectM\config
2016-11-04T07:50:13.756Z - info: Sending metrics to Collectd with hostname bns (case sensitive).
2016-11-04T07:50:13.756Z - info: Sending metrics to Collectd localhost:25826.
2016-11-04T07:50:13.772Z - info: Default interval is 60000 ms
2016-11-04T07:50:13.788Z - info: Plugin collectdCompat : reInit done
2016-11-04T07:50:13.788Z - info: Plugin sysconfig : reInit done
2016-11-04T07:50:13.788Z - info: Plugin perfmon : reInit done
2016-11-04T07:50:13.788Z - info: Plugin process : reInit done
2016-11-04T07:50:13.788Z - info: Plugin df : reInit done
2016-11-04T07:50:13.788Z - info: Plugin disk : reInit done
2016-11-04T07:50:13.788Z - info: Plugin load : reInit done
2016-11-04T07:50:13.788Z - info: Plugin collectdCompat : reloadConfig done
2016-11-04T07:50:13.788Z - info: Plugin sysconfig : reloadConfig done
2016-11-04T07:50:13.788Z - info: Plugin perfmon : reloadConfig done
2016-11-04T07:50:13.788Z - info: Plugin process : reloadConfig done
2016-11-04T07:50:13.788Z - info: Plugin df : reloadConfig done
2016-11-04T07:50:13.788Z - info: Plugin disk : reloadConfig done
2016-11-04T07:50:13.788Z - info: Plugin load : reloadConfig done
2016-11-04T07:50:13.803Z - info: Enabling CollectdCompat cpu module
2016-11-04T07:50:13.819Z - info: Enabling CollectdCompat memory module
2016-11-04T07:50:13.819Z - info: Enabling CollectdCompat interface module
2016-11-04T07:50:13.834Z - info: Enabling CollectdCompat uptime module
2016-11-04T07:50:13.834Z - info: Enabling CollectdCompat process module
2016-11-04T07:50:13.834Z - info: Enabling CollectdCompat module
2016-11-04T07:50:13.834Z - info: Plugin collectdCompat : monitoring enabled
2016-11-04T07:50:13.834Z - info: Plugin sysconfig : monitoring enabled
2016-11-04T07:50:13.850Z - info: Plugin perfmon : monitoring enabled
2016-11-04T07:50:13.850Z - info: Plugin process : monitoring enabled
2016-11-04T07:50:13.850Z - info: Df plugin monitoring: _Total
2016-11-04T07:50:13.850Z - info: Autodiscover is turned on for df plugin.
2016-11-04T07:50:13.850Z - info: Plugin df : monitoring enabled
2016-11-04T07:50:13.850Z - info: Disk plugin monitoring: _Total
2016-11-04T07:50:13.850Z - info: Autodiscover is turned on for disk plugin.
2016-11-04T07:50:13.850Z - info: Plugin disk : monitoring enabled
2016-11-04T07:50:13.850Z - info: Plugin load : monitoring enabled
2016-11-04T07:50:13.913Z - info: Enabled httpconfig server
2016-11-04T07:50:13.913Z - info: Log files will be deleted after 30 days
2016-11-04T07:50:14.163Z - error: uncaughtException: Cannot read property 'counters' of undefined
{ date: 'Fri Nov 04 2016 08:50:14 GMT+0100 (Střední Evropa (běžný čas))',
process:
{ pid: 1160,
uid: null,
gid: null,
cwd: 'C:\Program Files\CollectM\bin',
execPath: 'C:\Program Files\CollectM\bin\node.exe',
version: 'v0.12.7',
argv:
[ 'C:\Program Files\CollectM\bin\node.exe',
'C:\Program Files\CollectM\lib\collectm.js' ],
memoryUsage: { rss: 33513472, heapTotal: 29534208, heapUsed: 13555320 } },
os: { loadavg: [ 0, 0, 0 ], uptime: 7410604.3798497 },
trace:
[ { column: 32,
file: 'Files\CollectM\plugins\collectdCompat.js',
function: 'PerfmonStream. (C:\Program',
line: 98,
method: ' (C:\Program',
native: false },
{ column: 17,
file: 'events.js',
function: 'PerfmonStream.emit',
line: 107,
method: 'emit',
native: false },
{ column: 13,
file: 'Files\CollectM\node_modules\perfmon\perfmon.js',
function: 'C:\Program',
line: 167,
method: null,
native: false },
{ column: 3,
file: 'Files\CollectM\node_modules\perfmon\lib\TypePerf.js',
function: 'ChildProcess. (C:\Program',
line: 65,
method: ' (C:\Program',
native: false },
{ column: 17,
file: 'events.js',
function: 'ChildProcess.emit',
line: 110,
method: 'emit',
native: false },
{ column: 12,
file: 'child_process.js',
function: 'Process.ChildProcess._handle.onexit',
line: 1074,
method: 'ChildProcess._handle.onexit',
native: false } ],
stack:
[ 'TypeError: Cannot read property 'counters' of undefined',
' at PerfmonStream. (C:\Program Files\CollectM\plugins\collectdCompat.js:98:32)',
' at PerfmonStream.emit (events.js:107:17)',
' at C:\Program Files\CollectM\node_modules\perfmon\perfmon.js:167:13',
' at ChildProcess. (C:\Program Files\CollectM\node_modules\perfmon\lib\TypePerf.js:65:3)',
' at ChildProcess.emit (events.js:110:17)',
' at Process.ChildProcess._handle.onexit (child_process.js:1074:12)' ] }

===========================================

Test config that works on Server 2003 and Windows 7:

{
"Hostname":"bns", // should not be set in default.json
"HostnameCase": "lower", // "lower", "upper" or "default"

"Interval": 60,

// Cryptography/Security settings
"Crypto": {
"SecurityLevel": 0,
"Username": "alice",
"Password": "12345678"
},

// Time To Live : will stop Collectm after this value (86400sec = 1 day).
// This is useful if you think there is a memory leak and want to reboot periodically
// Collectm. Stop Collectm after 86400sec and Windows will restart the service.
// Of course we do our best to avoid memory leaks.
// Note : values < 60 will be ignored : no stop. This is the default.

// "CollectmTimeToLive": 86400,

// Every day, remove old logs (based on modified time).
// If unset, or if set to 0, no logs will be deleted.
"LogDeletionDays": 30,

"HttpConfig": {
"enable": 1,
"listenPort": 25826,
"login": "admin",
"password": "admin"
},

"Network": {
"servers": [
{
"hostname": "collector",
"port": 25826
}
]
},

"Plugin": {
"collectdCompat": {
"enable": 1,
"modules": {
// This is an example. You can select which of the modules of the collectdCompat plugin you want to be active.
// Available modules are: load, disk, cpu, memory, uptime, df, interface, process, swap.
// Set to 0 to deactivate, 1 or don't set at all to activate.
// If you want all to be active remove "modules" value completely.
"load": 1,
"disk": 1
}
},
"sysconfig": {
"enable": 1
},
"perfmon": {
"enable": 1,
"counters" : [
// This is an example :
{
"counter": "\LogicalDisk(C:)\% Free Space",
"enable": 1,
"plugin": "perfmon_LogicalDisk",
"plugin_instance": "C",
"type": "percent",
"type_instance": "Free Space"
}
]
},
"process": {
"enable": 1,
"process": {
// The key is not used in Collectm. It only helps for config overwrite in local.json
// The value is an array with "plugin", "instance" and "commandline".

// This is an example :
"My Collectm": {
"plugin": "process",
"instance": "collectm",
"commandline": ".*node.collectm\.js."
},
"My NSSM for Collectm": {
"plugin": "process",
"instance": "nssm",
"commandline": ".*collectm.nssm\.exe."
}
}
},
"df": {
"enable": 1,
"autodiscover": 1,
// This is an example.
// Plugin can either autodiscover all the disks mounted on the system
// or point to a particular disk like below.
// If set to autodiscover it will also find new disks that are mounted after starting the service.
// Autodiscover will override the disks setting if both are set.
// If disk is no longer available no data will be sent.
"disks": {
"total" : 1
}
},
"disk": {
"enable": 1,
"autodiscover": 1,
// This is an example.
// Plugin can either autodiscover all the disks mounted on the system
// or point to a particular disk like below.
// If set to autodiscover it will also find new disks that are mounted after starting the service.
// Autodiscover will override the disks setting if both are set.
// If disk is no longer available no data will be sent.
"disks": {
"total" : 1
}
},
"load": {
"enable": 1
}
}
}

OS in this specific case is Windows Server 2012 R2 with CZ locales.

Please, is there a way to get it working?

Collectd's network plugin supports authentication and encryption. We're working on adding crypto support for collectm on https://github.com/perfwatcher/collectm/compare/master...mistio:crypto?w=1 . Is this a feature you'd be interested in merging, once completed? Any thoughts/comments/suggestions on the approach?

(Some notes of mine on collectd's crypto specs: https://gist.github.com/dimrozakis/1b326ac4ee7d3682b7dc)

Hello, i'm trying to deal with
collectm on Win7 and collectd on ubuntu 16 -
the main issue, after i'm enabling crypto -

    "SecurityLevel": 2,
    "Username": "client1",
    "Password": "testtest"

Server unable to get any data from client (w\o crypto everything is fine):

Jul 17 22:19:12 v1 collectd[31256]: network plugin: parse_packet: Received truncated packet, try increasing `MaxPacketSize'
Jul 17 22:19:12 v1 collectd[31256]: [2017-07-17 22:19:12] network plugin: parse_packet: Received truncated packet, try increasing `MaxPacketSize'
Jul 17 22:19:21 v1 collectd[31256]: network plugin: Unencrypted packet or part has been ignored.
Jul 17 22:19:21 v1 collectd[31256]: [2017-07-17 22:19:22] network plugin: Unencrypted packet or part has been ignored.
Jul 17 22:19:21 v1 collectd[31256]: [2017-07-17 22:19:22] network plugin: Unencrypted packet or part has been ignored.
Jul 17 22:19:21 v1 collectd[31256]: network plugin: Unencrypted packet or part has been ignored.

Otherwise, in config i set max packet :

<Plugin "network">
       <Listen "192.168.1.105" "1234">
               SecurityLevel  "Encrypt"
               AuthFile "/etc/collectd/passwd"
             MaxPacketSize 65535
       </Listen>
</Plugin>

Client v - CollectM-1.6.0.install (with latest colletdout module)
Server v - 5.7.2.6.g02a2996-1~xenial

Could you please suggest me something to check ?

I have multiple instances of this appearing in syslog:

Feb 10 14:39:36 mail collectd[19646]: plugin_dispatch_values: Dataset not found: nbcpu (from "pc.hostname.local/cpu-total/nbcpu"), check your types.db!

What's the best approach to fix? Should types.db be edited, or can we make an adjustment to CollectM so that users don't need to edit the types.db file?

I'm noticing something strange. It seems that for each network interface, the rx and tx metrics sample the same data source. I tried downloading a large file from the monitored server, and both rx and tx dataseries displayed exactly the same spikes. Could this be a bug in collectm's collectdCompat plugin? It seems like the interface metrics (if_octets, if_packets) send the rx values properly as rx, but incorrectly as tx as well.