JMX2Graphite not exposing connector and task status metrics about jmx2graphite HOT 17 CLOSED

@mbairagi85 Please re-run it , this time with TRACE log level. It should print out the metrics it fetched. It's a little more hint in the direction.

from jmx2graphite.

According to Graphite docs, it can't store string metrics - "Store numeric time-series data".
Therefore, we won't issue the above fix.

from jmx2graphite.

@Yatharth0045 Jmx2graphite filters metrics only if you set the BLACK_LIST_REGEX or WHITE_LIST_REGEX parameters.
Try to run the agent in debug mode and see if you get any "Failed" messages.
Could you please share the Dockerfile you used to build the image?

from jmx2graphite.

@yyyogev , Here's the Docker file. I have downloaded the jar on my local and build this Dockerfile

FROM confluentinc/cp-kafka-connect:4.0.0
COPY jmx2graphite-1.4.1-javaagent.jar /

Also, I haven't provided any WHITE_LIST_REGEX and BLACK_LIST_REGEX

from jmx2graphite.

@Yatharth0045 did you try to run the agent in debug mode and see if you get any "Failed" messages?

from jmx2graphite.

Yes @yyyogev
I even tried with LOG_LEVEL as debug and got the following logs

19/12/16 07:31:23 INFO jmx2graphite.MetricsPipeline: Found 117 metric beans. Time = 8ms, for 2019-12-16T07:31:00.000+0000
19/12/16 07:31:23 INFO jmx2graphite.MetricsPipeline: metrics fetched. Time: 32 ms; Metrics: 1048
19/12/16 07:31:23 DEBUG jmx2graphite.PickledGraphite: Wrote 400 metrics
19/12/16 07:31:23 DEBUG jmx2graphite.PickledGraphite: Wrote 400 metrics
19/12/16 07:31:23 DEBUG jmx2graphite.PickledGraphite: Wrote 248 metrics
19/12/16 07:31:23 INFO jmx2graphite.MetricsPipeline: metrics sent to Graphite. Time: 9 ms, Failed metrics: 0

By the way, @yyyogev I'm not getting only the specified metrics of kafka.

Also, regarding the Blacklisting metrics property, I've tried the jmx2graphite 3.2.0 version also, but still didn't able to get those metrics.

from jmx2graphite.

Also, According to jmx2graphite README, I've updated the MAX_CREATES_PER_MINUTE and MAX_UPDATES_PER_SECOND configuration to 10000 instead of default.

from jmx2graphite.

@yyyogev , I am facing the same issue as @Yatharth0045. On stackoverflow, I found a similar thread. Can anyone provide pointers to address the issue?

from jmx2graphite.

Could you share the steps to reproduce? Dockerfile?

from jmx2graphite.

Do any of you guys using logz.io?
We have a much more stable tool to extract jmx metrics to logz.io - Jmx2Logzio
Try and see if you can get those metrics with this tool, then we'll know for sure that there is something wrong.

from jmx2graphite.

Thanks @asafm. This was helpful for debugging. And yes, jmx2graphite is not able to fetch the required metrics.

from jmx2graphite.

Could you share the steps to reproduce? Dockerfile?

@yyyogev I've already shared the Dockerfile above in the thread in which I've just added the jmx2graphite.jar in the kafka connect docker and passed its config as KAFKA_OPTS variable as follows

KAFKA_OPTS="-javaagent:jmx2graphite.jar=GRAPHITE_HOSTNAME=172.17.0.6,GRAPHITE_PORT=2004,SERVICE_HOST=service,SERVICE_NAME=kafka-connect,LOG_LEVEL=DEBUG"

from jmx2graphite.

And yes, jmx2graphite is not able to fetch the required metrics.

@Yatharth0045 What do you mean? Do you see those metrics with a failed message or can't see them at all?

from jmx2graphite.

Yes @yyyogev, I can't see them at all.
Have you tried to reproduce the issue? Were you able to fetch those metrics?

from jmx2graphite.

I have but I couldn't manage to raise the whole kafka-zookeeper-graphite architecture.
Anyway, if you don't see the metrics at all with TRACE mode, it's not being exposed. This means that for your configuration, these metrics are unavailable/irrelevant.
Have you tried to observe these metrics with an external tool? Such as Jconsole, Hawtio?

from jmx2graphite.

Yes @yyyogev, I'm able to get those metrics via Jconsole. Also, I have observed that all the metrics which I'm not able to fetch has some string value and not a numeric value. Maybe that's why JMX2Graphite wasn't able to pick and store them in graphite.

from jmx2graphite.

That has to be it. Our metric base unit is "name to a number" although it's super weird that you don't get any failed messages..
I'll try and release a fix for that in a few days

from jmx2graphite.