Giter Club home page Giter Club logo

jmxmon's Introduction

jmxmon 简介

jmxmon是一个基于open-falcon的jmx监控插件,通过这个插件,结合open-falcon agent,可以采集任何开启了JMX服务端口的java进程的服务状态,并将采集信息自动上报给open-falcon服务端

主要功能

通过jmx采集java进程的jvm信息,包括gc耗时、gc次数、gc吞吐、老年代使用率、新生代晋升大小、活跃线程数等信息。

对应用程序代码无侵入,几乎不占用系统资源。

环境需求

Linux

JDK>=1.6

Open-Falcon>=0.0.5

目标java进程开启jmx端口

jmxmon部署

  1. 安装并启动open-falcon agent
  2. 下载并解压编译好的 release包 到目标安装目录下
  3. cp conf.example.properties conf.properties
  4. 修改conf.properties配置文件,一般情况下只需要将jmx.ports的端口号配置上就可以了
  5. sh control start
  6. sh control tail查看日志,或者cat var/app.log以确认程序是否正常启动

配置说明

配置文件默认文件名为conf.properties,内容说明如下:

# 工作目录用来存放jmxmon的临时缓存文件,注意不要修改此目录下的文件
workDir=./

# 需要监听的本地jmx端口,支持监听多个端口,多端口用逗号分隔
jmx.ports=10000,10001,10002,10003

# 本地agent的上报url,如果使用open-falcon的默认配置,则这里不需要改变
agent.posturl=http://localhost:1988/v1/push

# 可选项:上报给open-falcon的endpoint,默认值为本机hostname。不建议修改
#hostname=

# 可选项:上报给open-falcon的上报间隔,默认值60,单位秒。不建议修改
#step=

采集指标

Counters Type Notes
parnew.gc.avg.time GAUGE 一分钟内,每次YoungGC(parnew)的平均耗时
concurrentmarksweep.gc.avg.time GAUGE 一分钟内,每次CMSGC的平均耗时
parnew.gc.count GAUGE 一分钟内,YoungGC(parnew)的总次数
concurrentmarksweep.gc.count GAUGE 一分钟内,CMSGC的总次数
gc.throughput GAUGE GC的总吞吐率(应用运行时间/进程总运行时间)
new.gen.promotion GAUGE 一分钟内,新生代的内存晋升总大小
new.gen.avg.promotion GAUGE 一分钟内,平均每次YoungGC的新生代内存晋升大小
old.gen.mem.used GAUGE 老年代的内存使用量
old.gen.mem.ratio GAUGE 老年代的内存使用率
thread.active.count GAUGE 当前活跃线程数
thread.peak.count GAUGE 峰值线程数

建议设置监控告警项

不同应用根据其特点,可以灵活调整触发条件及触发阈值

告警项 触发条件 备注
gc.throughput all(#3)<98 gc吞吐率低于98%,影响性能
old.gen.mem.ratio all(#3)>90 老年代内存使用率高于90%,需要调优
thread.active.count all(#3)>500 线程数过多,影响性能

jmxmon's People

Contributors

toomanyopenfiles avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jmxmon's Issues

how to handle jmx monitor with authentication error

The java process was started with imx user and password, which resulted in jmx monitor error with authentication error, is there any parameter in property file indicating for authentication use?

thanks in advance

jmxmon里监控指标有总线程数吗

请问thread.active.count是只包含NEW和RUNNABLE这两种状态吗?
另外就是这个thread.peak.count(峰值线程数)没明白是什么时间的峰值?

error:non-JRMP server at remote endpoint

2017-03-20 16:01:57,241 [main] INFO [com.stephan.tof.jmxmon.Config] - init config
2017-03-20 16:01:57,370 [main] INFO [com.stephan.tof.jmxmon.Config] - /usr/local/falcon-agent/jmxmon/./jmxmon.jvm.context.json is not exist
2017-03-20 16:01:57,370 [main] INFO [com.stephan.tof.jmxmon.Config] - init config ok
2017-03-20 16:01:58,262 [pool-1-thread-1] ERROR [com.stephan.tof.jmxmon.JMXMonitor] - non-JRMP server at remote endpoint
java.rmi.ConnectIOException: non-JRMP server at remote endpoint
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:248)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.checkSslConfig(ProxyClient.java:274)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.(ProxyClient.java:167)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.getProxyClient(ProxyClient.java:518)
at com.stephan.tof.jmxmon.JMXMonitor.runTask(JMXMonitor.java:58)
at com.stephan.tof.jmxmon.JMXMonitor.access$0(JMXMonitor.java:50)
at com.stephan.tof.jmxmon.JMXMonitor$1.run(JMXMonitor.java:41)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

这大概什么原因?

这个插件有个Bug,就是当jvm使用G1的垃圾收集器的时候

这个插件有个Bug,就是当jvm使用G1的垃圾收集器的时候,吐出的指标带有空格,导致open-falcon 识别不了,比如这个指标:"metric":"g1 young generation.gc.avg.time",里面有空格

下面是JVM采用G1的时候,jmx吐出的指标详情 :
content=[{"metric":"g1 young generation.gc.avg.time","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":0.0,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"g1 young generation.gc.count","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":0.0,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"g1 old generation.gc.avg.time","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":0.0,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"g1 old generation.gc.count","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":-1.0,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"gc.throughput","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":99.99,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"old.gen.mem.used","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":3.53209116E9,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"old.gen.mem.ratio","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":10.28,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"new.gen.promotion","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":0.0,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"new.gen.avg.promotion","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":0.0,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"thread.active.count","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":219.0,"counterType":"GAUGE","tags":"jmxport=9998"},{"metric":"thread.peak.count","endpoint":"UAT-HADOOP-13","timestamp":1565224431,"step":60,"value":349.0,"counterType":"GAUGE","tags":"jmxport=9998"}]

上报的metrics不完整问题

我一个机器上开了两个jmx的java进程(一个端口1099,另一个12345),看日志发现,上报的metrics两个端口不一致。
举例(看截图,截图是jmxmon的一条上报日志,为方便期间,多打了几个回车而已):
parnew.gc.avg.time 这个metrics只有1099端口有,而12345端口没有;
gc.throughput这个metrics 两个端口都有;

这是为什么?还是存在bug?
image

日志输出级别不生效

log4j.properties中做配置: log4j.rootLogger=ERROR, Console (想改成error级别,否则输出信息太多;建议上报信息在debug下才输出)

但是在日志文件中还有info信息输出,这个能否确认下?

谢谢

执行报错

2016-12-19 15:31:34,134 [pool-1-thread-1] ERROR [com.stephan.tof.jmxmon.JMXMonitor] - error during JRMP connection establishment; nested exception is:
java.io.EOFException
java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
java.io.EOFException
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:304)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:341)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.checkSslConfig(ProxyClient.java:274)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.(ProxyClient.java:167)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.getProxyClient(ProxyClient.java:518)
at com.stephan.tof.jmxmon.JMXMonitor.runTask(JMXMonitor.java:58)
at com.stephan.tof.jmxmon.JMXMonitor.access$0(JMXMonitor.java:50)
at com.stephan.tof.jmxmon.JMXMonitor$1.run(JMXMonitor.java:41)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:267)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:246)
... 16 more

本地打包出现以下问题

mvn assembly:single

这个问题如何处理
[WARNING] Cannot include project artifact: com.stephan.tof:jmxmon:jar:0.0.2; it doesn't have an associated file or directory.

日志报错

2017-12-27 12:19:05,687 [main] INFO [com.stephan.tof.jmxmon.Config] - init config
2017-12-27 12:19:05,815 [main] INFO [com.stephan.tof.jmxmon.Config] - /home/wwh/open-falcon/wwh/jmxmon/jmxmon-v0.0.2/./jmxmon.jvm.context.json is not exist
2017-12-27 12:19:05,815 [main] INFO [com.stephan.tof.jmxmon.Config] - init config ok

显示:jmxmon.jvm.context.json is not exist

你好,我对源码打maven包报错,请帮忙看下

[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< com.stephan.tof:jmxmon >-----------------------
[INFO] Building jmxmon 0.0.2
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.490 s
[INFO] Finished at: 2019-08-07T19:47:22+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-install-plugin:2.4 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-install-plugin:jar:2.4: Could not transfer artifact org.apache.maven.plugins:maven-install-plugin:pom:2.4 from/to nexus-public (http://maven.aliyun.com/nexus/content/groups/public): Cannot access http://maven.aliyun.com/nexus/content/groups/public with type default using the available connector factories: BasicRepositoryConnectorFactory: Cannot access http://maven.aliyun.com/nexus/content/groups/public using the registered transporter factories: WagonTransporterFactory: java.util.NoSuchElementException
[ERROR] role: org.apache.maven.wagon.Wagon
[ERROR] roleHint: http
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

refused to host: localhost; nested exception is: java.net.ConnectException: 拒绝连接

conf.properties

the working dir

workDir=./

localhost jmx ports, split by comma

agent port url

jmx.ports=8099
agent.posturl=http://localhost:1988/v1/push

this program always print this logs,can not get data?

30 15:04:56,502 [pool-1-thread-1] ERROR [com.stephan.tof.jmxmon.JMXMonitor] - Connection refused to host: localhost; nested exception is:
java.net.ConnectException: 拒绝连接
java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: 拒绝连接
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.checkSslConfig(ProxyClient.java:274)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.(ProxyClient.java:167)
at com.stephan.tof.jmxmon.jmxutil.ProxyClient.getProxyClient(ProxyClient.java:518)
at com.stephan.tof.jmxmon.JMXMonitor.runTask(JMXMonitor.java:58)
at com.stephan.tof.jmxmon.JMXMonitor.access$0(JMXMonitor.java:50)
at com.stephan.tof.jmxmon.JMXMonitor$1.run(JMXMonitor.java:41)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.