smartloli / efak Goto Github PK
View Code? Open in Web Editor NEWA easy and high-performance monitoring system, for comprehensive monitoring and management of kafka cluster.
Home Page: https://www.kafka-eagle.org/
License: Apache License 2.0
A easy and high-performance monitoring system, for comprehensive monitoring and management of kafka cluster.
Home Page: https://www.kafka-eagle.org/
License: Apache License 2.0
启动后,访问localhost:8048/ke 后,显示登录页面,请问下用户名和密码要在哪里配置么?本地mysql中也有新建ke库。下面是我的配置,还请帮忙看下,谢谢。
######################################
######################################
kafka.eagle.zk.cluster.alias=cluster1
cluster1.zk.list=localhost:2181
######################################
######################################
kafka.zk.limit.size=25
######################################
######################################
kafka.eagle.webui.port=8048
######################################
######################################
kafka.eagle.offset.storage=kafka
######################################
######################################
kafka.eagle.mail.enable=true
kafka.eagle.mail.sa=alert_sa
kafka.eagle.mail.username=[email protected]
kafka.eagle.mail.password=mqslimczkdqabbbg
kafka.eagle.mail.server.host=smtp.163.com
kafka.eagle.mail.server.port=25
######################################
######################################
kafka.eagle.topic.token=test
######################################
######################################
#kafka.eagle.sasl.enable=false
#kafka.eagle.sasl.protocol=SASL_PLAINTEXT
#kafka.eagle.sasl.mechanism=PLAIN
#kafka.eagle.sasl.client=kafka_client_jaas.conf
######################################
######################################
kafka.eagle.driver=com.mysql.jdbc.Driver
kafka.eagle.url=jdbc:mysql://localhost:3306/ke?useUnicode=true&characterEncoding=UTF8
kafka.eagle.username=root
kafka.eagle.password=root
看到使用quartz job去跑创建表的sql,但是报错了,是不是和这个有关呢?ke数据库是不是要初始化表等。
errorInfo:
cluster
,key
,value
,hour
,tm
) values (?,?,?,?,?) , (?,?,?,?,?) , (?,?,?,?,?) , (?,?,?,?,?) , (?,?,?,?,?); bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ke.ke_trend' doesn't exist
thx
when I run KE, I got following error in ./logs/error.log.
org.springframework.scheduling.quartz.JobMethodInvocationFailedException: Invocation of method 'mbeanQuartz' on target class [class org.smartloli.kafka.eagle.web.quartz.MBeanQuartz] failed; nested exception is java.lang.ArrayIndexOutOfBoundsExcept
what should I do?
Technically, can we support both at the same time. I mean if there is any conflict or issue when using both storages ?
SELECT "partition","offset","msg" FROM "test" WHERE "partition" IN (0) limit 10;
弹出来这个DataTables warning: table id=result_children0 - Cannot reinitialise DataTable. For more information about this error, please see http://datatables.net/tn/3
logs:Finished by [0.61s].
result:是空
只有partition0分区 logsize700 offset700 lag0
生产速度、消费速度总显示为0,感觉是bug,而且要不了多长时间,会挂掉
error details as following :
2017-11-24 18:35:00 ERROR [Mx4jServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#1_Worker-10] - JMX service url[localhost:-1] create has error,msg is Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: 拒绝连接 (Connection refused)]
2017-11-24 18:35:00 ERROR [Mx4jServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#1_Worker-10] - JMX service url[localhost:-1] create has error,msg is Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: 拒绝连接 (Connection refused)]
2017-11-24 18:35:00 ERROR [Mx4jServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#1_Worker-10] - JMX service url[localhost:-1] create has error,msg is Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: 拒绝连接 (Connection refused)]
2017-11-24 18:35:00 ERROR [Mx4jServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#1_Worker-10] - JMX service url[localhost:-1] create has error,msg is Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: 拒绝连接 (Connection refused)]
2017-11-24 18:35:00 ERROR [Mx4jServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#1_Worker-10] - JMX service url[localhost:-1] create has error,msg is Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: 拒绝连接 (Connection refused)]
使用sparkstreaming direct api消费kafka,获取不到groupidea,zookeeper可以查看到offset,请问是什么问题呢?谢谢!
kafka版本 0.10.x
kafka-eagle版本:1.1.6
目前能看到Consumers,但是进入Consumers Offsets details页面一直显示Processing...
后台日志如下:
[2017-07-27 14:16:13] VerifiableProperties.kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-leader-finder-thread - INFO - Verifying properties
[2017-07-27 14:16:13] VerifiableProperties.kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-leader-finder-thread - INFO - Property client.id is overridden to kafka.eagle.system.group
[2017-07-27 14:16:13] VerifiableProperties.kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-leader-finder-thread - INFO - Property metadata.broker.list is overridden to kafkanode0:9092,kafkanode1:9092,kafkanode2:9092
[2017-07-27 14:16:13] VerifiableProperties.kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-leader-finder-thread - INFO - Property request.timeout.ms is overridden to 30000
[2017-07-27 14:16:13] ClientUtils$.kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-leader-finder-thread - INFO - Fetching metadata from broker BrokerEndPoint(1,kafkanode1,9092) with correlation id 153 for 1 topic(s) Set(__consumer_offsets)
[2017-07-27 14:16:13] SyncProducer.kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-leader-finder-thread - INFO - Connected to kafkanode1:9092 for producing
[2017-07-27 14:16:13] SyncProducer.kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-leader-finder-thread - INFO - Disconnecting from kafkanode1:9092
[2017-07-27 14:16:13] SimpleConsumer.ConsumerFetcherThread-kafka.eagle.system.group_iZ25x3jocmtZ-1501133782430-71836807-0-1 - INFO - Reconnect due to error:
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)
at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:85)
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:83)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:132)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:131)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:130)
at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:111)
at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:31)
at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
while running the web project in idea, visiting localhost:8080,I got an error shows too many redirects.
Broswer had been redirected to "localhost:8080/ke/account/signin?/ke/accounts/signin" , which supposed to be "localhost:port/ke/account/singin?/ke",I guess , cause that when I package and deploy it in tomcat server , I made it to visit the web UI, and it shows like that.
[2017-10-19 11:40:00] MBeanQuartz.org.springframework.scheduling.quartz.SchedulerFactoryBean#1_Worker-1 - ERROR - Collector mbean data has error,msg is
cluster
,key
,value
,hour
,tm
) values; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 2
~
I have tried kafka-consumer-groups.sh and see that it doesn't support non java based clients (producer, consumer). Does the project support clients written in go?
Versions
Kafka Version: 0.10.2.1
Go Version: 1.8
是否支持kafka1.0版本
When will this framework work under Windows?thanks
i have created a topic named "test",when i want to remove it ,web prompt input Admin token, i get an Internal server error ~~ after i put my token,can you help me?
你好,我跟着文档把环境搭建起来之后,发现Consumers这个tab的内容展示不出来,查看日志报错如下:
2017-07-08 15:04:07 ERROR [KafkaServiceImpl.http-nio-8048-exec-19] - Get kafka consumer has error,msg is Request METADATA failed on brokers List(ip-172-31-4-53:9092 (id: -3 rack: null), ip-172-31-4-54:9092 (id: -2 rack: null), ip-172-31-4-55:9092 (id: -1 rack: null))
HTTP Status 500 - Servlet.init() for servlet SpringMvc threw exception
type Exception report
message Servlet.init() for servlet SpringMvc threw exception
description The server encountered an internal error that prevented it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet.init() for servlet SpringMvc threw exception
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
root cause
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mappingJacksonHttpMessageConverter' defined in class path resource [spring-mvc.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter]: Constructor threw exception; nested exception is java.lang.NoClassDefFoundError: Could not initialize class com.fasterxml.jackson.databind.SerializationConfig
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1101)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:663)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:629)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:677)
org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:548)
org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:489)
org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
javax.servlet.GenericServlet.init(GenericServlet.java:158)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
root cause
org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter]: Constructor threw exception; nested exception is java.lang.NoClassDefFoundError: Could not initialize class com.fasterxml.jackson.databind.SerializationConfig
org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:163)
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:89)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1094)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:663)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:629)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:677)
org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:548)
org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:489)
org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
javax.servlet.GenericServlet.init(GenericServlet.java:158)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
root cause
java.lang.NoClassDefFoundError: Could not initialize class com.fasterxml.jackson.databind.SerializationConfig
com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:558)
com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:474)
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.build(Jackson2ObjectMapperBuilder.java:521)
org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.(MappingJackson2HttpMessageConverter.java:56)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147)
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:89)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1094)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:663)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:629)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:677)
org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:548)
org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:489)
org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
javax.servlet.GenericServlet.init(GenericServlet.java:158)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
Constructor threw exception; nested exception is java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
线上运行的kafka集群是0.10.2。打开eagle后,后台没有显示任何信息,使用的eagle版本为最新的 1.1.2 错误是
java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:81)
at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:129)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:120)
at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:99)
at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:83)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:132)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132)
at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:131)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131)
at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:130)
at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:111)
at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:31)
at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
感觉应该是API的不兼容,还没兼容0.10.2版本么? 从readme文档看,0.10.x是在未来的计划中。
org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'expire_timestamp': java.nio.BufferUnderflowException
at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:73)
at kafka.coordinator.GroupMetadataManager$.readOffsetMessageValue(GroupMetadataManager.scala:972)
at kafka.coordinator.GroupMetadataManager.readOffsetMessageValue(GroupMetadataManager.scala)
at com.wfl.kafka.monitor.KafkaOffsetGetter.startSaslOffsetListener(KafkaOffsetGetter.java:57)
at com.wfl.kafka.monitor.KafkaOffsetGetter.run(KafkaOffsetGetter.java:117)
what should i do
Hi, I've been struggling with MySql related errors.
I couldn't find any information in the readme what version is required, and that it's really required.
Once I've installed it, it worked fine.
Although there are few assumptions in the schema.sh file.
Unknown character set: 'utf8mb4'
, as I had an older version installed on the server登录页面输入用户密码后,登录不了,界面一直都是空白,使用1.1.6,kafka 0.8,请问有人可以帮忙解决一下吗?谢谢
Hi
I can not show consumer offset on my cluster 2 and cluster 3 and I got error as below:
ERROR [OffsetsQuartz.org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-3] - Quartz statistics offset has error,msg is null.
is this as expect?
but i can get correct consumer offset on cluster 1 if I changed cluster from cluster 2 to cluster 1.
my system-config.properties as below:
######################################
######################################
kafka.eagle.zk.cluster.alias=cluster1,cluster2,cluster3
cluster1.zk.list=192.168.137.182:2181,192.168.137.182:2182,192.168.137.182:2183
cluster2.zk.list=192.168.137.183:2181,192.168.137.183:2182,192.168.137.183:2183
cluster3.zk.list=192.168.137.184:2181,192.168.137.184:2182,192.168.137.184:2183
######################################
######################################
kafka.zk.limit.size=25
######################################
######################################
kafka.eagle.webui.port=8999
######################################
######################################
kafka.eagle.offset.storage=kafka
by the way , I can get correct consumer offset on each cluster via kafka-manager.
thanks
update:
it seems only response first cluster alias.
新版本中权限控制需要通过数据库,但是在不同的系统中,一般使用SSO进行统一登陆,新版这样的设计是否显得多余?
HTTP Status 500 - Servlet.init() for servlet SpringMvc threw exception
type Exception report
message Servlet.init() for servlet SpringMvc threw exception
description The server encountered an internal error that prevented it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet.init() for servlet SpringMvc threw exception
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
root cause
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mappingJacksonHttpMessageConverter' defined in class path resource [spring-mvc.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1101)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:663)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:629)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:677)
org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:548)
org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:489)
org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
javax.servlet.GenericServlet.init(GenericServlet.java:158)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
root cause
org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:163)
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:89)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1094)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:663)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:629)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:677)
org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:548)
org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:489)
org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
javax.servlet.GenericServlet.init(GenericServlet.java:158)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
root cause
java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:564)
com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:474)
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.build(Jackson2ObjectMapperBuilder.java:521)
org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.(MappingJackson2HttpMessageConverter.java:56)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:422)
org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147)
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:89)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1094)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:663)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:629)
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:677)
org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:548)
org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:489)
org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
javax.servlet.GenericServlet.init(GenericServlet.java:158)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425)
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
note The full stack trace of the root cause is available in the Apache Tomcat/8.5.4 logs.
Apache Tomcat/8.5.4
Kafka must turn on sasal?
vi kafka-server-start.sh
...
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:PermSize=128m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70"
export JMX_PORT="9999"
#export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi
but i doesn't find kafka-server-start.sh file
When I setup SASL for Kafka, and connect it with kafka-eagle, the consumers section show nothing.
i do this : https://ke.smartloli.org/2.Install/2.Installing.html
and i start kafka-eagle,
it's run in Red hat 6 virtual machine in virtualBox
java is ok.
and i can access ip:port/ke, enter login page. but when i input "admin" and "123456", /signin/action not lead me to dashboard.
i check the conf and find "mysql", and it's have eagle.user, eagle.password, is it my mysql account?
when i call stats, i see my ip in "ESTABLISHED/TIME_OUT Status Block.
Consumers 页面中没有消费者信息,zk /consumers路径下是有消费者的。
另外建议Mock message中增加上传文件功能,文件按行导入到指定主题,该功能比较实用。
java.lang.NullPointerException at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89) at org.smartloli.kafka.eagle.web.controller.OffsetController.offsetGraphAjax(OffsetController.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778) at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:108) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:522) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1110) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:785) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1425) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) 2017-05-18 17:25:00 ERROR [KafkaServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1] - Get kafka consumer metadata has error, msg is Request GROUP_COORDINATOR failed on brokers List(172.31.0.41:9092 (id: -2 rack: null), 172.31.0.42:9092 (id: -3 rack: null), 172.31.0.40:9092 (id: -1 rack: null)) 2017-05-18 17:30:00 ERROR [KafkaServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-2] - Get kafka consumer metadata has error, msg is Request GROUP_COORDINATOR failed on brokers List(172.31.0.42:9092 (id: -3 rack: null), 172.31.0.41:9092 (id: -2 rack: null), 172.31.0.40:9092 (id: -1 rack: null))
hello,when using v1.1.9 happened the following problems:
My kafka version is 0.10.1 (HDP-2.6.1.0)
My config:
`kafka.eagle.sasl.enable=true
kafka.eagle.sasl.protocol=SASL_PLAINTEXT
kafka.eagle.sasl.mechanism=PLAIN
kafka.eagle.sasl.client=/opt/kafka/kafka-eagle-web-1.1.9/conf/kafka_client_jaas.conf`
`[root@hd177 kafka-eagle-web-1.1.9]# cat conf/kafka_client_jaas.conf
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useTicketCache=true
principal="kafka/hd177.bd@BD"
useKeyTab=true
serviceName="kafka"
keyTab="/etc/security/keytabs/kafka.service.keytab"
client=true;
};`
following issues:
[2017-11-02 16:30:00] SaslChannelBuilder.org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-5 - INFO - Failed to create channel due to org.apache.kafka.common.KafkaException: Failed to configure SaslClientAuthenticator at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:123) at org.apache.kafka.common.network.SaslChannelBuilder.buildChannel(SaslChannelBuilder.java:110) at org.apache.kafka.common.network.Selector.connect(Selector.java:188) at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:629) at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:186) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.trySend(ConsumerNetworkClient.java:397) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:217) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:172) at kafka.admin.AdminClient.kafka$admin$AdminClient$$send(AdminClient.scala:49) at kafka.admin.AdminClient$$anonfun$sendAnyNode$1.apply(AdminClient.scala:60) at kafka.admin.AdminClient$$anonfun$sendAnyNode$1.apply(AdminClient.scala:58) at scala.collection.immutable.List.foreach(List.scala:381) at kafka.admin.AdminClient.sendAnyNode(AdminClient.scala:58) at kafka.admin.AdminClient.findAllBrokers(AdminClient.scala:90) at kafka.admin.AdminClient.listAllGroups(AdminClient.scala:98) at kafka.admin.AdminClient.listAllConsumerGroups(AdminClient.scala:112) at org.smartloli.kafka.eagle.core.factory.KafkaServiceImpl.getKafkaConsumer(KafkaServiceImpl.java:785) at org.smartloli.kafka.eagle.web.quartz.OffsetsQuartz.execute(OffsetsQuartz.java:160) at org.smartloli.kafka.eagle.web.quartz.OffsetsQuartz.jobQuartz(OffsetsQuartz.java:146) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:269) at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:257) at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:75) at org.quartz.core.JobRunShell.run(JobRunShell.java:202) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) Caused by: org.apache.kafka.common.KafkaException: Failed to create SaslClient with mechanism PLAIN at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:138) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.configure(SaslClientAuthenticator.java:121) ... 27 more Caused by: javax.security.sasl.SaslException: Cannot get userid/password [Caused by javax.security.auth.callback.UnsupportedCallbackException: Could not login: the client is being asked for a password, but the Kafka client code doe s not currently support obtaining a password from the user.] at com.sun.security.sasl.ClientFactoryImpl.getUserInfo(ClientFactoryImpl.java:157) at com.sun.security.sasl.ClientFactoryImpl.createSaslClient(ClientFactoryImpl.java:94) at javax.security.sasl.Sasl.createSaslClient(Sasl.java:384) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:134) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator$1.run(SaslClientAuthenticator.java:129) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslClient(SaslClientAuthenticator.java:129) ... 28 more Caused by: javax.security.auth.callback.UnsupportedCallbackException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. at org.apache.kafka.common.security.authenticator.SaslClientCallbackHandler.handle(SaslClientCallbackHandler.java:73) at com.sun.security.sasl.ClientFactoryImpl.getUserInfo(ClientFactoryImpl.java:136) ... 35 more
Reconnect due to error: java.net.SocketTimeoutException at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:211) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103) at java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385) at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:81) at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:129) at kafka.network.BlockingChannel.receive(BlockingChannel.scala:120) at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:86) at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:83) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:132) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:132) at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:131) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131) at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:131) at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:130) at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:111) at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:31) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
win10 + kafka eagle 1.2.0,consumer 、zkcli、topic list都可以正常,但是后台报错如上
[2017-09-25 16:50:01] Mx4jServiceImpl.org.springframework.scheduling.quartz.SchedulerFactoryBean#1_Worker-4 - ERROR - JMX service url[x.x.x.x:19090] create has error,msg is Authentication failed! Credentials required
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'consumersController': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.smartloli.kafka.eagle.web.service.ConsumerService org.smartloli.kafka.eagle.web.controller.ConsumersController.consumerService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'consumerServiceImpl' defined in file [/home/haha/下载/chronic-disease/kafka-eagle-web/target/kafka-eagle-web-1.1.9/WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/ConsumerServiceImpl.class]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.smartloli.kafka.eagle.web.service.impl.ConsumerServiceImpl]: Constructor threw exception; nested exception is java.lang.ExceptionInInitializerError
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1210)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:757)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:403)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:106)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.apache.catalina.startup.HostConfig.manageApp(HostConfig.java:1739)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:300)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at org.apache.catalina.mbeans.MBeanFactory.createStandardContext(MBeanFactory.java:482)
at org.apache.catalina.mbeans.MBeanFactory.createStandardContext(MBeanFactory.java:431)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:300)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.smartloli.kafka.eagle.web.service.ConsumerService org.smartloli.kafka.eagle.web.controller.ConsumersController.consumerService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'consumerServiceImpl' defined in file [/home/haha/下载/chronic-disease/kafka-eagle-web/target/kafka-eagle-web-1.1.9/WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/ConsumerServiceImpl.class]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.smartloli.kafka.eagle.web.service.impl.ConsumerServiceImpl]: Constructor threw exception; nested exception is java.lang.ExceptionInInitializerError
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:561)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:331)
... 58 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'consumerServiceImpl' defined in file [/home/haha/下载/chronic-disease/kafka-eagle-web/target/kafka-eagle-web-1.1.9/WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/ConsumerServiceImpl.class]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.smartloli.kafka.eagle.web.service.impl.ConsumerServiceImpl]: Constructor threw exception; nested exception is java.lang.ExceptionInInitializerError
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1101)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1120)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1044)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:942)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:533)
... 60 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.smartloli.kafka.eagle.web.service.impl.ConsumerServiceImpl]: Constructor threw exception; nested exception is java.lang.ExceptionInInitializerError
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:163)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:89)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1094)
... 71 more
Caused by: java.lang.ExceptionInInitializerError
at org.smartloli.kafka.eagle.core.factory.KafkaServiceImpl.(KafkaServiceImpl.java:109)
at org.smartloli.kafka.eagle.core.factory.KafkaFactory.create(KafkaFactory.java:33)
at org.smartloli.kafka.eagle.web.service.impl.ConsumerServiceImpl.(ConsumerServiceImpl.java:55)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147)
... 73 more
Caused by: java.lang.NullPointerException
at org.smartloli.kafka.eagle.common.util.ZKPoolUtils.(ZKPoolUtils.java:55)
... 81 more
能不能写个具体的怎么部署 需要改哪些配置呢
After using startOffsetSaslListener to listen offset thread method with sasl,the old consumer offset is wrong。For example,group ”test“ logsize is 40000 and actually offset is also 40000,we can see it before。but after using startOffsetSaslListener,the offset we can see becomes 20000。And a new group don’t hava the same problem。Why it happen?How can i make the result we can see become true?
As tittle, I want send alert mail from our mail relay server instead of private email, is there any solution?
按照文档安装成功,也启动起来了。但是在dashboard,无法正确显示broker数量,topic数量。查看后台发现error,表现为Rpc client getConsumer has error.
stack 如下
Rpc Client getConsumer has error,msg is null
java.lang.NullPointerException
at org.smartloli.kafka.eagle.service.impl.DashboardServiceImpl.getKafkaConsumerNumbers(DashboardServiceImpl.java:78)
at org.smartloli.kafka.eagle.service.impl.DashboardServiceImpl.panel(DashboardServiceImpl.java:123)
at org.smartloli.kafka.eagle.service.impl.DashboardServiceImpl.getDashboard(DashboardServiceImpl.java:68)
at org.smartloli.kafka.eagle.controller.DashboardController.dashboardAjax(DashboardController.java:77)
at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
I'm seeing that in logs that metrics are not available:
[2017-08-15 12:39:47] Mx4jServiceImpl.http-nio-8080-exec-25 - ERROR - JMX service url[kafka-host-1:9990] create has error,msg is Connection refused to host: 127.0.0.1; nested exception is:
java.net.ConnectException: Connection refused (Connection refused)
I can telnet to the jmx port: telnet kafka-host-1 9990
I'm using kafka 0.10.2.0
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.