citahub / cita-monitor Goto Github PK

designed at https://github.com/citahub/cita-monitor/blob/master/docs/alert_policies.md

 磁盘空间不足：剩余磁盘空间小于告警阀值

• this policy is active by default.
• alert when available space lower then 10%.
• 10% is a configuration.

each item of Metrics in a pie chart means one part of the whole pie.
so if here A means the total size of current node data used, while B means the disk size of current node, the percentages of them will be meaningless.

i think B should be

$NodeDisk-(Node_Get_DirInfo_TotalFileSize{job="$job",instance="$node:$port"} * 1024)

and the label of it should be something like unused disk.

Docker Container

• citamon_agent_process_exporter
• citamon_agent_rabbitmq_exporter
• citamon_agent_host_exporter
• citamon_server_grafana
• citamon_server_prometheus
• citamon_server_alertmanager

should refactor agent/docker-compose.yml to repect the design

designed at https://github.com/citahub/cita-monitor/blob/master/docs/alert_policies.md

 共识节点是否出块：共识节点是否有参与共识，超过告警设置时间则发送错误告警

• this policy is active by default.
• alert when there is an consensus node didn't vote more then 3 times.
• 3 is a configuration.

CPU use percent => CPU usage?

Set CITA chain's SysConfig.Operator to Chinese:

$ scm SysConfig setOperator --operator “中文”  --admin-priva xxxx

cita-monitor run panic with error:

UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xe6 in position 215: ordinal not in range(128)

See more details in talk in citahub:
https://talk.citahub.com/t/topic/1131/67

I noticed that the docker-compose uses the '/etc/localtime' and the '/etc/timezone', but this doesn't work in macos, in macos you can't use volumn on /etc directory, it will report an error. I solved this problem by modify the docker-compose and volumn an another localtime into docker.

User just need to install one service, it can have both CITA and agent installed

CITA bin was v0.20.2 and is updated with v0.24.1, but SoftVersion is still showing as v0.20.2

do this after https://github.com/citahub/cita-sdk-python officially released

#101

问题：使用du 命令获取目录容量会影响性能
解决：降低data dir size 检查的频度，改为1个小时检查获取一次。
做法：把 du 的结果导到一个文件，执行 du 前判断文件修改时间和当前时间对比是否达到采集区间，没达到阀值就cat 结果，这样来做采集频率控制

1. 开发环境的 python 版本应该要跟 agent 里的python image 版本一致，这样能及早发现问题
2. agent 里的python image，要尽量选择体积最小的版本，这样运行时不需要pull 很大的image

refs:

based on PR #110

1. run make generate-build-version to get build version
2. run make show-full-version to get full version number string
3. replace the full version number string in dashboard config json file when running docker build

user was considering CITA_CHAIN_ID id of the chain, but this var is expecting for the id of a node.

discussion detail: https://talk.citahub.com/t/topic/1131/75

When I use this monitor, in the "Node Info Dashboard"--> "quota info" district, I didn't see the difference between the a non-zero Quota Used and the zero quota-used. screenshot pasted here.

candidate list:

# ↪  docker-images-sort-by-size  | grep python | sort -k 2                                                                                              Sun Sep 29 09:46:01 CST 2019
# 923MB   python:3                                                          ad01f54ca3a9
# 143MB   python:3.7-slim                                                   338ae06dfca5
# 179MB   python:3.7-slim-buster                                            b5a7c089ece3
# 940MB   python:3.7-stretch                                                b9d77e48a75c
# 98.7MB  python:3.7-alpine                                                 39fb80313465
#    

Refs：

https://blog.realkinetic.com/building-minimal-docker-containers-for-python-applications-37d0272c52f3

https://pythonspeed.com/articles/base-image-python-docker-images/

可以封装一个 build 用的image ，里面装好开发工具的依赖库，这样开发环境也很简单了，也是用docker 来执行命令

1. create a docker image for developing, e.g.: 'citamon/devenv', use python as base image, install depenedencies used in make deps, should save Dockerfile to devtools/devenv/ dir
2. refactor make deps, make code-quality to run dev tools with docker, e.g. : docker exec `

The cita_exporter d image is building for each new host, and it use an out-of-date cita-cli.

https://github.com/citahub/cita-monitor/tree/develop/agent/cita_exporter

Is it better to use a pre-build cita_exporter image ?

这样可以通过一个节点，即可知道所有共识节点的出块情况：如果不均衡，表示有共识节点有不可用

参考 https://microscope.citahub.com/#/graphs 的 proposal distribution 图

currently:

CITA-Monitor agent:

• host_exporter：1920
• process_exporter：1921
• rabbitmq_exporter：1922
• cita_exporter：1923

expected:

CITA-Monitor agent

• agent_service：1920

agent_service can proxy all exporters