Comments (3)
监控指标:
1.执行中失败的任务
2.执行中的任务
3.schedule轮训次数较多的数据
from bk-sops.
执行失败任务sql
EXPLAIN SELECT
tt.id,cp.name, pp.name
FROM
`taskflow3_taskflowinstance` AS tt,
`core_project` AS cp,
`pipeline_pipelineinstance` AS pp,
`eri_state` AS es
WHERE
pp.instance_id = es.root_id
AND tt.pipeline_instance_id=pp.id
AND tt.project_id=cp.id
AND pp.is_deleted = 0
AND pp.is_expired = 0
AND pp.is_finished = 0
AND pp.is_revoked = 0
AND pp.is_started = 1
AND es.NAME = "FAILED"
ORDER BY
pp.id DESC
执行失败任务ORM
sql = """
EXPLAIN SELECT
tt.id,cp.name, pp.name
FROM
`taskflow3_taskflowinstance` AS tt,
`core_project` AS cp,
`pipeline_pipelineinstance` AS pp,
`eri_state` AS es
WHERE
pp.instance_id = es.root_id
AND tt.pipeline_instance_id=pp.id
AND tt.project_id=cp.id
AND pp.is_deleted = 0
AND pp.is_expired = 0
AND pp.is_finished = 0
AND pp.is_revoked = 0
AND pp.is_started = 1
AND es.NAME = "FAILED"
ORDER BY
pp.id DESC
"""
with connection.cursor() as cursor:
cursor.execute(sql)
results = cursor.fetchall()
print(results)
执行中任务sql
explain SELECT
es.root_id,
COUNT( es.root_id ) AS total_count,
SUM( CASE WHEN es.NAME = 'Failed' THEN 1 ELSE 0 END ) AS failed_count,
SUM( CASE WHEN es.NAME = 'FINISHED' THEN 1 ELSE 0 END ) AS finished_count
FROM
pipeline_pipelineinstance AS pp
INNER JOIN eri_state AS es ON pp.instance_id = es.root_id
WHERE
pp.is_started = 1
AND pp.is_finished = 0
AND pp.is_revoked = 0
AND pp.is_expired = 0
AND is_deleted = 0
GROUP BY
es.root_id
HAVING
failed_count = 0
AND finished_count != total_count
ORDER BY
pp.id
limit 1000
执行中任务ORM
sql = """
SELECT
es.root_id,
COUNT( es.root_id ) AS total_count,
SUM( CASE WHEN es.NAME = 'Failed' THEN 1 ELSE 0 END ) AS failed_count,
SUM( CASE WHEN es.NAME = 'FINISHED' THEN 1 ELSE 0 END ) AS finished_count
FROM
pipeline_pipelineinstance AS pp
INNER JOIN eri_state AS es ON pp.instance_id = es.root_id
WHERE
pp.is_started = 1
AND pp.is_finished = 0
AND pp.is_revoked = 0
AND pp.is_expired = 0
AND is_deleted = 0
GROUP BY
es.root_id
HAVING
failed_count = 0
AND finished_count != total_count
ORDER BY
pp.id
limit 1000
"""
with connection.cursor() as cursor:
cursor.execute(sql)
results = cursor.fetchall()
print(results)
schedule轮训次数
EXPLAIN SELECT
pp.creator,
pp.id,
esc.schedule_times
FROM
eri_schedule AS esc,
eri_state AS es,
pipeline_pipelineinstance AS pp
WHERE
esc.node_id = es.node_id
AND es.root_id = pp.instance_id
AND esc.scheduling = 0
ORDER BY
esc.schedule_times DESC
LIMIT 1000;
sql = """
EXPLAIN SELECT
pp.creator,
pp.id,
esc.schedule_times
FROM
eri_schedule AS esc,
eri_state AS es,
pipeline_pipelineinstance AS pp
WHERE
esc.node_id = es.node_id
AND es.root_id = pp.instance_id
AND esc.scheduling = 0
ORDER BY
esc.schedule_times DESC
LIMIT 1000;
"""
with connection.cursor() as cursor:
cursor.execute(sql)
results = cursor.fetchall()
print(results)
from bk-sops.
补充:
- sql 性能优化:可以考虑拆分查询,减少联表
- mq 数据获取:通过 client 获取 mq 数据,并提供查询接口
- 聚合数据接口:某段时间(1天/3天)正在执行/失败任务数聚合
from bk-sops.
Related Issues (20)
- 远程插件可以选择,但是执行时报错pipeline.exceptions.ComponentNotExistException
- sync apigw 只发布对应环境的网关接口
- 快速分发文件插件支持文件源类型 HOT 1
- 涉及文本下拉框变量的模板(父流程引用子流程文本值下拉框变量),计划任务、api 任务创建失败
- 标准运维公共流程导入导入YAML中“覆盖已有流程”中的搜索框前端接口调用有误
- 远程插件包源管理编辑报错
- 任务列表响应速度优化
- 【我的动态】列表默认仅展示 1个月数据
- migrate_schedule 周期任务优化
- 支持环境统一 footer 配置读取 & 渲染
- 文本值下拉框text 顺序问题
- 接入蓝鲸审计中心
- 第三方插件支持上传文件的同时传递参数
- 子流程批量更新问题
- 点击审计中心后会使得业务选择出现已归档业务
- 流程编辑的全局变量中,IP选择器在点击使用IP搜索框后导致选择组件无法正常使用
- retry_v2变量免渲染问题处理
- 流程导入,选择覆盖已有的,获取流程限制15条
- 流程列表支持过滤和展示执行代理人
- 内置插件nodeman支持根据给定IP自动填写推荐的安装通道 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bk-sops.