Giter Club home page Giter Club logo

Comments (3)

lTimej avatar lTimej commented on August 19, 2024
监控指标:
    1.执行中失败的任务
    2.执行中的任务
    3.schedule轮训次数较多的数据

from bk-sops.

lTimej avatar lTimej commented on August 19, 2024

执行失败任务sql

EXPLAIN SELECT
	tt.id,cp.name, pp.name
FROM
	`taskflow3_taskflowinstance` AS tt,
	`core_project` AS cp,
	`pipeline_pipelineinstance` AS pp,
	`eri_state` AS es
WHERE
		pp.instance_id = es.root_id 
		AND tt.pipeline_instance_id=pp.id
		AND tt.project_id=cp.id
		AND pp.is_deleted = 0 
		AND pp.is_expired = 0 
		AND pp.is_finished = 0 
		AND pp.is_revoked = 0 
		AND pp.is_started = 1 
		AND es.NAME = "FAILED" 
ORDER BY
pp.id DESC

执行失败任务ORM

        sql = """
            EXPLAIN SELECT
	tt.id,cp.name, pp.name
FROM
	`taskflow3_taskflowinstance` AS tt,
	`core_project` AS cp,
	`pipeline_pipelineinstance` AS pp,
	`eri_state` AS es
WHERE
		pp.instance_id = es.root_id 
		AND tt.pipeline_instance_id=pp.id
		AND tt.project_id=cp.id
		AND pp.is_deleted = 0 
		AND pp.is_expired = 0 
		AND pp.is_finished = 0 
		AND pp.is_revoked = 0 
		AND pp.is_started = 1 
		AND es.NAME = "FAILED" 
ORDER BY
pp.id DESC
        """
        with connection.cursor() as cursor:
            cursor.execute(sql)
            results = cursor.fetchall()
            print(results)

执行中任务sql

explain SELECT
	es.root_id,
	COUNT( es.root_id ) AS total_count,
	SUM( CASE WHEN es.NAME = 'Failed' THEN 1 ELSE 0 END ) AS failed_count,
	SUM( CASE WHEN es.NAME = 'FINISHED' THEN 1 ELSE 0 END ) AS finished_count 
FROM
	pipeline_pipelineinstance AS pp
	INNER JOIN eri_state AS es ON pp.instance_id = es.root_id 
WHERE
	pp.is_started = 1 
	AND pp.is_finished = 0 
	AND pp.is_revoked = 0 
	AND pp.is_expired = 0 
	AND is_deleted = 0 
GROUP BY
	es.root_id
HAVING
	failed_count = 0 
	AND finished_count != total_count 
ORDER BY
	pp.id
limit 1000

执行中任务ORM

sql = """
            SELECT
                es.root_id,
                COUNT( es.root_id ) AS total_count,
                SUM( CASE WHEN es.NAME = 'Failed' THEN 1 ELSE 0 END ) AS failed_count,
                SUM( CASE WHEN es.NAME = 'FINISHED' THEN 1 ELSE 0 END ) AS finished_count 
            FROM
                pipeline_pipelineinstance AS pp
                INNER JOIN eri_state AS es ON pp.instance_id = es.root_id 
            WHERE
                pp.is_started = 1 
                AND pp.is_finished = 0 
                AND pp.is_revoked = 0 
                AND pp.is_expired = 0 
                AND is_deleted = 0 
            GROUP BY
                es.root_id
            HAVING
                failed_count = 0 
                AND finished_count != total_count 
            ORDER BY
                pp.id
            limit 1000
        """
        with connection.cursor() as cursor:
            cursor.execute(sql)
            results = cursor.fetchall()
            print(results)

schedule轮训次数

EXPLAIN SELECT
                pp.creator,
                pp.id,
                esc.schedule_times 
            FROM
                eri_schedule AS esc,
                eri_state AS es,
                pipeline_pipelineinstance AS pp 
            WHERE
                esc.node_id = es.node_id 
                AND es.root_id = pp.instance_id 
                AND esc.scheduling = 0 
            ORDER BY
                esc.schedule_times DESC
            LIMIT 1000;
sql = """
            EXPLAIN SELECT
                pp.creator,
                pp.id,
                esc.schedule_times 
            FROM
                eri_schedule AS esc,
                eri_state AS es,
                pipeline_pipelineinstance AS pp 
            WHERE
                esc.node_id = es.node_id 
                AND es.root_id = pp.instance_id 
                AND esc.scheduling = 0 
            ORDER BY
                esc.schedule_times DESC
            LIMIT 1000;
        """
        with connection.cursor() as cursor:
            cursor.execute(sql)
            results = cursor.fetchall()
            print(results)

from bk-sops.

normal-wls avatar normal-wls commented on August 19, 2024

补充:

  1. sql 性能优化:可以考虑拆分查询,减少联表
  2. mq 数据获取:通过 client 获取 mq 数据,并提供查询接口
  3. 聚合数据接口:某段时间(1天/3天)正在执行/失败任务数聚合

from bk-sops.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.