Giter Club home page Giter Club logo

jenkins-autoscaler's Introduction

Jenkins AutoScaler

build codecov Go Report Card GoDoc License Downloads

Jenkins Autoscaler is a tool that automatically adjusts the size of the Jenkins worker nodes in AWS or GCP when one of the following conditions is true:

  1. If the usage of Jenkins nodes is smaller then the specified threshold, it will scale down or if the usage is larger then a specified threshold, will scale up to the maximum of specified size limit.
  2. Running garbage collector to remove zombie nodes.
  3. When the queue is empty, it checks whether any node can be terminated by establishing whether the node is in idle state.
  4. One node is kept running all the time, apart during non working hours.
  5. Once a node is online, it's kept alive for at least 20 minutes.

AWS, Requires cloud permissions (IAM)

Jenkins Autoscaler utilizes Amazon EC2 Auto Scaling Groups to manage node groups.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeAutoScalingInstances",
        "autoscaling:DescribeScalingActivities",
        "autoscaling:SetDesiredCapacity",
        "autoscaling:TerminateInstanceInAutoScalingGroup",
        "ec2:DescribeInstances"
      ],
      "Resource": ["*"]
    }
  ]
}

GCP, Requires one of the following OAuth scopes

https://www.googleapis.com/auth/compute
https://www.googleapis.com/auth/cloud-platform

Compile and run

make install
jas --help # to see all options

example how to run in gce

jas  \
    --backend=gce \
    --gce-project=my-gce-project \
    --gce-region=us-east1 \
    --gce-instance-group-manager=my-jenkins-instance-group \
    --jenkins-url=https://ci.jenkins.com \
    --jenkins-user=jenkins-admin \
    --jenkins-token=jenkins6user6token6here

example how to run in aws

jas  \
    --backend=aws \
    --aws-region=us-east-1 \
    --aws-autoscaling-group-name=my-jenkins-asg \
    --jenkins-url=https://ci.jenkins.com \
    --jenkins-user=jenkins-admin \
    --jenkins-token=jenkins6user6token6here

example how to run with the config file

scaler:
  log_level: debug
  dry_run: true
  run_interval: 3s
  gc_run_interval: 3s
  scale_up_grace_period: 5m
  scale_down_grace_period: 10m
  scale_down_grace_period_during_working_hours: 1h
  controller_node_name: Built-In Node
  scale_up_threshold: 70
  scale_down_threshold: 30
  max_nodes: 10
  min_nodes_during_working_hours: 2
  jenkins_url: https://my.jenkins.co
  jenkins_user: ""
  jenkins_token: ""

backend:
  gce:
    project: gcp-project
    region: us-east1
    instance_group_manager: jenkins-nodes
  aws:
    autoscaling_group_name: jenkins-nodes
    region: us-east-1
# run with configuration file
jas --backend=gce -c ./configs/my-jenkins-autoscaler.yaml

Contribution

Feel free to open Pull-Request for small fixes and changes. For bigger changes and new backends please open an issue first to prevent double work and discuss relevant stuff.

License


This is free software under the terms of the MIT the license (check the LICENSE file included in this package).

jenkins-autoscaler's People

Contributors

shamil avatar shareed2k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

yasithab

jenkins-autoscaler's Issues

Add helm chart and repo

We should add a chart and helm repository for this project so it will be easier to deploy on Kubernetes. I'll start working on the chart, we can configure the helm repository using gh-pages once it's done

autoscaler scales to minimum when jenkins controller is unavailable

When jenkins controller is unavailable, autoscaler thinks that that there is 0 agents running and adjust the number of nodes to one.
The proper operation should be if jenkins controller is inaccessible, all scaling activities should be suspended until the it can contact the controller.

Here are the relevant logs:

time="2022-07-10T19:45:18+03:00" level=error msg="can't get current jenkins usage: Get \"http://jenkins-master.service.integration.consul:8080/computer/api/json?depth=1\": dial tcp: lookup jenkins-master.service.integration.consul on 10.11.240.10:53: no such host" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="current nodes usage is -9223372036854775808%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="number of nodes: 0" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=info msg="not a single node off work hours. will adjust to one" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="current nodes usage is 0%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="number of nodes: 1" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="current nodes usage is 0%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="number of nodes: 1" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="current nodes usage is 0%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="number of nodes: 1" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false

Add health and ready endpoints

We already have an http server for exposing metrics, we should also add /health and /ready endpoints to ease kubernetes deployments

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.