Light

uswitch / sqs-autoscaler-controller Goto Github PK

View Code? Open in Web Editor NEW

30.0 17.0 15.0 27.1 MB

Kubernetes controller for scaling Deployments based on AWS SQS queue length

License: Other

Go 100.00%

sqs-autoscaler-controller's Introduction

SQS Autoscaler Controller

This provides a controller to autoscale Deployments according to the queue size of an SQS queue.

Creating

apiVersion: aws.uswitch.com/v1
kind: SqsAutoScaler
metadata:
  name: testing-scaler
  namespace: myns
spec:
  deployment: testapp
  maxPods: 20
  minPods: 1
  queue: https://sqs.eu-west-1.amazonaws.com/1234567890/some-queue
  scaleDown:
    amount: 2
    threshold: 10
  scaleUp:
    amount: 5
    threshold: 100

$ kubectl apply -f scaler.yaml
$ kubectl get sqsautoscalers

sqs-autoscaler-controller's People

Contributors

Stargazers

Watchers

Forkers

gertjan-carbon decoursin made2591 bugsbio practo tabossert sureshamk malikghanim acumenix akshay201 razorpay ankitjain28may intellihr openmed

sqs-autoscaler-controller's Issues

RBAC example

I am running k8s 1.8.1 and I had to change somethings including RBAC to get it working. Here is my example with RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: kube-system
  name: sqs-autoscaler-controller
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  namespace: kube-system
  name: sqs-autoscaler-controller-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: sqs-autoscaler-controller-role
subjects:
- kind: ServiceAccount
  name: sqs-autoscaler-controller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  namespace: kube-system
  name: sqs-autoscaler-controller-role
rules:
- apiGroups:
  - "apiextensions.k8s.io"
  resources:
  - customresourcedefinitions
  verbs:
  - create
- apiGroups:
  - "aws.uswitch.com"
  resources:
  - sqsautoscalers
  verbs:
  - list
  - get
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - get
  - create
  - update
- apiGroups:
  - "apps"
  resources:
  - deployments
  verbs:
  - get
  - update
---
kind: Deployment
apiVersion: apps/v1beta1
metadata:
  name: sqs-autoscaler-controller
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: sqs-autoscaler-controller
      annotations:
        iam.amazonaws.com/role: sqs_read_only
    spec:
      serviceAccountName: sqs-autoscaler-controller
      volumes:
        - name: ssl-certs
          hostPath:
            path: "/etc/ssl/certs/ca-certificates.crt"
      containers:
      - image: 12345.dkr.ecr.us-west-2.amazonaws.com/sqs-autoscaler-controller:1.0
        imagePullPolicy: Always
        env:
        - name: AWS_REGION
          value: us-west-2
        - name: NODE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        args:
          - --statsd=$(NODE_IP):30565
        name: sqs-autoscaler-controller
        volumeMounts:
          - name: ssl-certs
            mountPath: /etc/ssl/certs/ca-certificates.crt
            readOnly: true

Another issue was with using $(NODE_NAME):30565, I was getting DNS lookup failures which was really weird. Moved to an IP.

Next I was having issues with the Root Cert, so I changed some paths and also changed the Dockerfile to:

FROM alpine:3.6

RUN  apk add --no-cache --update ca-certificates

ADD bin/sqs-autoscaler-controller /sqs-autoscaler-controller

ENTRYPOINT ["/sqs-autoscaler-controller"]

Output log data as JSON

To integrate better with ELK

Publish to Quay.io

Rather than publishing to our own private registry it'd be nice to have images built and pushed to Quay like kiam, klint etc.

Maintenance Windows

The CRM team often have to scale some Deployments to 0 due to Responsys maintenance windows.

Rather than trying to hit the API when its unavailable it'd be nice to automatically scale to 0.

Makefile?

Is there a Makefile available for this project? I would love to try it but can’t get it to compile. Also the image url is down.

Thanks

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.