Giter Club home page Giter Club logo

stagetime-creator's Introduction

stuttgart-things/stageTime-creator

dynamic rendering and creation of k8s manifests/resources polled from redis streams/json

DEV-TASKS

task --list: Available tasks for this project:
* build:               Build the app
* build-image:         Build image
* git-push:            Commit & push the module
* lint:                Lint code
* package:             Update Chart.yaml and package archive
* push:                Push to registry
* release:             Build amd release to github w/ goreleaser
* run:                 Run app
* run-container:       Run container
* run-test:            Run test-producer
* tag:                 Commit, push & tag the module
* test:                Test code

HELMFILE-DEPLOYMENTS

SET VAULT CONNECTION
export VAULT_ADDR=https://${VAULT_FQDN}}
export VAULT_NAMESPACE=root

# APPROLE AUTH
export VAULT_AUTH_METHOD=approle
export VAULT_ROLE_ID=${VAULT_ROLE_ID}
export VAULT_SECRET_ID=${VAULT_SECRET_ID}

# TOKEN AUTH
export VAULT_AUTH_METHOD=token #default
export VAULT_TOKEN=${VAULT_TOKEN}
RENDER/APPLY
helmfile template --environment labul-pve-dev
helmfile sync --environment labul-pve-dev

DEPLOY DEV CODE TO CLUSTER

DEPLOYMENT
helm pull oci://eu.gcr.io/stuttgart-things/stagetime-creator --version v0.1.44
cat <<EOF > stageTime-creator.yaml
---
secrets:
  redis-connection:
    name: redis-connection
    labels:
      app: stagetime-server
    dataType: stringData
    secretKVs:
      REDIS_SERVER: redis-stack-deployment-headless.redis-stack.svc.cluster.local
      REDIS_PORT: 6379
      REDIS_PASSWORD: <PASSWORD>
EOF
helm upgrade --install stagetime-creator oci://eu.gcr.io/stuttgart-things/stagetime-creator --version v0.1.44 --values stageTime-creator.yaml -n stagetime-creator --create-namespace
CHECK REDIS DATA w/ CLI
# INSTALL REDIS-CLI
sudo apt-get update
sudo apt-get install redis
# SHELL #1
kubectl -n stagetime port-forward redis-stack-node-0 28015:6379 -n stagetime-redis
# SHELL #2
redis-cli -h 127.0.0.1 -p 28015 -a ${PASSWORD}

# CHECK ALL REDIS KEYS
KEYS *

# READ STREAM
XREAD COUNT 2 STREAMS stagetime-revisionruns writers 0-0 0-0

# DELETE STREAM
DEL stagetime-revisionruns

TEST SERVICE LOCALLY (OUTSIDE CLUSTER)

START CONSUMER
export KUBECONFIG=~/.kube/dev11
export TEMPLATE_PATH=~/projects/go/src/github/stageTime-creator/tests
export TEMPLATE_NAME=job-template.yaml
export REDIS_STREAM=stagetime:test
export REDIS_PASSWORD=<SET-ME>
export REDIS_SERVER=redis-pve.labul.sva.de
export REDIS_PORT=6379
task run
START TEST PRODUCING (EXTERNAL REDIS)
# kubectl -n stagetime-redis port-forward redis-stagetime-deployment-node-0 28015:6379
task run-test
START TEST PRODUCING (REDIS INSIDE CLUSTER)
kubectl -n <REDIS-NS> port-forward redis-stagetime-deployment-node-0 <HOST-PORT>:<CONTAINER-PORT>

# kubectl -n stagetime-redis port-forward redis-stagetime-deployment-node-0 28015:6379

export REDIS_STREAM=stagetime-revisionruns
export REDIS_PASSWORD=<SETME>
export REDIS_SERVER=127.0.0.1
export REDIS_PORT=28015 # HOST-PORT
go run tests/test-json-set.go
VERIFY REDIS
redis-cli -h <REDIS_SERVER>-p <HOST-PORT> -a <SETME>

# redis-cli -h 127.0.0.1 -p 28015 -a test

KEYS *
# GET VALUE
GET <KEYNAME>
# GET STREAM
XREAD COUNT 2 STREAMS <STREAM-NAME> writers 0-0 0-0

LICENSE

APACHE 2.0

Copyright 2023 patrick hermann.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Author Information

Patrick Hermann, stuttgart-things 06/2023

stagetime-creator's People

Contributors

ankitsharma12-ops avatar patrick-hermann-sva avatar

Stargazers

 avatar

stagetime-creator's Issues

EXTRACT VARIABLES FROM TEMPLATE

THE EXEPECTED RESULT FROM THE EXAMPLE CODE SHOULD BE WORK W/ EVERY GOLANG TEMPLATE STRING (renderedJobManifest)

package main

import (
	"fmt"
	"regexp"
)

const renderedJobManifest = `
apiVersion: batch/v1
kind: {{ .Kind }}
metadata:
  name: {{ .Name }}
  namespace: machine-shop-{{ .Name }}
`

func main() {

	foundVarsInRegex := GetAllRegexMatches(renderedJobManifest, `\{\{(.*?)\}\}`)
	fmt.Println(foundVarsInRegex)

	// REMOVE BRACKETS
	// REMOVE DOT
	// REMOVE DUPLICATES

	expectedResult := []string{"Kind", "Name"}
	fmt.Println("\n\n\nEXPECTED RESULT - ALL VAR NAMES FROM TEMPLATE:")

	for _, variable := range expectedResult {
		fmt.Println(variable)
	}

}

func GetAllRegexMatches(scanText, regexPattern string) []string {

	re := regexp.MustCompile(regexPattern)

	return re.FindAllString(scanText, -1)
}

[FEATURE] - USE REDIS JSON FOR GETTING/RENDERING PIPELINERUNS

PREPARATION

  • DEPLOY REDIS STACK /W PASSWORD + REDIS INSIGHTS

CREATE TEST DATA ON REDIS (STAGE)

  • CREATE TEST PR OBJECTS ON REDIS JSON (STAGE = MULTIPLE PRs)
  • CREATE REDIS SET W/ ALL PRs PER STAGE

VALDIATION/RENDERING

  • VALIDATE REQUIRED STAGE DATA FROM STREAM (STAGE ID) ON SET (AT LEAST ONE PR MUST EXIST)
  • VALIDATE ALL PR OBJECTS FROM REDIS SET (DOES ALL PRs EXISTS AS JSON OBJECTS)
  • RENDER ALL PIPELINERUNS
  • CREATE ALL PIPELINERUNS ON CLUSTER

ADD TEST CASES TO THE RESULT CODE OF EXTRACT VARIABLES FROM TEMPLATE #1

for those two functions we need test cases (each)!

example! try it out to understand!

cat <<EOF > ankit.go
package main

import "fmt"

func Sum(x int, y int) int {
        return x + y
}

func main() {
        Sum(5, 5)
        fmt.Println(Sum(5, 5))
}
EOF

cat <<EOF > ankit_test.go
package main

import "testing"

func TestSum(t *testing.T) {

	type test struct {
		number1 int
		number2 int
		want    int
	}

	tests := []test{
		{number1: 5, number2: 2, want: 7},
		{number1: 6, number2: 9, want: 15},
		{number1: 7, number2: 112, want: 119},
	}

	for _, tc := range tests {

		total := Sum(tc.number1, tc.number2)

		if total != tc.want {
			t.Errorf("Sum was incorrect, got: %d, want: %d.", total, tc.want)
		}

	}

}

EOF

go mod init ankit
go mod tidy

go run ankit.go
go test -v

we want to test those two functions w/ proper test cases

e.g. input: []string{"Kind", "Name", "Name"}
expected output: []string{"Kind", "Name"}

func removeDuplicateStr(results []string) []string {
    allKeys := make(map[string]bool)
    list := []string{}
    for _, item := range results {
        if _, value := allKeys[item]; !value {
            allKeys[item] = true
            list = append(list, item)
        }
    }
    return list
}
func GetAllRegexMatches(scanText, regexPattern string) []string {

	re := regexp.MustCompile(regexPattern)

	return re.FindAllString(scanText, -1)
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.