sanjay-george / nick-name-generator Goto Github PK

Todo

• Create a random name generator - 2/3 microservices
  Format: {adjective}-{color}-{animal}
  eg: annoying-blue-panda

• Create one endpoint Get that returns static data { name, server }

• Set folder structure according to best standards - monolithic repo

• Set k8 services and deployments for web service to make calls to microservices

• Test on local k8s

• Build animal MS to complete the development

Articles

1. https://expressjs.com/en/guide/routing.html#route-parameters
2. https://nodejs.org/api/http2.html
3. https://learn.hashicorp.com/tutorials/consul/service-mesh

Todo

• Quickly go through GKE concepts
• Deploy nick-name-gen as it is on GKE (using current API gateway) using the Console
• Test and ensure it works properly
• Write tf for creating GKE cluster and deploying

• Build API interface similar to existing node services
• Dockerize
• Deploy to minikube and test
• Update architecture diagram in README

Issue with getting metrics on minikube:

• Enabled metrics addon, still not able to retrieve metrics

https://stackoverflow.com/questions/70838207/kubernetes-unable-to-get-metrics
kubernetes-sigs/metrics-server#41

Todo

• Create scoped public packages for Common directory
  https://www.npmjs.com/package/@sanjay-george/nng-common
• Download and use this package in other services

Commands

npm init --scope=@my-username - for initializig the directory to be packaged
npm publish --access public - for publishing the package

Articles

Follow npm package tutoial here: https://docs.npmjs.com/creating-and-publishing-scoped-public-packages

Further improvements

• Move api gateway itself into an NPM package, once built properly
• Entry point of nng-common is app.js. It contains nothing. Read about common practices for packages with multiple files / utilities

RKE

• Try RKE deployment on AWS : https://github.com/LukeMwila/bootstrap-rke-cluster-in-aws

RKE2

RKE2 is the standard now, uses containerd runtime instead of Docker

• https://github.com/rancherfederal/rke2-aws-tf
• https://docs.rke2.io/

Steps

• Modify terraform script to handle changes for AWS academy

• Run terraform to provision RKE cluster

Incase of issue with deleting internet gateway, delete the ALB
https://stackoverflow.com/questions/37232965/issue-when-trying-to-delete-vpc-and-network-interface

• Create alias to read the kubeconfig
  

• Deploy nick name app

Todo - Create new stories

• Figure out how to get public IP for all Loadbalancer services. Currently external ip remains in <PENDING> state

• This worked, but after very long time. Also, service discovery doesn't seem to be working on path /nick-name
  

• Specify SSH key for the cluster vockey
• Deploy dashboard

• https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
• https://k21academy.com/docker-kubernetes/kubernetes-dashboard/

• Check auto scaling parameters. Tune if needed
• Understand nodepool vs server. Adjust count for HA
• Check the availability zones deployment logic. Fix, if needed.

Articles

• RKE vs RKE2

Rancher on Openstack

• Deploy rancher (class) code on Openstack using Terraform.

• BLOCKER 🚫 Openstack instances not getting provisioned properly. Have to check from Germany

• Play around with this code and understand all the providers and modules used.

• - [ ] Understand what rancher is capable of

• Try adding more worker and control plane nodes on OpenStack

• Deploy an application on cluster

Rancher on AWS

• Try Rancher deployment on AWS Simple deployment official guide

Issues

• "ssh" module doesn't work for win386 machines 🤷‍♂️, fixed by changing local tf installation to amd64 ✅
• issue with cert-manager helm chart.

• Try with WSL ububtu

Still not working

Play 1 - Failed after 53 mins

Play 2 - Failed after 49 mins

• Jest for Node

• XUnit for .NET

Explore how to build an API gateway in node JS

• API Gateway is a Facade Pattern
• Acts like a reverse proxy, but has additional features such as aggregation, handling partial failure, authentication, etc..

Why API gateway?

1. Aggregate multiple calls to avoid roundtrips (series vs parallel calls)
2. The services might be using protocols that aren't web friendly, eg: thrift RPC. Calling from client won't be possible
3. Refactoring of microservices (breaking down, or combining) would need changes on every client.

Features of API Gateway?

1. High performance and scalability

2. Service Discovery

3. Service Invocation (either async - queues, or sync - http/grpc)

4. Handling partial failures

5. Data aggregation (from multiple services)

6. Protocol transformation
   

7. Authentication (cookie / token based )

Articles

Check the resources mentioned here : https://github.com/Sanjay-George/Learning-Materials/blob/master/README.md#api-gateway

Getting the following error message

Steps followed:

1. Use the provided terraform script to deploy all infra on AWS
2. Update remote user as mentioned here: https://schoolofdevops.github.io/ultimate-kubernetes-bootcamp/cluster_setup_kubespray/#:~:text=Set%20Remote%20User%20for%20Ansible
3. Run ansbile playbook cluster.yml pointing to the inventory created by tf.

Since etcd works inside K8 cluster to discover services and most of the services are not exposed externally, there should be a better way to run and test the application on local.

Check image in comments describing the issue with current architecture.

Possible solutions

• keep seaprate config for local and update ports for each service

• Keep separate YAML files for local k8 cluster and make all services LoadBalancer (better solution)
  Use HELM Custom values for managing local vs prod.

HELM

Subcharts and globals

Values

Use custom value files to separate out local and prod config
On local : helm install -f values.yaml -f local-values.yaml bitnami/wordpress --generate-name
On prod: helm install -f values.yaml bitnami/wordpress --generate-name

YouTube - intro to helm

Blogs

https://medium.com/@prbldeb/dealing-with-helm-charts-of-micro-services-with-interdependent-api-end-points-2fe9b508e484

https://dev.to/pavanbelagatti/how-to-create-a-simple-helm-chart-12fe

Todo - Create separate stories if needed

• Figure out how to get public IP for all Loadbalancer services. Currently external ip remains in <PENDING> state

• This worked, but after very long time. Also, service discovery doesn't seem to be working on path /nick-name
  

• Specify SSH key for the cluster vockey

• Deploy dashboard

• https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
• https://k21academy.com/docker-kubernetes/kubernetes-dashboard/

• Check auto scaling parameters. Tune if needed

• Understand nodepool vs server. Adjust count for HA

• Check the availability zones deployment logic. Fix, if needed.

For kubespray

https://github.com/gregbkr/kubernetes-kargo-logging-monitoring

kubespray: ansible playbooks, need python to execute ansible. Terraform module only for AWS 😑. But GCP is supported with python 🙂

Blocker

kubespray ansible book failing randomly.
Issue raised here: #4)

Articles

1. Installing kubernetes with kubespray

2. Quickstart setup using kubespray on GCP

Steps followed

1. Install ansible 🐍

• Installation guide

• Issue #1 with ansible not working from cmd line -> reinstall python with defaults

• Issue #2 some 💩 not available
  

• • might not work with windows directly. read this : http://blog.rolpdog.com/2020/03/why-no-ansible-controller-for-windows.html
• • Install ubuntu on wsl

• Install ansible on WSL instead.
• Install terraform on WSL
• Set up virtual env guide and install all deps inside
• Install specific version of ansible as mentioned in requirements.txt

2. Setup dev environment on ubuntu WSL

• Git - ssh, push / pull, etc..

• Facing issue with pushing even after setting up SSH. Find solution here:

• AWS credentials - stored here: $HOME/.aws/credentials
• nano editor
• kubectl CLI

3. Run Terraform to provision the cluster

• make necessary changes to access_key, region, secret_key, ssh_key etc..
• Remove creation of IAM role. Just use the existing LabRole and LabInstanceProfile

• Note: Once tf runs, it creates the inventory file for ansible playbook. Check here

4. Run ansible playbooks to set up k8 on the already-provisioned cluster

• Tutorial 1: https://schoolofdevops.github.io/ultimate-kubernetes-bootcamp/cluster_setup_kubespray/

• Issue #1: #2

• Issue #2: #3

• • SSH into private servers not happening. Two possible solutions:
• • • Eliminate bastion host. Make all nodes public
• • • Assign elastic / static IP to bastion. Fix SSH config for proper connection. (will do later)

• Issue #3: Memory too low for master node. Minimum memory 1.5GB, but free tier t2.micro has only 1 GB.

• • Removed the checks here: Sanjay-George/kubespray-deployment@89b7593

• Issue #4: Play failing randomly with server connection issue

4a. Figure out way to pass in the key through CLI instead of inventory

• Try command ansible-playbook -b -v -i inventory/hosts cluster.yml -u admin --private-key=~/.ssh/vockey
• Or add this in ansible.cfg

private_key_file=~/.ssh/id_rsa_sbexx
remote_user=core

5. Check kube cluster status