Giter Club home page Giter Club logo

aws-minikube's Introduction

AWS Minikube

AWS Minikube is a single node Kubernetes deployment in AWS. It creates an EC2 host and deploys the Kubernetes cluster using Kubeadm tool. It provides full integration with AWS. It is also able to handle ELB load balancers, EBS disks, Route53 domains and other AWS resources.

Updates

  • 29.4.2024 Update to Kubernetes 1.30.0
  • 31.3.2024 Update to Kubernetes 1.29.3 + Ingress and External DNS add-on updates
  • 18.2.2024 Update to Kubernetes 1.29.2 + Ingress add-on update
  • 30.12.2023 Update to Kubernetes 1.29.0
  • 26.11.2023 Update to Kubernetes 1.28.4
  • 12.11.2023 Update to Kubernetes 1.28.3 + Update some add-ons
  • 15.10.2023 Update to Kubernetes 1.28.2 + Update some add-ons
  • 16.4.2023 Update to Kubernetes 1.27.1 + Use external AWS Cloud Provider
  • 1.4.2023 Update to Kubernetes 1.26.3 + update add-ons (Ingress-NGINX Controller, External DNS, Metrics Server, AWS EBS CSI Driver)
  • 4.3.2023 Update to Kubernetes 1.26.2 + update add-ons (Ingress-NGINX Controller)
  • 22.1.2023 Update to Kubernetes 1.26.1 + update add-ons (External DNS)
  • 10.12.2022 Update to Kubernetes 1.26.0 + update add-ons (AWS EBS CSI Driver, Metrics server)
  • 13.11.2022 Update to Kubernetes 1.25.4 + update add-ons
  • 2.10.2022 Update to Kubernetes 1.25.2 + update add-ons
  • 26.8.2022 Update to Kubernetes 1.25.0 + Calico upgrade

Prerequisites and Dependencies

AWS Minikube deploys into an existing VPC / public subnet. If you don't have your VPC / subnet yet, you can use this configuration to create one.

  • The VPC / subnet should be properly linked with Internet Gateway (IGW) and should have DNS and DHCP enabled.
  • Hosted DNS zone configured in Route53 (in case the zone is private you have to use IP address to copy kubeconfig and access the cluster). To deploy AWS Minikube there are no other dependencies apart from Terraform. Kubeadm is used only on the EC2 host and doesn't have to be installed locally.

Configuration

The configuration is done through Terraform variables. Example tfvars file is part of this repo and is named example.tfvars. Change the variables to match your environment / requirements before running terraform apply ....

Option Explanation Example
aws_region AWS region which should be used eu-central-1
cluster_name Name of the Kubernetes cluster (also used to name different AWS resources) my-minikube
aws_instance_type AWS EC2 instance type t2.medium
ssh_public_key SSH key to connect to the remote machine ~/.ssh/id_rsa.pub
aws_subnet_id Subnet ID where Minikube should run subnet-8d3407e5
ami_image_id ID of the AMI image which should be used. If empty, the latest CentOS 7 image will be used. See README.md for AMI image requirements. ami-b81dbfc5
hosted_zone DNS zone which should be used my-domain.com
hosted_zone_private Is the DNS zone public or private false
addons List of add-ons which should be installed [ "https://raw.githubusercontent.com/scholzj/aws-minikube/master/addons//storage-class.yaml" ]
tags Tags which should be applied to all resources { Hello = "World" }
ssh_access_cidr Network CIDR from which SSH access will be allowed 0.0.0.0/0
api_access_cidr Network CIDR from which API access will be allowed 0.0.0.0/0

Creating AWS Minikube

To create AWS Minikube,

  • Export AWS credentials into environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  • Apply Terraform configuration:
terraform apply --var-file example.tfvars

Deleting AWS Minikube

To delete AWS Minikube,

  • Export AWS credentials into environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  • Destroy Terraform configuration:
terraform destroy --var-file example.tfvars

Using custom AMI Image

AWS Minikube is built and tested on CentOS 7. But gives you the possibility to use their own AMI images. Your custom AMI image should be based on RPM distribution and should be similar to Cent OS 7. When ami_image_id variable is not specified, the latest available CentOS 7 image will be used.

Add-ons

Currently, following add-ons are supported:

  • Kubernetes dashboard
  • Heapster for resource monitoring
  • Storage class and CSI driver for automatic provisioning of persistent volumes
  • External DNS
  • Ingress

The add-ons will be installed automatically based on the Terraform variables.

Custom Add-ons

Custom add-ons can be added if needed. From every URL in the addons list, the initialization scripts will automatically call kubectl -f apply <Addon URL> to deploy it. Minikube is using RBAC. So the custom add-ons have to be RBAC ready.

Tagging

If you need to tag resources created by your Kubernetes cluster (EBS volumes, ELB load balancers etc.) check this AWS Lambda function which can do the tagging.

Frequently Asked Questions

How to access Kubernetes Dashboard

The Kubernetes Dashboard add-on is by default not exposed to the internet. This is intentional for security reasons (no authentication / authorization) and to save costs for Amazon AWS ELB load balancer.

You can access the dashboard easily fro any computer with installed and configured kubectl:

  1. From command line start kubectl proxy
  2. Go to your browser and open http://127.0.0.1:8001/ui

aws-minikube's People

Contributors

jtvhk avatar scholzj avatar the-alchemist avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

aws-minikube's Issues

Cloudinit failing due to Kubernetes repo's GPG key

So I started using this module yesterday, and it worked fine.

Today, the kubeconfig files weren't being generated, and seeing this error within the init-aws-minikube.log file:

Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0x307EA071:
 Userid     : "Rapture Automatic Signing Key (cloud-rapture-signing-key-2021-03-01-08_01_09.pub)"
 Fingerprint: 7f92 e05b 3109 3bef 5a3c 2d38 feea 9169 307e a071
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno -1] repomd.xml signature could not be verified for kubernetes
Trying other mirror.


 One of the configured repositories failed (Kubernetes),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=kubernetes ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable kubernetes
        or
            subscription-manager repos --disable=kubernetes

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=kubernetes.skip_if_unavailable=true

failure: repodata/repomd.xml from kubernetes: [Errno 256] No more mirrors to try.
https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno -1] repomd.xml signature could not be verified for kubernetes

Anyone started seeing this issue? Is this a temporary issue? Any ideas on a fix?

Can't Connect

After installing when trying to ssh in or do

copy_config_dns = To copy the kubectl config file using DNS record, run: 'scp [email protected]:/home/centos/kubeconfig .'

Can't get in with error
[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

Initializing modules... - minikube in Error: Unreadable module directory

Hello,

When i run the terraform init in aws-minikube folder as you provided guided steps, im getting following error. Am i missing minor something ? How can i fix this issue ?

_Test2 Folder has all the files by the way.

C:\Users\andromeda\26042020_Test2>terraform init

Initializing modules...

  • minikube in

Error: Unreadable module directory

Unable to evaluate directory symlink: CreateFile ..\terraform-aws-minikube:
The system cannot find the file specified.

Error: Failed to read module directory

Module directory does not exist or cannot be read.

Error: Unreadable module directory

Unable to evaluate directory symlink: CreateFile ..\terraform-aws-minikube:
The system cannot find the file specified.

Error: Failed to read module directory

Module directory does not exist or cannot be read.

Cannot locate private key after running terraform apply

So to my knowledge I'm going to need the private key that was created during terraform apply so I can ssh into the ec2 instance, but I can't find the private key anywhere on my file system. Is there a default directory or file where that private key is placed?

Cannot connect to kubectl proxy

Hello,

I did a clear installation, and I had few issues

  1. documentation - there is written that it is enough just to run "kubectl proxy", this is not really true, you have to export different port than 8080 first:

export KUBERNETES_MASTER=https://your.domain:6443

  1. documentation - it is not mentioned that if you are using SSL which is enabled by default, you have to use also CA validation, so

kubectl --certificate-authority=/etc/kubernetes/pki/ca.crt

  1. RBAC - In documantation is stated

This is intentional for security reasons (no authentication / authorization)

This is not exactly true with new version of Kubernetes, which is included in actual package. You have to use RBAC authentification according to Kubernetes documentation. I am still fighting how to get rid of this message

"message": "services "https:kubernetes-dashboard:" is forbidden: User "system:anonymous" cannot get services/proxy in the namespace "kube-system"",

After running
kubectl --certificate-authority=/etc/kubernetes/pki/ca.crt proxy

and tunneling it through SSH to my localhost

and accessing

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login

Can you please help. I cannot even get pods even when I use

kubeadm token list

and use selected tokens because tokens has low privileges

[root@ip-10-0-0-18 kubernetes]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
******* 23h 2018-09-30T14:34:31Z authentication,signing system:bootstrappers:kubeadm:default-node-token

******** authentication,signing system:bootstrappers:kubeadm:default-node-token

I need a token with system.master privileges but I have no idea how to get it.

Specify custom AMI?

Where I work, we use Netflix BLESS, and we would like to specify a custom AMI running on the EC2 instance, so we can configured the EC2 instance to trust our self-signed CA.

BLESS uses SSH certificates, added to OpenSSH in 5.4. SSH certificates allow a certificate authority to sign a user’s public key, along with a list of constraints; the user presents this certificate to the server during authentication. The server only needs to trust the CA, and does not need previous knowledge of the user’s public key.

Do you have an interest in an optional parameter for specifying a custom AMI?

No nodes when running kubectl get nodes

kubectl get nodes return no nodes
also we cant create deployments
we get
Type Reason Age From Message


Warning FailedScheduling 3m (x176 over 53m) default-scheduler no nodes available to schedule pods

Can I use kubectl from my host?

Hello, so everything is working fine, If I login into my EC2 instance and type kubectl cluster-info --kubeconfig=kubeconfig it work.

Now I'm wondering if I could have access to my cluster from my computer, do you know how I could configure kubectl to use the remote cluster ? I tried a lot of thing but couldn't get it work.

Thanks!

Can't access the EC2 instance

Terraform runs successfully, ending with:

Apply complete! Resources: 12 added, 0 changed, 0 destroyed.

Outputs:

copy_config = To copy the kubectl config file, run: 'scp centos@aws-minikube-
1.kube.public:/home/centos/kubeconfig .'
kubeadm_token = w8q6uz.w8q6uzdbev9lcskr

minikube_dns = aws-minikube-1.kube.public```

As I don't have DNS configured (using private hosted zone), I try to connect to the instance through the IP address I see in the AWS console. The result :

ssh  [email protected]
ssh: connect to host 35.158.12.108 port 22: Operation timed out

Issues with DNS in kubernetes

Hello,

Firstly, thanks for releasing this. It makes deploying a minikube instance pretty damn easy.

I'm just having an issue with DNS services provided to my pods. The issue is primarily that there isn't any. While I can connect to sites via a bare IP address, I can't connect via DNS.

Each pod is configured to use the kube-dns pod for dns which is then configured to the a DNS server in my VPC, there are other EC2 instances using this DNS service just fine. I've checked the security group (which aws-minikube created anyway) and can't find any restrictions on outgoing traffic. Everything seems fine.

Do you have any idea what might be causing this?

Thanks,

Anthony

not authorized to perform: iam:CreateServiceLinkedRole

I was receiving the following error within the kubernetes service when it was attempting to deploy an ELB from a kubernetes_ingress_v1 resource.

Warning SyncLoadBalancerFailed 2m49s service-controller Error syncing load balancer: failed to ensure load balancer: AccessDenied: User: arn:aws:sts::XXXXXXXXXXXXX:assumed-role/my-minikube/i-0a921073fe8c4d39f is not authorized to perform: iam:CreateServiceLinkedRole on resource: arn:aws:iam::XXXXXXXXXXXXX:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing because no identity-based policy allows the iam:CreateServiceLinkedRole action

Manually creating the following policy and attaching it to the role created by aws-minikube resolved the issue:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "iam:CreateServiceLinkedRole",
            "Resource": "*",
            "Condition": {
                "StringLike": {
                    "iam:AWSServiceName": "elasticloadbalancing.amazonaws.com"
                }
            }
        }
    ]
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.