Giter Club home page Giter Club logo

kubernetes-on-openstack's Issues

Master Secgroups need two terraform applys to be set

I experience the issue that after a terraform apply the security group of the master node is set to the default secgroup instead of the specified two secgroups.
When doing a second terraform apply, terraform recognizes that and sets the correct secgroups.

Used versions:

Terraform v0.11.11
+ provider.local v1.1.0
+ provider.openstack v1.16.0
+ provider.random v2.0.0
+ provider.template v2.1.0

LoadBalancer type Service does not seem to work properly

When exposing a Pod externally with a Service of type LoadBalancer, curling the External-IP exits with curl: (52) Empty reply from server while curling the internal Cluster-IP of the Service is working though.

kubectl shows that the Service exists and an ExternalIP is associated:

NAME           TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)        AGE
nginxservice   LoadBalancer   10.96.239.133   x.x.x.x   80:30512/TCP   86s

To exclude that my resource-configs contain errors, I tried this example from the official OpenStack docs:

---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
   app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginxservice
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: nginx
  type: LoadBalancer

When checking the automatically provisioned load balancer, I found out that the load balancer members were in operating status "ERROR", probably due to failing health monitoring checks.
After deleting the health monitor, load balancer members reach operating status "ONLINE", but the External-IP of the service still cannot be curled successfully.

+---------------------+--------------------------------------------------------------------------------------+
| Field               | Value                                                                                |
+---------------------+--------------------------------------------------------------------------------------+
| admin_state_up      | True                                                                                 |
| created_at          | 2019-05-24T09:24:15.139226                                                           |
| description         | Kubernetes external service ab267c9947e0511e9a6f5fa163e13d14 from cluster kubernetes |
| flavor_id           |                                                                                      |
| id                  | 6896bb42-b40e-4d49-be99-f3484ed1daed                                                 |
| listeners           | fe8b074c-d08a-464f-b19f-4e00d6729b1c                                                 |
| name                | ab267c9947e0511e9a6f5fa163e13d14                                                     |
| operating_status    | ERROR                                                                                |
| pools               | c8779882-bb3a-4b29-bb6e-e611e6f1420b                                                 |
| project_id          | b71e73698acd4e64bf72962e8fcd711f                                                     |
| provider            | amphora                                                                              |
| provisioning_status | ACTIVE                                                                               |
| updated_at          | 2019-05-24T09:25:33.693805                                                           |
| vip_address         | 172.16.0.9                                                                           |
| vip_network_id      | a0bb965c-25e9-4bd2-a748-404d0fb8ee3c                                                 |
| vip_port_id         | e732daf2-5502-4cc5-9dbb-3aba3d5e2c1e                                                 |
| vip_qos_policy_id   | None                                                                                 |
| vip_subnet_id       | 8c97a9e5-fa11-4b6a-8364-529e7387c3b5                                                 |
+---------------------+--------------------------------------------------------------------------------------+

Kubernetes Cluster Setup:

module "example-cluster" {
  source = "git::https://github.com/inovex/kubernetes-on-openstack.git"

  auth_url         = "${var.auth_url}"
  cluster_name     = "example-cluster"
  username         = "${var.username}"
  password         = "${var.password}"
  domain_name      = "${data.openstack_identity_auth_scope_v3.scope.project_domain_name}"
  user_domain_name = "${data.openstack_identity_auth_scope_v3.scope.user_domain_name}"
  tenant_name      = "${data.openstack_identity_auth_scope_v3.scope.project_name}"
  project_id = "${data.openstack_identity_auth_scope_v3.scope.project_id}"

  kubernetes_version        = "1.13.2"
  kubernetes_cni_version    = "0.6.0"
  containerd_version        = "1.2.4"
  cluster_network_router_id = "${openstack_networking_router_v2.router.id}"
  node_count                = "1"
  flavor                    = "c4.large"
  master_data_volume_size   = "10"
  node_data_volume_size     = "10"
  public_network_name       = "public"
}

Setup fails due to version constraint with k8s-cni

Related to kubernetes/kubernetes#75683

When installing kubeadm and kubelet, the cloud-init script fails because of a version constraint with kubernetes-cni as of today:

   kubeadm : Depends: kubernetes-cni (= 0.6.0) but 0.7.5-00 is to be installed
   kubelet : Depends: kubernetes-cni (= 0.6.0) but 0.7.5-00 is to be installed

Solution would be to explicitly define the CNI-Version to be used. I will provide a PR for this.

Incompatible with other openstack providers

The current setup is not compatible with other openstack providers because the public network name and the image visibility are hardcoded. The latter even requires a user to have the privileges to create publicly visible images.

clouds.yaml not usable for specifying auth_url

With the given example setup it is not possible for me to use the clouds.yaml to authenticate against openstack using terraform:

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage.

Error: Error refreshing state: 1 error(s) occurred:
* provider.openstack: One of 'auth_url' or 'cloud' must be specified

However sourcing the openstack rc file works fine for me as described here:
https://github.com/inovex/kubernetes-on-openstack/blob/master/example/Readme.md

Don't use latest version of Image

Currently we use the latest available version of the VM image. This causes a complete recreation of the cluster if one runs the terraform script after the release of a new version of this base image. This will ultimately cause a downtime in production clusters, since the cluster will be empty after recreation. Furthermore, we do not update the nodes in a rolling release.

Image (architecture/docs)

We should create a minimal image how the setup work and which resources are created (no terraform graph).

kube-apiserver does not respond when bootstrapping cluster

Bildschirmfoto 2019-04-27 um 17 29 30

Bildschirmfoto 2019-04-27 um 17 20 03

netstat on the master-instance shows that kube-apiserver exists, but using tcp6 (not sure if this can be an issue), cloud-init seems to complete successfully on the master-instance.
curling kube-apiserver from the nodes also does not work.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.