helm / kind-action Goto Github PK

View Code? Open in Web Editor NEW

278.0 10.0 70.0 15.87 MB

A GitHub Action for Kubernetes IN Docker - local clusters for testing Kubernetes

Home Page: https://kind.sigs.k8s.io/

License: Apache License 2.0

Shell 85.55% JavaScript 14.45%

kubernetes helm charts ci-cd actions kind

kind-action's Introduction

Kind Action

A GitHub Action for Kubernetes IN Docker - local clusters for testing Kubernetes using kubernetes-sigs/kind.

Usage

Pre-requisites

Create a workflow YAML file in your .github/workflows directory. An example workflow is available below. For more information, reference the GitHub Help Documentation for Creating a workflow file.

Inputs

For more information on inputs, see the API Documentation

version: The kind version to use (default: v0.21.0)
config: The path to the kind config file
node_image: The Docker image for the cluster nodes
cluster_name: The name of the cluster to create (default: chart-testing)
wait: The duration to wait for the control plane to become ready (default: 60s)
verbosity: info log verbosity, higher value produces more output
kubectl_version: The kubectl version to use (default: v1.28.6)
install_only: Skips cluster creation, only install kind (default: false)
ignore_failed_clean: Whether to ignore the post delete cluster action failing (default: false)

Example Workflow

Create a workflow (eg: .github/workflows/create-cluster.yml):

name: Create Cluster

on: pull_request

jobs:
  create-cluster:
    runs-on: ubuntu-latest
    steps:
      - name: Create k8s Kind Cluster
        uses: helm/kind-action@v1

This uses @helm/kind-action GitHub Action to spin up a kind Kubernetes cluster on every Pull Request. See @helm/chart-testing-action for a more practical example.

Code of conduct

Participation in the Helm community is governed by the Code of Conduct.

kind-action's People

Contributors

Stargazers

Watchers

kind-action's Issues

Make kubectl version confgurable, as not all versions are fully compatible across all clusters

Allow to specify knkubernetes version

I think it would be very helpful to allow to specify k8s api version in the same way as kubectl_version so that we can test charts against different k8s api version before upgrading k8s control planes ie.

Cut release to address Node.js 16 deprecation warnings

Hey @davidkarlsen,

I see that you approved and merged #102 addressing the Node.js 16 warning in GH actions:

Would you mind cutting a release including this change, please? That would address the deprecation warning when updating the action.

Thanks in advance!

It takes longer to set up a Kind cluster with this action than Minikube

I'm switching to Kind to get better performance but I'm actually getting a performance regression after switching from Minikube: relaycorp/awala-gateway-internet@16a3230

With that Minikube action, I'd have a cluster set up in 1m35s give or take. With this action, it takes more than 2 minutes.

I'd expect this action to be as fast (if not faster) than the original one for Minikube. Maybe I'm missing something?

Here's the output from this action in case it's relevant:

Installing kind...
Adding kind directory to PATH...
Installing kubectl...
Adding kubectl directory to PATH...
kind v0.9.0 go1.15.2 linux/amd64
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.1", GitCommit:"206bcadf021e76c27513500ca24182692aabd17e", GitTreeState:"clean", BuildDate:"2020-09-09T11:26:42Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Creating kind cluster...
Creating cluster "default" ...
 • Ensuring node image (kindest/node:v1.19.1) 🖼  ...
 ✓ Ensuring node image (kindest/node:v1.19.1) 🖼
 • Preparing nodes 📦   ...
 ✓ Preparing nodes 📦 
 • Writing configuration 📜  ...
 ✓ Writing configuration 📜
 • Starting control-plane 🕹️  ...
 ✓ Starting control-plane 🕹️
 • Installing CNI 🔌  ...
 ✓ Installing CNI 🔌
 • Installing StorageClass 💾  ...
 ✓ Installing StorageClass 💾
 • Waiting ≤ 1m0s for control-plane = Ready ⏳  ...
 ✓ Waiting ≤ 1m0s for control-plane = Ready ⏳
 • Ready after 54s 💚
Set kubectl context to "kind-default"
You can now use your cluster with:

kubectl cluster-info --context kind-default

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂

Note the Ready after 54s. Though I'm not sure how reliable that is.

config path not found

If my kind.config.yaml is located in git root repository in folder scripts that won't work:

jobs:
  create-cluster:
    runs-on: ubuntu-latest
    steps:
      - name: Create k8s Kind Cluster
        uses: helm/[email protected]
        with:
          config: ./scripts/kind.config.yaml

How can i point the config path to the proper file?
That won't work also: $GITHUB_WORKSPACE/scripts/kind.config.yaml

It there a way to use this action on a rootless docker setup?

https://kind.sigs.k8s.io/docs/user/rootless/#creating-a-kind-cluster-with-rootless-docker

I have tried following in my workflow:

job:
  test:
    env: 
      DOCKER_HOST: unix://${XDG_RUNTIME_DIR}/docker.sock
    steps:
      - name: Create kind v1.19.7 cluster
        uses: helm/[email protected]
        with:
          node_image: kindest/node:v1.19.7

However that fails with

Run helm/[email protected]
  with:
    node_image: kindest/node:v1.19.7
  env:
    DOCKER_HOST: unix://${XDG_RUNTIME_DIR}/docker.sock
    pythonLocation: /home/runners/actions-runner/_work/_tool/Python/3.7.10/x64
    LD_LIBRARY_PATH: /home/runners/actions-runner/_work/_tool/Python/3.7.10/x64/lib
    CT_CONFIG_DIR: /home/runners/actions-runner/_work/_tool/ct/v3.3.1/x86_64/etc
    VIRTUAL_ENV: /home/runners/actions-runner/_work/_tool/ct/v3.3.1/x86_64/venv
Adding kind directory to PATH...
Adding kubectl directory to PATH...
kind v0.9.0 go1.15.2 linux/amd64
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.1", GitCommit:"206bcadf021e76c27513500ca24182692aabd17e", GitTreeState:"clean", BuildDate:"2020-09-09T11:26:42Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Creating kind cluster...
ERROR: failed to create cluster: failed to list clusters: command "docker ps -a --filter label=io.x-k8s.kind.cluster=chart-testing --format '{{.Names}}'" failed with error: exit status 1

Command Output: parse unix://${XDG_RUNTIME_DIR}/docker.sock: invalid character "{" in host name

I think this action needs to have a parameter to run as rootless so this can be handled within the shell script of this action.

Question: Support for PersistentVolumeClaims?

Does kind-action support (and Github allows) PersistentVolumeClaims while running K8s inside a Github Action? I have seen an Insufficient CPU error while testing a RabbitMQ StatefulSet and it made me wonder if the resource limits are the issue or whether it is even possible to create persistent volumes during the lifecycle of GA.

Helm Kind action seems broken

To me it looks like the helm/kind-action is broken now in general. It fails to set up the network.

Run helm/[email protected]
Installing kind...
Adding kind directory to PATH...
Installing kubectl...
Adding kubectl directory to PATH...
kind v0.9.0 go1.15.2 linux/amd64
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.1", GitCommit:"206bcadf021e76c27513500ca24182692aabd17e", GitTreeState:"clean", BuildDate:"2020-09-09T11:26:42Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Creating kind cluster...
Creating cluster "kind" ...
 • Ensuring node image (kindest/node:v1.19.1) 🖼  ...
 ✓ Ensuring node image (kindest/node:v1.19.1) 🖼
 • Preparing nodes 📦   ...
 ✓ Preparing nodes 📦 
 • Writing configuration 📜  ...
 ✓ Writing configuration 📜
 • Starting control-plane 🕹️  ...
 ✓ Starting control-plane 🕹️
 • Installing CNI 🔌  ...
 ✓ Installing CNI 🔌
 • Installing StorageClass 💾  ...
 ✓ Installing StorageClass 💾
 • Waiting ≤ 5m0s for control-plane = Ready ⏳  ...
 ✗ Waiting ≤ 5m0s for control-plane = Ready ⏳
 • WARNING: Timed out waiting for Ready ⚠️
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂

Also see: https://github.com/ctron/kind-reproducer/runs/2951832350?check_suite_focus=true

Note: The fact that job is finished completed seems to be another issue with this action: see #41

Why is cleanup of kind cluster required?

Hi 👋

I was wondering why the cleanup.sh script is required to manually delete the kind cluster. As far as I understand after the job is completed all the containers are anyways removed what additional benefit is gained by manually running kind delete cluster?

tag v1 is not pointing to v1.8.0

👋 it looks like v1 is not pointing to v1.8.0, but rather pointing to v1.4.0, raise this issue for some awareness. Thanks!

How to mount host directory to kind pod in GA?

Hi team. In minikube, I can use minikube mount <source directory>:<target directory> to complete it. What's steps in kind-action? Shall we need input parameter to support?
Any help will be appreciated.

kind-action doesn't start with custom arc runners

I'm having trouble with this action on custom arc runners https://github.com/actions/actions-runner-controller
Any help would be appreciated.
11_Post Run [email protected]
4_Run [email protected]

the install_only option isn't compatible with the listed version

The quick-start says to use a version that doesn't support the install_only options listed. Can we cut a new version?

kind install fails

Run helm/kind-action@master
Installing kind...
Installing kubectl...
Creating kind cluster...
Creating cluster "chart-testing" ...
• Ensuring node image (kindest/node:v1.16.3) 🖼 ...
✓ Ensuring node image (kindest/node:v1.16.3) 🖼
• Preparing nodes 📦 ...
✓ Preparing nodes 📦
• Writing configuration 📜 ...
✓ Writing configuration 📜
• Starting control-plane 🕹️ ...
✓ Starting control-plane 🕹️
• Installing CNI 🔌 ...
✓ Installing CNI 🔌
• Installing StorageClass 💾 ...
✓ Installing StorageClass 💾
• Joining worker nodes 🚜 ...
✓ Joining worker nodes 🚜
• Waiting ≤ 1m0s for control-plane = Ready ⏳ ...
✓ Waiting ≤ 1m0s for control-plane = Ready ⏳
• Ready after 25s 💚

then:

Set kubectl context to "kind-chart-testing"
You can now use your cluster with:

kubectl cluster-info --context kind-chart-testing

Have a nice day! 👋
`kind get kubeconfig-path` is deprecated!

KIND will export and merge kubeconfig like kops, minikube, etc.
This command is now unnecessary and will be removed in a future release.

For more info see: https://github.com/kubernetes-sigs/kind/issues/1060
See also the output of `kind create cluster`

mv: '/home/runner/.kube/config' and '/home/runner/.kube/config' are the same file
##[error]Node run failed with exit code 1

https://github.com/evryfs/helm-charts/commit/a25539817a43b939e13a398b4622b1cbae3fd784/checks?check_suite_id=368305548

When kind fails to deploy, the GitHub action should fail

When a Kind deploy fails, then the GitHub action should fail too:

The deployment did not become ready, and the Kind cluster was not usable. The cross mark indicates that something has failed. And yet the GitHub action completely successfully.

Release management: improve semantic versioning

Present tag $MAJOR_VERSION (in this case: v1) pointing to the last released version. In that case all action consumers can stick to the v1 version, without needing to keep track of version changes with dependabot and updating their caller actions each time.

After installation how do I connect to it?

It shows how to run this action to create a cluster but I need to then have the kubectl context configured to interact with the cluster.

The example in the main readme seems to not show that, and I can't seem to find any documentation on it. How does one connect to the cluster after it is created?

kind 0.9.0 --loglevel is deprecated, please switch to -v and -q

The action should replace the --loglevel with -v and -q to set loglevel.

Installing kind...
Adding kind directory to PATH...
Installing kubectl...
Adding kubectl directory to PATH...
kind v0.9.0 go1.15.2 linux/amd64
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.1", GitCommit:"206bcadf021e76c27513500ca24182692aabd17e", GitTreeState:"clean", BuildDate:"2020-09-09T11:26:42Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Creating kind cluster...
WARNING: --loglevel is deprecated, please switch to -v and -q!
Creating cluster "chart-testing" ...

post-action clean up job occasional failed on self-hosted github action runners

I have seen the cleanup jobs fails on our ci for multiple times in our self-hosted github runners

ERROR: failed to delete cluster "local": failed to delete nodes: command "docker rm -f -v local-control-plane" failed with error: exit status 1

Command Output: Error response from daemon: Could not kill running container 25a65[4] xxxx cannot remove - tried to kill container, but did not receive an exit event

This might be caused by kubernetes-sigs/kind#708.

Is it possible to add an option input failed_on_cleanup to force kind delete to return ture eg:

   kind delete xxx || true # if failed_on_cleanup set to false

bump the version

Hey,
Please could you release a new version that includes the kubectl_version re #47 (comment)
Thanks

Post action clean up

Normally, we don't need to perform any post action clean up (e.g. remove kind cluster) with public github action. However, for self-hosted github action, kind will fail if there is existing cluster with the same name.

We can just add the support for post action clean up, which removes kind given kind cluster.

kubectl_version input is advertised in readme but unexpected IRL

Hello and thanks for this action!

In the readme it is advertised that we can set the kubectl version

However when I run this action on github I see the warning:
Warning: Unexpected input(s) 'kubectl_version', valid inputs are ['version', 'config', 'node_image', 'cluster_name', 'wait', 'log_level']

My action looks like:

    steps:
      - name: Create k8s Kind Cluster
        uses: helm/[email protected]
        with:
          version: v0.13.0
          kubectl_version: v1.24.0
          cluster_name: kinder

when I check the versions in my action I see:

kubectl version --short
Client Version: v1.20.8
Server Version: v1.24.0

which isn't the advertised default version of 1.20.10 (same line/link above into the readme.md) and definitely violates the k8s "version skew" directive since I've selected kind v13 which defaults to k8s 1.24.0.

Is there a way to set the kubectl version?

Kind doesn't work on Ubuntu ARM64

After 4dc5550, our GH Actions runners in Equinix (Ubuntu) can't detect the arch properly, throwing this error:
/root/actions-runner-3/_work/_actions/helm/kind-action/main/kind.sh: line 66: arch: unbound variable

When I run uname -m I get aarch64, which is not supported in the new case

root@keda-github-actions-runner-arm:~# uname -m
aarch64

The previous behaviour (dpkg --print-architecture) returned arm64

syntax error near unexpected token `newline'

In my Github action, sometimes using kind-action will have problem installing cluster fail

The error logs are:

Run helm/[email protected]
  with:
/opt/hostedtoolcache/kind/v0.11.1/x86_64/kind/bin//kind: line 1: syntax error near unexpected token `newline'
/opt/hostedtoolcache/kind/v0.11.1/x86_64/kind/bin//kind: line 1: `<?xml version="1.0" encoding="UTF-8"?>'
Installing kind...
Adding kind directory to PATH...
Installing kubectl...
Adding kubectl directory to PATH...

kind action ignores node image argument

kind action ignores image argument and uses kindest/node:v1.17.0 image all the time

my config:

https://github.com/kiwigrid/helm-charts/blob/master/.github/workflows/ci.yaml#L58

workflow where the configured versions are not used:

https://github.com/kiwigrid/helm-charts/runs/431815301

Feature request: local registry

Hi,

we use helm/chart-testing and tried helm/kind-action but to do a proper test, i needed to push a temp image to the local kind registry.

To achieve this, i switched from helm/kind-action to container-tools/kind-action (https://github.com/container-tools/kind-action) which provides this functionality.

Would be great if you could support this too so we could have all of it from one trusted source.

Tx,

Sigi

Image pull secrets for private repository

Hi! Thank you for developing this action. I'm trying to pull images from private repository on Docker Hub. My Helm Chart works on local machine but not in Github Actions.

I have added the reference to the image pull secrets to service account (templates/serviceaccount.yaml):

{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
...
imagePullSecrets:
  - name: {{ .Values.imageCredentials.name }}
{{- end }}

In Github Actions I'm receiving the following error:

4m40s Warning Failed pod/-skg0mhcv90-microservice-7657f7ff98-hqgfn spec.containers{microservice} kubelet, chart-testing-control-plane Failed to pull image "/microservice:1.0.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io//microservice:1.0.0": failed to resolve reference "docker.io//microservice:1.0.0": failed to authorize: failed to fetch oauth token: unexpected status: 401 Unauthorized 4m56s 2 -skg0mhcv90-microservice-7657f7ff98-hqgfn.167e0ba654861edd
3m23s Warning Failed pod/-skg0mhcv90-microservice-7657f7ff98-hqgfn spec.containers{microservice} kubelet, chart-testing-control-plane Failed to pull image "/microservice:1.0.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io//microservice:1.0.0": failed to resolve reference "docker.io/***/microservice:1.0.0": failed to authorize: failed to fetch oauth token: unexpected status: 429 Too Many Requests 4m18s 2 ***-skg0mhcv90-microservice-7657f7ff98-hqgfn.167e0baf29d75e8a

Improve kubeadm conf to create kind cluster of v1.24.x

Kind action failing to create cluster with image kindest/node:v1.24.0

ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged chart-testing-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1

Command Output: I0511 08:44:02.867950     246 initconfiguration.go:255] loading configuration from "/kind/kubeadm.conf"
W0511 08:44:02.868702     246 common.go:83] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W0511 08:44:02.8[69](https://github.com/logicmonitor/helm-charts-qa/runs/6384658057?check_suite_focus=true#step:11:69)167     246 common.go:83] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 
kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W0511 08:44:02.869420     246 initconfiguration.go:332] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta2, Kind=JoinConfiguration
W0511 08:44:02.8[70](https://github.com/logicmonitor/helm-charts-qa/runs/6384658057?check_suite_focus=true#step:11:70)055     246 common.go:83] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.

wait parameter does not have any effect on actual wait time

Creating a cluster with the following configuration:

      - name: Create kind cluster
        if: steps.list-changed.outputs.changed == 'true'
        uses: helm/kind-action@dda0770415bac9fc20092cacbc54aa298604d140 # v1.8.0
        with:
          wait: 300s

Leads to the following log output:

Log output

Run helm/kind-action@dda0770415bac9fc20092cacbc54aa298604d140
  with:
    wait: 300s
    version: v0.20.0
    cluster_name: chart-testing
    verbosity: 0
    kubectl_version: v1.26.4
    ignore_failed_clean: false
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.10.13/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.10.13/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.13/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.13/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.13/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.13/x64/lib
    CT_CONFIG_DIR: /opt/hostedtoolcache/ct/3.10.1/amd64/etc
    VIRTUAL_ENV: /opt/hostedtoolcache/ct/3.10.1/amd64/venv
Installing kind...
Adding kind directory to PATH...
Installing kubectl...
Adding kubectl directory to PATH...
kind v0.20.0 go1.20.4 linux/amd64
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.4", GitCommit:"f89670c3aa4059d6999cb42e23ccb4f0b9a03979", GitTreeState:"clean", BuildDate:"2023-04-12T12:13:53Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Creating kind cluster...
Creating cluster "chart-testing" ...
 • Ensuring node image (kindest/node:v1.27.3) 🖼  ...
 ✓ Ensuring node image (kindest/node:v1.27.3) 🖼
 • Preparing nodes 📦   ...
 ✓ Preparing nodes 📦 
 • Writing configuration 📜  ...
 ✓ Writing configuration 📜
 • Starting control-plane 🕹️  ...
 ✗ Starting control-plane 🕹️
Deleted nodes: ["chart-testing-control-plane"]
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged chart-testing-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I1108 12:49:46.493748     205 initconfiguration.go:255] loading configuration from "/kind/kubeadm.conf"
W1108 12:49:46.494741     205 initconfiguration.go:332] [config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
[init] Using Kubernetes version: v1.27.3
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I1108 12:49:46.499265     205 certs.go:112] creating a new certificate authority for ca
[certs] Generating "ca" certificate and key
I1108 12:49:46.546647     205 certs.go:519] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [chart-testing-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.18.0.2 127.0.0.1]
[certs] Generating "apiserver-kubelet-client" certificate and key
I1108 12:49:46.796387     205 certs.go:112] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I1108 12:49:46.945107     205 certs.go:519] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I1108 12:49:47.048850     205 certs.go:112] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I1108 12:49:47.111089     205 certs.go:519] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [chart-testing-control-plane localhost] and IPs [172.18.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [chart-testing-control-plane localhost] and IPs [172.18.0.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I1108 12:49:47.599140     205 certs.go:78] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I1108 12:49:47.688165     205 kubeconfig.go:103] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I1108 12:49:47.744624     205 kubeconfig.go:103] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I1108 12:49:47.929695     205 kubeconfig.go:103] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I1108 12:49:47.988203     205 kubeconfig.go:103] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
I1108 12:49:48.048089     205 kubelet.go:67] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I1108 12:49:48.119177     205 manifests.go:99] [control-plane] getting StaticPodSpecs
I1108 12:49:48.119401     205 certs.go:519] validating certificate period for CA certificate
I1108 12:49:48.119468     205 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I1108 12:49:48.119478     205 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I1108 12:49:48.119483     205 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I1108 12:49:48.119489     205 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I1108 12:49:48.119499     205 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I1108 12:49:48.122180     205 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I1108 12:49:48.122205     205 manifests.go:99] [control-plane] getting StaticPodSpecs
I1108 12:49:48.122380     205 manifests.go:125] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I1108 12:49:48.122395     205 manifests.go:125] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I1108 12:49:48.122402     205 manifests.go:125] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I1108 12:49:48.122408     205 manifests.go:125] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I1108 12:49:48.122413     205 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I1108 12:49:48.122419     205 manifests.go:125] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I1108 12:49:48.122422     205 manifests.go:125] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I1108 12:49:48.122936     205 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I1108 12:49:48.122951     205 manifests.go:99] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I1108 12:49:48.123105     205 manifests.go:125] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I1108 12:49:48.123430     205 manifests.go:154] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I1108 12:49:48.123896     205 local.go:65] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I1108 12:49:48.123911     205 waitcontrolplane.go:83] [wait-control-plane] Waiting for the API server to be healthy
I1108 12:49:48.124350     205 loader.go:373] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I1108 12:49:48.125530     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:48.627085     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:49.127390     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:49.626374     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:50.126561     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:50.626746     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:51.127159     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:51.626421     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:52.126637     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:52.626736     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:53.127072     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:53.626323     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:54.126296     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:54.626318     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:55.126453     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:55.626356     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:56.126526     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:56.626769     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:57.126749     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:57.626647     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:58.126881     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:58.627304     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:59.126552     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:49:59.626498     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:00.126385     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:00.626540     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:01.126992     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:01.627133     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:02.126085     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:02.626301     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:03.126542     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:03.626695     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:04.126870     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:04.626904     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:05.127172     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:05.626258     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:06.126454     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:06.626624     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:07.126406     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:07.626360     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:08.126656     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:08.626863     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:09.126934     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:09.626895     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:10.126992     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:10.627235     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:11.126562     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:11.626771     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:12.126973     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:12.626291     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:13.126462     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:13.626739     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:14.126951     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:14.626975     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:15.127203     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:15.626155     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:16.126408     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:16.626541     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:17.126446     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:17.626710     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:18.126990     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:18.627173     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:19.126220     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:19.626117     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:20.126267     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:20.626417     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:21.126642     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:21.626454     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:22.126502     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:22.626470     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:23.126744     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:23.626979     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:24.127150     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:24.626135     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:25.126348     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:25.626479     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:26.126626     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:26.626815     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:27.126781     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:27.627124     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I1108 12:50:28.126297     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:28.626403     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:29.126543     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:29.626405     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:30.126540     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:30.627455     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:31.126649     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:31.627046     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:32.127051     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:32.626444     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:33.126579     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I1108 12:50:33.626825     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:34.126992     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:34.626949     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:35.127135     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:35.626065     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:36.126164     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:36.626629     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:37.126455     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:37.626489     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:38.126822     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:38.627168     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:39.126402     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:39.626275     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:40.126269     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:40.626482     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:41.126732     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:41.626931     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:42.126962     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:42.627388     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:43.126533     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I1108 12:50:43.627046     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:44.127084     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:44.626110     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:45.126470     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:45.626629     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:46.127007     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:46.627079     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:47.126020     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:47.627151     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:48.126364     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:48.626637     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:49.126980     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:49.627142     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:50.126189     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:50.626366     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:51.126568     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:51.626627     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:52.126548     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:52.626778     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:53.127044     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:53.627336     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:54.126519     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:54.626425     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:55.126816     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:55.626760     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:56.127120     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:56.626318     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:57.126202     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:57.626409     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:58.126431     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:58.626619     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:59.126630     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:50:59.626755     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:00.127092     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:00.626273     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:01.126582     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:01.626730     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:02.126812     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:02.626883     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:03.127082     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I1108 12:51:03.626594     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:04.126673     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:04.626880     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:05.126968     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:05.627100     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:06.127047     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:06.626183     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:07.126104     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:07.626073     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:08.126394     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:08.626603     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:09.126557     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:09.626421     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:10.126360     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:10.626736     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:11.126871     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:11.627034     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:12.127034     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:12.627354     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:13.126561     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:13.626814     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:14.126950     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:14.626955     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:15.127183     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:15.626112     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:16.126263     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:16.626395     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:17.126496     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:17.626677     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:18.126892     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:18.627104     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:19.126210     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:19.626819     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:20.127123     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:20.626275     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:21.126526     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:21.626687     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:22.126932     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:22.626938     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:23.127278     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:23.626495     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:24.126699     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:24.626613     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:25.127053     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:25.627245     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:26.126360     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:26.626511     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:27.126453     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:27.626723     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:28.126935     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:28.627189     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:29.126301     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:29.626246     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:30.126150     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:30.626309     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:31.126517     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:31.626620     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:32.126601     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:32.626932     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:33.126827     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:33.627178     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:34.126322     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:34.626514     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:35.126672     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:35.626659     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:36.126792     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:36.627133     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:37.126111     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:37.626384     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:38.126550     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:38.626756     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:39.126981     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:39.626969     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:40.127081     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:40.625944     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:41.126962     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:41.626983     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:42.126976     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:42.627252     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
I1108 12:51:43.126429     205 round_trippers.go:553] GET https://chart-testing-control-plane:6443/healthz?timeout=10s  in 0 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
Unfortunately, an error has occurred:
	timed out waiting for the condition
This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
couldn't initialize a Kubernetes cluster
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init.runWaitControlPlanePhase
	cmd/kubeadm/app/cmd/phases/init/waitcontrolplane.go:108
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:259
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:111
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1598
error execution phase wait-control-plane
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:260
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	cmd/kubeadm/app/cmd/init.go:111
github.com/spf13/cobra.(*Command).execute
	vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
	vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
	cmd/kubeadm/app/kubeadm.go:50
main.main
	cmd/kubeadm/kubeadm.go:25
runtime.main
	/usr/local/go/src/runtime/proc.go:250
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1598

which shows that the waiting time is ~2 minutes, not the ~5 minutes configured. This also is the case when wait is not set, which should default to 60s.

Is there any other configuration I'm overlooking?

Support ARM runners

Even Kind supports ARM arch, this action doesn't support it.
The main problem is that kind.sh scrip uses a hardcoded value for arch instead of using the agent arch.
I'm willing to contribute with the fix if you think that it's interesting

Default cluster_name

It is pretty common to expect action defaults to be the same as the underlying utility. But default cluster_name is chart-testing and the resulting context is kind-chart-testing.
I propose to set cluster_name to kind default.

kind.sh: line 60: dpkg: command not found

Having this action run on AlmaLinux, CentOS, or similar results in error kind.sh: line 60: dpkg: command not found.

Is the limitation to run on Debian distributions done on purpose? Why not using uname -m instead to make the GitHub action distribution agnostic?

Support KIND running inside container

Currently we have KIND running in our Kubernetes CI, where KIND runs inside of a pod/container, which requires the following mounts with the node:

                volumeMounts:
                  - mountPath: /lib/modules
                    name: modules
                    readOnly: true
                  - mountPath: /sys/fs/cgroup
                    name: cgroup
                  - name: dind-storage
                    mountPath: /var/lib/docker
                securityContext:
                  privileged: true
                imagePullPolicy: Always
              volumes:
                - name: modules
                  hostPath:
                    path: /lib/modules
                    type: Directory
                - name: cgroup
                  hostPath:
                    path: /sys/fs/cgroup
                    type: Directory
                - name: dind-storage
                  emptyDir: {}

When running a KIND docker action inside a KIND enabled container (which works on kubernetes) such as:

...
    runs-on: ubuntu-18.04
    container: repo/container:tag
...

It seems to work when running it locally using act (https://github.com/nektos/act), but when running it on the github Actions worker, I get the error Error: Kubernetes cluster unreachable: Get https://127.0.0.1:34221/version?timeout=32s: dial tcp 127.0.0.1:34221: connect: connection refused.

The KIND cluster does seem to get correctly created, but the issue seems to mainly be that the cluster is not reachable. Is this an issue that you have come across before?

Add an option to remove Kind cluster

I would like to create multiple Kind clusters in the same Github Action workflow. That would require deleting a previously created Kind cluster.

Would you accept a contribution to add a new option to remove a Kind cluster?

Example usage:

- name: Create kind cluster #1
  uses: helm/kind-action
    with:
      version: ${{ env.kind_version }}
      config: ${{ env.kind_config }}

- name: Run tests #1
...

- name: Delete kind cluster #1
  uses: helm/kind-action
  with:
     delete: true

- name: Create kind cluster #2
  uses: helm/kind-action
    with:
      version: ${{ env.kind_version }}
      config: ${{ env.kind_config }}

- name: Run tests #2
...

An alternative would be to to always delete a Kind cluster before creating a new one, but that might break existing GHA workflows in which multiple clusters are created with different cluster_name.

Would you accept such a contribution?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.