I've deployed kubedash to my Google Cloud Compute cluster, and originally it was working great: the information showed matched the information I had gotten from the GCP dashboard. However, the information shown is stuck at around the time I first created the service. I've tried deleting the service and ReplicationController and re-creating them; no luck. Any advice on troubleshooting?

I am at total lost. Not sure what to do next.

$ kubectl logs pods/kubedash-3318424355-y0tkm --namespace=kube-system heapster
I0415 13:32:04.754337       1 heapster.go:60] /heapster --source=kubernetes:'' --stats_resolution=30s --sink_frequency=1m
I0415 13:32:04.754444       1 heapster.go:61] Heapster version 0.18.5
I0415 13:32:04.754740       1 kube_factory.go:169] Using Kubernetes client with master "https://10.100.0.1:443" and version "v1"
I0415 13:32:04.754766       1 kube_factory.go:170] Using kubelet port 10255
I0415 13:32:04.756549       1 heapster.go:71] Starting heapster on port 8082
E0415 13:32:04.799362       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:32:30.001756       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:33:00.001365       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:33:30.003154       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:34:00.001975       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:34:30.001384       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:35:00.001507       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:35:30.001034       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)
E0415 13:36:00.077206       1 kube_events.go:124] Failed to load events: the server has asked for the client to provide credentials (get events)

root       913     1  0 22:38 ?        00:00:06 /usr/bin/kubelet --register-node=false --allow-privileged=true --config=/etc/kubernetes/manifests --cluster-dns=10.100.0.10 --cluster_domain=cluster.local --logtostderr=true
root      1107   785  4 22:38 ?        00:00:57 /hyperkube apiserver --allow-privileged=true --bind-address=192.168.122.163 --secure-port=443 --etcd-servers=http://127.0.0.1:2379 --service-cluster-ip-range=10.100.0.0/24 --admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota --service-account-key-file=/etc/kubernetes/ssl/apiserver-key.pem --tls-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem --tls-cert-file=/etc/kubernetes/ssl/apiserver.pem --client-ca-file=/etc/kubernetes/ssl/ca.pem --logtostderr=true
root      1122   785  0 22:38 ?        00:00:04 /hyperkube controller-manager --master=http://127.0.0.1:8080 --service-account-private-key-file=/etc/kubernetes/ssl/apiserver-key.pem --root-ca-file=/etc/kubernetes/ssl/ca.pem
root      1139   785  0 22:38 ?        00:00:00 /hyperkube scheduler --master=http://127.0.0.1:8080
root      1165   785  0 22:38 ?        00:00:00 /hyperkube proxy --master=http://127.0.0.1:8080 --proxy-mode=iptables

Hello,

When I deploy bundle.yaml I got the next error in the pod

[root@kubernetes ~]# kubectl create -f kubedash/deploy/bundle.yaml
deployment "kubedash" created
service "kubedash" created

[root@kubernetes ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
kubedash-3370066188-uhz6o 1/2 CrashLoopBackOff 1 14s

[root@kubernetes ~]# kubectl logs kubedash-3370066188-uhz6o
Error from server: a container name must be specified for pod kubedash-3370066188-uhz6o, choose one of: [kubedash heapster]

Thanks.

I tried

$ kubectl create -f deploy/bundle.yaml 
error validating "deploy/bundle.yaml": error validating data: the server could not find the requested resource; if you choose to ignore these errors, turn validation off with --validate=false

Here are my configuration:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.0", GitCommit:"b57e8bdc7c3871e3f6077b13c42d205ae1813fbd", GitTreeState:"clean"}
Server Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.0", GitCommit:"b57e8bdc7c3871e3f6077b13c42d205ae1813fbd", GitTreeState:"clean"}
$ kubectl get nodes
NAME           LABELS                                                 STATUS    AGE
kube-master    kubernetes.io/hostname=kube-master,role=loadbalancer   Ready     1h
kube-minion1   kubernetes.io/hostname=kube-minion1                    Ready     1d
kube-minion2   kubernetes.io/hostname=kube-minion2                    Ready     1d

Thanks for any help :)

Pod Metadata can be presented in the pod view page, such as the corresponding service or replication controller.

Alternatively, a new Service view can be added, where the corresponding pods are listed at the bottom. The service view shows different information than other utilization views, as we are interested in the average utilization across all pods of that service, as well as identifying outliers.

Suggestion: line chart of the average utilization across all pods, with error bars for the maximum offset across all pods.

Even though there probably isn't a stable yet, releases published as docker images should be tagged in git.

@afein @vishh

According to popular demand, the Day field should be populated even if not 24 hours have elapsed. The Uptime field of the Information box can be used to identify that the numbers there do not reflect an actual day's values.

Moreover, there should be separate handling in the underlying DayStore for the Max field of a Day, and the percentiles/averages. Right now it is possible that the Max of the last hour is larger than the Max of the last Day.

This Issue is not related to the current state of heapster, and should probably be addressed before the corresponding PR is merged in heapster.

404 on the cross-referenced http://kubernetes.io/img/desktop/nav_logo.svg

I think it should be:

http://kubernetes.io/images/nav_logo.svg

It would be great to be able to see graphs of network rx/tx for cluster/nodes/pods. How hard do you think it would be to implement this?

The Ansible contrib playbooks function in such a way that the current kube-config.yaml file fails to parse. This is because both the rc and the svc are defined in the same file. I propose that we break these into two separate yaml files, as well as add the necessary kubernetes.io/cluster-service: "true" label to each. This falls in line with the other kube-system services that are deployed with these playboooks.

See https://raw.githubusercontent.com/GoogleCloudPlatform/kubernetes/master/cluster/addons/kube-ui/kube-ui-rc.yaml for a functioning example.

All current listings (nodes/namespaces/pods/containers) are randomly ordered. Instead, they could be tables that list the latest CPU and memory Usage, allowing the user to sort them.

This would facilitate the use case of viewing the top memory and cpu consumers in an easy top-down UX.

The backend is still sending shares (1024 per core) as limit values, leading to slightly skewed utilization values.

Pretty much self-explanatory :) We can probably do some syntax checks for HTML/CSS on top of the usual go build/test.

Specifically, the image location and memory limit fields contain invalid values.

Hi,

I have deployed kubedash on a 1.2 K8s cluster with deploy/bundle.yaml, and I sometimes have "503 Service Unavailable" error.
No more GET requests appears in kubedash container logs, and heapster container have same logs as usual

2016/09/06 08:18:56 http: multiple response.WriteHeader calls
2016/09/06 08:18:56 http: multiple response.WriteHeader calls
E0906 08:19:07.917226       1 model_handlers.go:757] timestamp argument cannot be parsed: parsing time "2016-09-01T10:22:00 02:00" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "Z07:00"
2016/09/06 08:19:07 http: multiple response.WriteHeader calls
E0906 08:19:07.917390       1 model_handlers.go:626] unable to get pod metric: the requested pod is not present in the specified namespace
2016/09/06 08:19:07 http: multiple response.WriteHeader calls
E0906 08:19:08.010443       1 model_handlers.go:757] timestamp argument cannot be parsed: parsing time "2016-09-01T10:22:00 02:00" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "Z07:00"
2016/09/06 08:19:08 http: multiple response.WriteHeader calls
E0906 08:19:08.010536       1 model_handlers.go:626] unable to get pod metric: the requested pod is not present in the specified namespace
E0906 08:19:25.314026       1 model_handlers.go:757] timestamp argument cannot be parsed: parsing time "2016-09-01T10:22:00 02:00" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "Z07:00"
2016/09/06 08:19:25 http: multiple response.WriteHeader calls
E0906 08:19:25.423170       1 model_handlers.go:757] timestamp argument cannot be parsed: parsing time "2016-09-01T10:22:00 02:00" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "Z07:00"
2016/09/06 08:19:25 http: multiple response.WriteHeader calls

I have to restart pod to make it work

The model currently does not perform any garbage collection, which leads to very cluttered UI listings of pods and containers.

Ideal path: integrate kubernetes events into heapster and cross-reference pod existence to perform cleanup.

Shortcut: When kubedash requests a pod list or container list, check which entities do not have data in the past X*modelResolution time, and delete them

When I access the kubedash interface, it shows me no values (the only values shown are the nodes memory and cpu usages).

Kubernetes version:

Server Version: version.Info{Major:"1", Minor:"0+", GitVersion:"v1.0.0-290-gb2dafdaef5acea", GitCommit:"b2dafdaef5aceafad503ab56254b60f80da9e980", GitTreeState:"clean"}

Heapster version: v0.18.0 (using the deployment files from the readme)

I checked the pod logs for kubedash and it showed an "error":

E0820 09:02:10.610516       1 handlers.go:61] GET http://10.100.154.133:80/api/v1/model/metrics/cpu-usage?start=1970-01-01T00%3A00%3A00.000Z responded with status code: 500

I tried to curl the address and I get the following output:

curl -L http://10.100.154.133:80/api/v1/model/metrics/cpu-usage\?start\=1970-01-01T00%3A00%3A00.000Z  
��      n������   �0
                  ��Un�b�h�L19�l�煒�������<��Xk|hD�ƹ�R�?������2

The only things I changed, were the namespace for kubedash and I had to add the argument "--heapster_namespace=kube-system" because my heapster instance is running under the "kube-system" namespace.

Is there something wrong with my heapster setup?

Kubedash should be able to provide notifications for potential resource recommendations for all containers that do not have limits specified. The exact way this can be achieved is still to be determined

As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.

The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".

Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)

Thanks so much, let me know if you have any questions.

(This issue was generated from a tool, apologies for any weirdness.)

[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md

Hi,

I could not find setup instructions for setting up kubedash. Could someone pls. add it in the README or comment here and i'll be happy to send a PR.

• Peeyush

The kubernetes logo is loaded from kubernetes.io but the file is no longer where it used to be

Hello,
In the ReadMe you have mentioned that, Kubedash provides additional authentication. Could you provide some more information about this?

Thanks,
Shilpa

I think there should be an explanation in the README.md on how kubedash compares to https://github.com/kubernetes/dashboard .

Kubernetes UI provides a mechanism to extend its functionality by defining new tabs dynamically.
Kubedash would be a perfect candidate for such an extension.

We still need to work out he details around authentication and heapster dependency.

This is the currently most requested feature. Information regarding crash loops can propagate upwards in cluster entities, showing a warning at each entity about the number of its crash looping sub entities.

Pending: crash loop detection algorithm, when periodically given the created time of each container.

hello, i have followed the set up instructions for kubedash

https://github.com/kubernetes/kubedash

But unable to access it on

{}/api/v1/proxy/namespaces/kube-system/services/kubedash

i have kubernetes 1.2, and ran

kubectl create -f deploy/bundle.yaml

running

kubectl get -f deploy/bundle.yaml

Shows

NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kubedash 1 1 1 1 11m
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubedash 10.0.249.107 80/TCP 11m

So it is running, hitting the endpoint to access it will timeout and Chrome will show

took too long to respond.

Any suggestions please?

Thank you

When a limit is not specified, the utilization chart is plotted against the cluster limits.

This behavior is not apparent on the UI, and a warning box should appear in the utilization page to notify the user that the cluster limit is used.

The Information box is currently present in every utilization page, but not being populated. The Uptime is an important piece of information that should be populated in the model and exposed through a new REST endpoint.

During aggregation, each higher-level entity (namespace/cluster) obtains the minimum uptime of all the aggregated children.

In some cases, a shorter uptime is reported by Heapster than the actual uptime of an entity. I have not been able to consistently reproduce the problem, but I will investigate and post further information here.

Hello, my kubernetes cluster is actually a openshift-v3 cluster, due security restrictions heapster should talk to master over TLS.
My problem is kube-dash can't talk to heapster over TLS too, do you have a workaround or is something that will be developed ?

I have a brand new installation of Kubernetes on CoreOs with flannel, i'd like to try kubedash I created the Heapster rc and service, but when I try to run in a pod or on local master docker the afein/kubedash i got the following error:

I0903 14:28:42.422511       1 kubedash.go:82] /kubedash
I0903 14:28:42.422543       1 kubedash.go:83] Kubedash version 0.0.1
I0903 14:28:42.422548       1 kubedash.go:84] Starting kubedash on port 8289
E0903 14:28:42.422574       1 helper.go:251] expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory
F0903 14:28:42.427178       1 kubedash.go:49] unable to locate Heapster service: monitoring-heapster

here my confi files:
heapster-rc.json

{
    "apiVersion": "v1",
    "kind": "ReplicationController",
    "metadata": {
    "labels": {
        "name": "heapster"
    },
    "name": "monitoring-heapster"
    },
    "spec": {
    "replicas": 1,
    "selector": {
        "name": "heapster"
    },
    "template": {
        "metadata": {
        "labels": {
            "name": "heapster"
        }
        },
        "spec": {
        "containers": [
            {
            "image": "afein/heapster:latest",
            "name": "heapster",
            "command": [
                "/heapster",
                "--source=kubernetes:''",
                "--use_model=true",
                "--model_resolution=1m",
                "--cache_duration=4m",
                "--poll_duration=30s",
                "--stats_resolution=10s"
            ]
            }
        ]
        }
    }
    }
}

heapster-service.json

{
  "apiVersion": "v1",
  "kind": "Service",
  "metadata": {
    "labels": {
      "kubernetes.io/cluster-service": "true",
      "name": "monitoring-heapster"
    },
    "name": "monitoring-heapster"
  },
  "spec": {
    "ports": [
      {
        "port": 80,
        "targetPort": 8082
      }
    ],
    "selector": {
      "name": "heapster"
    }
  }
}

kubedash-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: kubedash
  labels: 
    name: kubedash
spec: 
  containers: 
    - name: kubedash
      image: afein/kubedash:latest
      ports: 
        - containerPort: 80
          hostPort: 80
          name: kubedash

The current pod addressing scheme follows the namespace/podname path, which does not reveal information about which pod is running on which node. This information is present in the heapster cache, but not the model.

Steps:

• [Heapster Model] Add a reverse link from each pod to the corresponding node
• In the pod listing view, export the hostname for each Pod.
• Pod Container views are accessed through Pods, which means that the hostname can be passed to the container view from the pod view.