Giter Club home page Giter Club logo

openshift-prometheus's Introduction

Prometheus overview

Prometheus components

Install Prometheus

https://docs.openshift.com/container-platform/3.11/install_config/prometheus_cluster_monitoring.html Set variables in inventory (small cluster)

openshift_cluster_monitoring_operator_install=true
openshift_cluster_monitoring_operator_prometheus_storage_enabled=true
openshift_cluster_monitoring_operator_prometheus_storage_capacity=50Gi
openshift_cluster_monitoring_operator_prometheus_storage_class_name=[tbd]
openshift_cluster_monitoring_operator_alertmanager_storage_enabled=true
openshift_cluster_monitoring_operator_alertmanager_storage_capacity=2Gi
openshift_cluster_monitoring_operator_alertmanager_storage_class_name=[tbd]
openshift_cluster_monitoring_operator_alertmanager_config=[tbd]

Run the installer

ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/openshift-monitoring/config.yml

Set retention time

oc patch prometheus k8s -p '{"spec":{"retention":"20d"}}' --type='merge'

Prepare Prometheus

Let Prometheus scrape service labels in different namespaces

oc adm policy add-cluster-role-to-user cluster-reader system:serviceaccount:openshift-monitoring:prometheus-k8s

Add custom ruleset

oc apply -f templates/prometheus-k8s-custom-rules.yaml -n openshift-monitoring

Additional targets (etcd)

https://docs.openshift.com/container-platform/3.11/install_config/prometheus_cluster_monitoring.html#configuring-etcd-monitoring

Bug OSE 3.11.53: Mount secret with certs manually

oc set volumes statefulset/prometheus-k8s --add --secret-name=kube-etcd-client-certs --mount-path=/etc/prometheus/secrets/kube-etcd-client-certs/

Additional targets (router)

oc apply -f templates/servicemonitor-router.yaml
oc apply -f templates/clusterrole-router-metrics.yaml
oc adm policy add-cluster-role-to-user router-metrics system:serviceaccount:openshift-monitoring:prometheus-k8s

Additional targets (logging)

oc apply -f templates/servicemonitor-elasticsearch.yaml
oc label svc logging-es-prometheus -n openshift-logging scrape=prometheus
oc apply -f templates/rolebinding_logging.yaml

Alerting

Configuring the alerting rules with the Red Hat Ansible playbooks. https://docs.openshift.com/container-platform/3.11/install_config/prometheus_cluster_monitoring.html#configuring-alertmanager

By hand

oc delete secret alertmanager-main
oc create secret generic alertmanager-main --from-file=templates/alertmanager.yaml

Application Monitoring

Simple REST Example

https://labs.consol.de/development/2018/01/19/openshift_application_monitoring.html

oc new-project rest-test
oc new-app -f https://raw.githubusercontent.com/ConSol/springboot-monitoring-example/master/templates/restservice_template.yaml -n rest-test
oc label svc restservice scrape="prometheus" --overwrite
oc apply -f templates/servicemonitor-application-rest-test.yaml -n openshift-monitoring

JMX Example

https://blog.openshift.com/enhanced-openshift-jboss-amq-container-image-for-production/

oc new-project amq-tests
oc new-build https://github.com/lbroudoux/openshift-cases.git --context-dir="/jboss-amq-custom/custom-amq" --name=custom-amq --strategy=docker --to=custom-jboss-amq-63
watch oc get pods
oc tag amq-tests/custom-jboss-amq-63:latest amq-tests/custom-jboss-amq-63:1.3
oc process -f https://raw.githubusercontent.com/lbroudoux/openshift-cases/master/jboss-amq-custom/custom-amq63-persistent-nosecret.yml     --param="APPLICATION_NAME=custom-broker"     --param="IMAGE_STREAM_NAMESPACE=amq-tests" | oc create -f -
watch oc get pods
oc edit dc custom-broker-amq 
oc label svc custom-broker-amq-tcp scrape=prometheus
oc apply -f templates/servicemonitor-application-amq-tests.yaml -n openshift-monitoring

Additional configuration

Add view role for developers

oc adm policy add-cluster-role-to-user cluster-monitoring-view [user]

Add metrics reader service account to access Prometheus metrics

oc create sa prometheus-metrics-reader -n openshift-monitoring
oc adm policy add-cluster-role-to-user cluster-monitoring-view -z prometheus-metrics-reader -n openshift-monitoring
oc sa get-token prometheus-metrics-reader -n openshift-monitoring

Allow Prometheus to scrape your metrics endpoints (if using ovs-networkpolicy plugin)

Create an additional network-policy.

oc create -f templates/networkpolicy.yaml

Known Issues

openshift-prometheus's People

Stargazers

somsakc avatar  avatar Felix Krohn avatar suvi29 avatar Lukas Grimm avatar  avatar

Watchers

somsakc avatar Felix Krohn avatar

openshift-prometheus's Issues

servicemonitor logging: server returned HTTP status 403 Forbidden

my step:
`shell
oc adm policy add-cluster-role-to-user cluster-reader system:serviceaccount:openshift-monitoring:prometheus-k8s

oc apply -f templates/servicemonitor-elasticsearch.yaml

oc label svc logging-es-prometheus -n openshift-logging scrape=prometheus

oc apply -f templates/rolebinding_logging.yaml

`
prometheus ui:
image

Not sure why 403, thank you very much

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.