Giter Club home page Giter Club logo

devoxx-uk-2019-observability-workshop's Introduction

Devoxx UK 2019 Observability Workshop

A hands on playground for learning observability basics in a hands on fashion.

Prework

Please have docker installed and working and run the following to get all the infrastructure up and running before getting to the conference.

Any questions or things not working reach out on twitter to @beckje01

{project}/infra/$ docker-compose up

FIX for Log Index

curl -XPOST -D- 'http://127.0.0.1:5601/api/saved_objects/index-pattern' -H 'Content-Type: application/json' -H 'kbn-version: 6.6.1' -d '{"attributes":{"title":"logstash-*","timeFieldName":"@timestamp"}}'

Notes

The labs all work best if you move into that folder or the subprojects folder to run gradle commands.

cd {project}/lab-01/edge-ratpack/
gradle run
  • links.html[Link Page for Local Services]

  • Each Lab has a TASKS.adoc file

  • Presentation PDF

devoxx-uk-2019-observability-workshop's People

Stargazers

 avatar  avatar  avatar

Watchers

 avatar

Forkers

dmg46664

devoxx-uk-2019-observability-workshop's Issues

Making my way through Lab01

Hi Jeff,
Thanks for providing this online. I'm going to try make my way through these.

Please see some inline comments on your instructions below.

== Get Logs to One Place

With MicroServices it is a must to have a single place to go through logs for everything at once. For this workshop we will be using the ELK stack, hosted locally.

The work will be to setup logging targets of logstash for all the microservices.

⚠️ We haven't introduced logstash yet in the original guide!? (How is someone unfamiliar supposed to know this is the L in ELK?), or what its purpose is. It's okay if you have mentioned this in the talk, but it needs to be in the notes as well in case anyone isn't following.

== Dynamic Log Filtering

Now that we have all the logs in one place, we quickly start having too many logs.

⚠️ We haven't done anything yet to put all the logs in one place? Are there no further instructions?

Next know we want some debug level logging but not all of it. Using Turbo Filters in logback is a great place to start. DynamicThresholdFilter will allow the most flexibility.

Checkout the logback https://logback.qos.ch/manual/filters.html[filter docs].

== Formatting

I don't think it's wise to have guidance in both a pdf and task file. Ideally there should be one source of truth to act as a guide and I would make it the task file (otherwise a reader's concentration has to be split).

I think that each lab should fully introduce and contain all the information required.
I.e. the following:

Lab 01 rewrite

docker-compose up starts up various programs defined in infra/docker-compose.yml and the subdirectories below it.

Next open links.html in the browser. The items under Infra links provides links to the docker containers if they have come online properly.

Kibana & Logstash & Elastic search make up the ELK stack.

  • Logstash collects and transforms the logs.
  • Elastic search stores the transformation.
  • Kibana is a web interface to search these logs.
    See https://www.elastic.co/elk-stack for more information.

Run the first application

edge-ratpack> gradle run
...
... logs should show up
...
<===========--> 90% EXECUTING [1m 5s]
> :lab-01:edge-ratpack:run

But where have the logs been saved to in your application?

Get Logs to One Place

With MicroServices it is a must to have a single place to go through logs for everything at once. For this workshop we will be using the ELK stack, hosted locally.

The work will be to setup logging targets of logstash for all the microservices.

You need to do this now...

Let's start with the first application edge-ratpack.

Hint: Add the following to the logback.xml at the appropriate place.


		<appender-ref ref="stash"/> 

Now relaunch the application

⚠️ Note that you cannot expect users to get familiar with your application structure. Usually Java devs are writing code, and some may not have had to change an appender in years.

⚠️ Further up in the stack I see the following. Ideally there shouldn't be any errors not mentioned in the tutorial.

time=23:12:01.577, loggingId=, level=DEBUG, logger=r.util.internal.TransportDetector, message=io.netty.channel.epoll.Epoll unavailability cause
java.lang.ExceptionInInitializerError: null
        at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:38)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at ratpack.util.internal.TransportDetector$NativeTransport.invokeIsAvailable(TransportDetector.java:180)
        at ratpack.util.internal.TransportDetector$NativeTransport.isAvailable(TransportDetector.java:152)
        at ratpack.util.internal.TransportDetector$NativeTransport.<init>(TransportDetector.java:109)
        at ratpack.util.internal.TransportDetector.determineTransport(TransportDetector.java:40)
        at ratpack.util.internal.TransportDetector.<clinit>(TransportDetector.java:37)
        at ratpack.exec.internal.DefaultExecController.<init>(DefaultExecController.java:61)
        at ratpack.server.internal.DefaultRatpackServer.start(DefaultRatpackServer.java:126)
        at ratpack.server.RatpackServer.start(RatpackServer.java:93)
        at ratpack.groovy.GroovyRatpackMain.main(GroovyRatpackMain.java:38)
Caused by: java.lang.IllegalStateException: Only supported on Linux
        at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:192)
        at io.netty.channel.epoll.Native.<clinit>(Native.java:61)
        ... 14 common frames omitted
time=23:12:01.578, loggingId=, level=DEBUG, logger=r.util.internal.TransportDetector, message=io.netty.channel.kqueue was not found

I cannot see the logs in Kibana. ⚠️ There is no link to logstash in links.html, so I assume that this all just feeds into Kibana by default or via your configuration (this should be mentioned explicitly in the guide), and I've assumed the error above is the reason why it's not working. Does this mean familiarity with a groovy project is required to sort this out? Your guide should ideally mention ALL changes for users to make and why in order for everyone to get the most out of it. If there are multiple applications, then the guide should deal with getting one working at a time. I can see the diff between the answers includes groovy files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.