Initial run requires 2+ GB of RAM about kafka-webview HOT 9 OPEN

@Crim I would suggest small stuffs:

• On Kubernetes liveness and readyness probes are fundamental, we have to leverage them excluding a couple of urls from security, in order to let kubernetes easily access them to understand if the application is healthy and able to receive requests. If I remember well we can even distinguish which kind of info deliver to anonymous users and to logged users. but I have to test it a bit more.
• Separate per default the actuator port from the apis port, in order to avoid interferences between different kind of requests.
• Add Prometheus metrics to leverage observability, more specifically monitoring. This can be easily done just integrating spring actuator with a couple of dependencies. Those will add a new endpoint to the actuator. In a second version we could secure this endpoint with basic auth, because Prometheus is kind of able to perform the right request.

I started already a prototype, that I should refine it a bit, but it seems I'm on the right path.
I have also some ready and tested Kubernetes YAML manifests, they need just integration with probes and metrics.

After this I would review the Dockerfile for the minimum amount of required security (avoid root) and maybe also leveraging a bit more the building cache.

Once done, I would probably also have a look at Helm, to produce a chart.

Please let me know what you think and, if you want, how to proceed :)

from kafka-webview.

Hmm which JVM version are you using and what JVM flags are you passing to start up the service?

I'm able to get it to boot with 256M of heap without any issues:

# java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

# export SPRING_CONFIG_LOCATION=classpath:/config/base.yml,config.yml
# java -XX:+UnlockExperimentalVMOptions -XshowSettings:vm -Xmx256M -jar kafka-webview-ui-*.jar
VM settings:
    Max. Heap Size: 256.00M
    Ergonomics Machine Class: server
    Using VM: Java HotSpot(TM) 64-Bit Server VM


  _  __      __ _          __          __  _      __      ___
 | |/ /     / _| |         \ \        / / | |     \ \    / (_)
 | ' / __ _| |_| | ____ _   \ \  /\  / /__| |__    \ \  / / _  _____      __
 |  < / _` |  _| |/ / _` |   \ \/  \/ / _ \ '_ \    \ \/ / | |/ _ \ \ /\ / /
 | . \ (_| | | |   < (_| |    \  /\  /  __/ |_) |    \  /  | |  __/\ V  V /
 |_|\_\__,_|_| |_|\_\__,_|     \/  \/ \___|_.__/      \/   |_|\___| \_/\_/
....<snip>.......
2019-03-20 16:50:15.666  INFO 14443 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2019-03-20 16:50:15.669  INFO 14443 --- [           main] o.s.kafka.webview.ui.Application         : Started Application in 8.246 seconds (JVM running for 8.813)

from kafka-webview.

Depending on which version of Java you're running, and what JVM options you are passing, you could be hitting this kubernetes/JVM issues: https://very-serio.us/2017/12/05/running-jvms-in-kubernetes/

from kafka-webview.

I wrote some kubernetes yaml manifests to let kafka-webview running properly.
I set resources like following:

• requests: 800m cpu and 1G ram
• limits: 1 cpu and 2G ram
  It works like a charm!!

@Crim if you agree, I would like to contribute and prepare the project to run properly on kubernetes. I found a couple of possible improvements. Can we discuss about that?

from kafka-webview.

@bygui86 absolutely interested. What are your thoughts/suggestions? What I have here now for Docker is pretty much the bare minimum, and I don't feel personally tied to it in anyway :p

from kafka-webview.

Those all sound like extremely useful and worthwhile features/changes to make! Feel free to send over PRs. My preference would be for more smaller iterative PRs over a single massive PR that does everything all at once. Just makes it easier to review :)

Thanks!

from kafka-webview.

I totally agree. I will organise a series of branches and PRs. Something like:

1. add prometheus metrics and change actuator port to 9090 (or else if you prefer) per default
2. expose actuator/info, actuator/health and actuator/prometheus/metrics (or similar) without security
3. add kubernetes manifests
4. add helm chart
5. review Dockerfile

After all these I would like to introduce also something really fancy to run integration tests :) testcontainers
So maybe we can make dev-cluster easier.

from kafka-webview.

Sounds fantastic, I look forward to seeing your improvements :)

from kafka-webview.

I'm running kafka webview using docker and it consumes about 2GB of memory. I noticed the following rule in start.sh:

exec java -jar kafka-webview-ui-*.jar $HEAP_OPTS $LOG_OPTS

I tried adding -Xmx256m to the HEAP_OPTS env variable, but that didn't change the memory usage. But after seeing the comment from @Crim in this issue, I noticed the -Xmx256m option was before the -jar ..., so I changed the line in the start.sh script to:

exec java $HEAP_OPTS -jar kafka-webview-ui-*.jar $LOG_OPTS

After that change, it consumes a lot less memory and it fixed my problem.

from kafka-webview.