Documentation issue: spark, heuristics and hdfs readers about garmadon HOT 2 CLOSED

Hi @kockockockoc ,

Only garmadon-agent, garmadon-forwarder and garmadon-readers-* are usefull to deploy garmadon
Other jars are dependencies to agent/forwarder/readers jars to share protobuf schema, some mechanism to trace metrics...

garmadon-frameworks-spark is loaded by the garmadon-agent to retreive metrics from spark application using Spark Listener mechanism.
The listener is automatically attached to spark driver if you use StandaloneModule (-javaagent:path-to-garmadon-agent-jar=com.criteo.hadoop.garmadon.agent.modules.StandaloneModule) or ContainerModule (-javaagent:path-to-garmadon-agent-jar=com.criteo.hadoop.garmadon.agent.modules.ContainerModule)

executorService.submit(() -> {
    SparkListenerTracer.setup(ContainerHeader.getInstance().getHeader(),
            (timestamp, header, event) -> eventProcessor.offer(timestamp, header, event));
});

Properties props = System.getProperties();
        props.setProperty("spark.extraListeners", "com.criteo.hadoop.garmadon.spark.listener.GarmadonSparkListener");

So you don't have any specific things to do, only need to bind the agent to all spark JVMs (client and containers).
Will see to improve the documentation around garmadon and spark

For readers:

• garmadon-readers-hdfs, will need to add some documentation about it, @gray-matter ?
• garmadon-readers-heuristics, it is still bind to some specificities from criteo so not really available for now.

from garmadon.

Thank you for explaining. I was a bit confused by the fact that there were separate jars after "mvn clean package".

from garmadon.