Scrapes Hadoop Metrics and exposes them in Prometheus format.

Supports both simple and SPNEGO authentication for the Hadoop endpoints.

flowchart LR
  id1[Prometheus] --no auth--> id2[Hadoop-HTTP-JMX-Exporter]
  id2[Hadoop-HTTP-JMX-Exporter] --no auth/SPNEGO--> id3[Hadoop Service]
  id3[Hadoop Service]

Configuration can be provided ether via environment variable:

JMX_URLS="http://10.0.0.1:8088/jmx,http://10.0.0.1:8042/jmx"
KERBEROS_PRINCIPAL=HTTP/localhost
KERBEROS_REALM=SOME_REALM
KERBEROS_KEYTAB_PATH=/path/to/keytab/with/HTTP/principal
KERBEROS_CONFIG_PATH=/etc/krb5.conf
HTTP_CLIENT_TIMEOUT=10s # default
LISTEN_ADDRESS=0.0.0.0 # default
LISTEN_PORT=9100 # default

or a config YAML file passed as the only argument to the CLI:

jmx_urls:
    - http://10.0.0.1:8088/jmx
    - http://10.0.0.1:8042/jmx
kerberos_principal: HTTP/localhost
kerberos_realm:  SOME_REALM
kerberos_keytab_path: /path/to/keytab/with/HTTP/principal
kerberos_config_path: /etc/krb5.conf
http_client_timeout: 10s # default
listen_address: 0.0.0.0 # default 
listen_port: 9100 # default 

Following metrics are currently exposed:

• JvmMetrics
  • MemNonHeapUsedM
  • MemNonHeapCommittedM
  • MemNonHeapMaxM
  • MemHeapUsedM
  • MemHeapCommittedM
  • MemHeapMaxM
  • MemMaxM
  • GcCount
  • GcTimeMillis
  • GcTotalExtraSleepTime
• datanode
  • BytesWritten
  • BytesRead
  • HeartbeatsAvgTime
• ClusterMetrics
  • NumLostNMs
  • NumUnhealthyNMs
  • NumRebootedNMs
• NodeManagerMetrics
  • ContainersLaunched
  • ContainersCompleted
  • ContainersFailed
  • ContainersKilled
  • ContainersIniting
  • ContainersRunning
  • AllocatedContainers
  • AllocatedGB
  • AvailableGB
  • AllocatedVCores
  • AvailableVCores
  • ContainerLaunchDuration
  • NodeUsedMemGB
  • NodeUsedVMemGB
  • NodeCpuUtilization
  • ContainerUsedMemGB
  • ContainerUsedVMemGB

Releases are acommpanied by a Docker image available here.