Giter Club home page Giter Club logo

datadog4s's Introduction

Datadog Scala Metrics Build Download

Toolkit for monitoring applications written in functional Scala using Datadog and Cats.

Goal of this project is to make great monitoring as easy as possible.

In addition to basic monitoring utilities, we also provide a bunch of plug-and-play modules that do monitoring for you. Currently, those are:

  • JVM monitoring
  • Http4s monitoring

Installation

For installation instructions, please see install guide

User Guide

For user guide, please see our guide

datadog4s's People

Contributors

augi avatar gstro avatar hanny24 avatar mergify[bot] avatar sbly avatar scala-steward avatar tomasherman avatar velppa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

datadog4s's Issues

JvmReporter cannot access class sun.management.ManagementFactoryHelper

Describe the bug

Exception in thread "main" java.lang.IllegalAccessError: class com.avast.datadog4s.extension.jvm.JvmReporter (in unnamed module @0x221ca495) cannot access class sun.management.ManagementFactoryHelper (in module java.management) because module java.management does not export sun.management to unnamed module @0x221ca495
        at com.avast.datadog4s.extension.jvm.JvmReporter.<init>(JvmReporter.scala:67)
        at com.avast.datadog4s.extension.jvm.JvmMonitoring$.configured(JvmMonitoring.scala:26)
        at com.avast.datadog4s.extension.jvm.JvmMonitoring$.default(JvmMonitoring.scala:19)

To Reproduce
Run with Zulu JDK 17 (https://cdn.azul.com/zulu/bin/zulu17.32.13-ca-jdk17.0.2-linux_x64.tar.gz) & datadog 0.31.2

Support for sending metrics directly to StatsD

Is your feature request related to a problem? Please describe.
I would like to be able to send metrics directly to the datadog API as I am able to do with micrometer to simplify my application deployment.

Describe the solution you'd like
An alternative to the StatsD implementation that talks directly to the DD API

Describe alternatives you've considered
Using micrometer directly, but this involved wrapping it

Timers are registread as a Counters in statsD

Describe the bug
The timer is stored as a counter entity

To Reproduce
Steps to reproduce the behavior:

          val interceptResponseTimeExecution: IO[F[Tag]] = factoryResource.use { factory =>
            val timer = factory.timer.distribution("latency", timeUnit = TimeUnit.MILLISECONDS)
            val measureTime: IO[Response] = timer.time(
              IO.pure(generateResponse(req, matches)), Tag.of("CCCC", "BBBB")
            )
            for {
              executionTime <- measureTime           // intercept required time execution
            } yield {
              val aTag  = serializeResponse(executionTime) + "\n"
              aTag.pure[F]                      // outside effect change
            }

Expected behavior
Should be stored as a timer entry

Additional context
It is stored as a counter entity

 echo "counters" | nc localhost 8126
{
  'statsd.bad_lines_seen': 0,
  'statsd.packets_received': 0,
  'statsd.metrics_received': 0,
  'response.200': 0,
  'response.auth.200': 0,
  'my-app-name.hits': 0,
  'my-app-name.latency;success=true;CCCC=BBBB': 0
}
END

The same for Histogram

Compatibility with Windows

running datadog4s on windows returns this cast exception:

17:56:47.310 [zio-default-async-1-40688307] ERROR com.avast.skyring.common.zio.ZIOFailureReporter$ - Uncaught exception on thread zio-default-async-1-40688307
zio.FiberFailure: Fiber failed.
A checked error was not handled.
java.lang.ClassCastException: class com.sun.management.internal.OperatingSystemImpl cannot be cast to class com.sun.management.UnixOperatingSystemMXBean (com.sun.management.internal.OperatingSystemImpl and com.sun.management.UnixOperatingSystemMXBean are in module jdk.management of loader 'bootstrap')
	at com.avast.datadog4s.extension.jvm.JvmReporter.<init>(JvmReporter.scala:36)
	at com.avast.datadog4s.extension.jvm.JvmMonitoring$.$anonfun$configured$5(JvmMonitoring.scala:29)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at zio.internal.FiberContext.evaluateNow(FiberContext.scala:295)
	at zio.internal.FiberContext.$anonfun$fork$1(FiberContext.scala:595)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Support for Scala 2.13

It would be good to have a version for Scala 2.13.

I prepared a branch to support this version: master...mgibowski:scala-2.13

Here are necessary changes:

  • replace usage of scala.collection.JavaConverters with scala.jdk.CollectionConverters,
  • update dependency on http4s, 0.20.15 -> 0.21.0-M6.

My branch is not ready for a PR, the cross compilation is not fully set up. To be able to publish artifacts locally, I first need to publish them with the setting scalaVersion := scala212 and then again with the setting scalaVersion := scala213 and each time they get only published with the current scala version. I don't have much time now to fix it, but maybe the above info is useful for you.

UDP Support

Is your feature request related to a problem? Please describe.
I use dogstatsd over UDP and I'm unsure how to use UDP with this library.

Describe the solution you'd like
Ability to specify the protocol (so UDP/TCP should be used)

Describe alternatives you've considered
Unsure of any good alternatives.

Additional context
N/A

Add support for Java 16

It seems that in order to compile jvm module on JDK 16, we need to add --add-exports java.management/sun.management=ALL-UNNAMED to compilation step.

It's likely that users of datadog4s will need to add this aswell when trying to run the on JDK 16 so we should mention this in documentation / readme

Runtime error + app hangs when upgrading to Java 17

Describe the bug
Trying to migrate our app using from Java 11 to Java 17 I get the below runtime error and the app hangs. This prevents us from upgrading to Java 17.

Here's the runtime error we're seeing:

Exception in thread "io-compute-blocker-8" java.lang.IllegalAccessError: class com.avast.datadog4s.extension.jvm.JvmReporter (in unnamed module @0x663c9e7a) cannot access class sun.management.ManagementFactoryHelper (in module java.management) because module java.management does not export sun.management to unnamed module @0x663c9e7a
	at com.avast.datadog4s.extension.jvm.JvmReporter.<init>(JvmReporter.scala:67)
	at com.avast.datadog4s.extension.jvm.JvmMonitoring$.configured(JvmMonitoring.scala:26)
	at com.avast.datadog4s.extension.jvm.JvmMonitoring$.default(JvmMonitoring.scala:19)
	at com.wolt.courierpayments.modules.Monitoring$.$anonfun$createMetricFactory$1(Monitoring.scala:26)
	at cats.effect.kernel.Resource.loop$1(Resource.scala:189)
	at cats.effect.kernel.Resource.continue$1(Resource.scala:166)
	at cats.effect.kernel.Resource.$anonfun$fold$1(Resource.scala:177)
	at cats.effect.IO$.$anonfun$bracketFull$3(IO.scala:1442)
	at cats.effect.IOFiber.runLoop(IOFiber.scala:403)
	at cats.effect.IOFiber.execR(IOFiber.scala:1319)
	at cats.effect.IOFiber.run(IOFiber.scala:118)
	at cats.effect.unsafe.WorkerThread.run(WorkerThread.scala:555)

To Reproduce
Invoke JvmMonitoring.default[F](metricFactory) on Java 17.

Expected behavior
App should not hang and no exception should get thrown.

Additional context

Update documentation

Documentation is now woefully out of date and should be updated - some areas:

  • jvm reporting (link leads to non-existing file)

Interest in using distribution for requestLatency in Http4s MetricsOps?

Hello,

I'm using this library to export HTTP4s metrics (and it works beautifully, thank you!).

The only downside is that requestLatency is reported as a timer:
https://github.com/avast/datadog4s/blob/master/code/http4s/src/main/scala/com/avast/datadog4s/extension/http4s/impl/DefaultMetricsOps.scala#L57

This has the effect of:

  1. Causing the statsd sync to aggregate these metrics at whatever percentile is configured, which makes them less useful unless you break your queries up by statsd reporter (because aggregating percentiles is statically invalid)
  2. In my case, leaving me at the mercy of a rather unhelpful 95th percentile statsd sync.

Obviously it's not that difficult to create our own MetricsOps, or wrap this one, to provide a distribution, but I wanted to check and see if there would be any interest in incorporating those changes upstream.

I see multiple forms his could take:

  1. Replace the current timer with a distribution. This is backwards-incompatible for existing dashboards, so probably not desirable, though it keeps things simple.
  2. Report the latency as both a timer, and a distribution. Wasteful, but backwards-compatible and still pretty simple
  3. Refactor the metrics ops to make the type of metric used for latency selectable by the user, leaving the DefaultMetricsOps implementation using a timer and having a Distribution-backed alternative
  4. Keep things as-is, we implement our own ops.

Just wanted to gauge interest before I went down any particular path.

Thanks!

API-Doc generation doesn't work

As described in #436, API docs generation does not work. This is likely caused by sbt/sbt#6652. Once the issue is closed, we should make sure that docs are generated and published correctly. So far i have not been able to find a workaround

New release with http4s 0.23.0

Similar to #394, it would be lovely if we could get a new release tagged now that #410 has merged.

org.http4s:http4s-core_2.13:0.23.0 (early-semver) is selected over 0.23.0-RC1
  +- org.http4s:http4s-jawn_2.13:0.23.0                 (depends on 0.23.0)
  +- org.http4s:http4s-blaze-core_2.13:0.23.0           (depends on 0.23.0)
  +- org.http4s:http4s-circe_2.13:0.23.0                (depends on 0.23.0)
  +- org.http4s:http4s-client_2.13:0.23.0               (depends on 0.23.0)
  +- com.avast.cloud:datadog4s-http4s_2.13:0.30.2       (depends on 0.23.0-RC1)

New Release with http4s 0.23.0-RC1

Is your feature request related to a problem? Please describe.
Latest datadog4s does not work with http4s 0.23.0-RC1

datadog4s 0.30.0 depends on http4s 0.23.0-RC1

cs resolve com.avast.cloud:datadog4s_2.13:0.30.0 | grep http4s

com.avast.cloud:datadog4s-http4s_2.13:0.30.0:default
org.http4s:http4s-core_2.13:0.23.0-M1:default

However, support for http4s 0.23.0-RC1 has been merged: #388

sbt flags 0.23.0-M1 and 0.23.0-RC1 as binary incompatible:

found version conflict(s) in library dependencies; some are suspected to be binary incompatible:

* org.http4s:http4s-core_2.13:0.23.0-RC1 (early-semver) is selected over 0.23.0-M1
    +- org.http4s:http4s-jawn_2.13:0.23.0-RC1             (depends on 0.23.0-RC1)
    +- org.http4s:http4s-blaze-core_2.13:0.23.0-RC1       (depends on 0.23.0-RC1)
    +- org.http4s:http4s-circe_2.13:0.23.0-RC1            (depends on 0.23.0-RC1)
    +- org.http4s:http4s-client_2.13:0.23.0-RC1           (depends on 0.23.0-RC1)
    +- com.avast.cloud:datadog4s-http4s_2.13:0.30.0       (depends on 0.23.0-M1)

Describe the solution you'd like
Cut a new 0.30.x release of datadog4s with http4s 0.23.0-RC1 support (already merged in)

Thanks for all your work on this library, it's been lovely to use ❤️

metrics not being reported

So I have a local netcat running on port 8130 via

$ nc -ul 8130

I can mimic a succesful metric publish via

echo "foo:1|c" | nc -u -w0 127.0.0.1 8130

with that I see the packet echoed out in my netcat listener.

So now I have a "test" that attempts to write to the same address using datadog4s primitives ...

        val port = 8130
        val statsDServer = InetSocketAddress.createUnresolved("localhost", port)
        val config = StatsDMetricFactoryConfig(Some("my-service"), statsDServer)
        val factoryResource: Resource[IO, MetricFactory[IO]] = StatsDMetricFactory.make(config)

        factoryResource.use { factory =>
          val count: Count[IO] = factory.count("foo")
          for {
            _ <- count.inc()
          } yield {
            ()
          }
        }.unsafeToFuture

Yet I never see metrics appear in my netcat console.

So I must be missing something here, just cant see what that is.

dependencies:

  "com.avast.cloud" %% "datadog4s-api" % "0.14.0",
  "com.avast.cloud" %% "datadog4s-statsd" % "0.14.0",
  "cats" %% "effect" % "2.5.1"	

`Timer[F]` to allow injecting tags from the response

Is your feature request related to a problem? Please describe.

I am currently implementing a timer Metric to monitor a http client and the calls that it makes. I want an easy way of tagging http Responses. I like the def time[A](value: F[A], tags: Tag*): F[A] helper method but it literally just ignores the results.

Describe the solution you'd like

I would like a new method such as this def time[A](value: F[A], f: A => Seq[Tag], tags: Tag*): F[A]

Describe alternatives you've considered

Can impl my own version of timer, but I'd rather not! :)

Additional context

N/A

Error in dependency names

Describe the bug
Adding "com.avast.cloud" %% "datadog4s" % "latestVersion" OR "com.avast.cloud" %% "datadog4s-api" % "latestVersion"
throws the error
sbt.librarymanagement.ResolveException: unresolved dependency: com.avast.cloud#datadog4s;0.3.1: not found

To Reproduce
Add "com.avast.cloud" %% "datadog4s" % "0.3.1" in the build.sbt

Expected behavior
Should ideally import the required packages

Additional context
scalaVersion := "2.12.8"

Currently I have to add the version number with datadog4s as well liek so "com.avast.cloud" % "datadog4s-api_2.12" % "0.3.1" for expected behavior

Also, I have to add each package manually like datadog4s-api_2.12 or datadog4s-statsd_2.12.
Simply adding datadog4s_2.12 does not work.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.