Vendor this in internal, as it's used by the already-vendored kr/pretty.

Implement tracing wrappers for github.com/gocql/gocql

Go 1.9 added profile (pprof) labels: https://golang.org/doc/go1.9#pprof-labels. We should add labels automatically in our instrumentation modules, including at least the transaction or span name and type.

We currently never capture stack frames for spans, because it's too expensive. We should follow other agents (at least the Java agent does it), and provide a configurable span duration threshold which, when reached, will cause the span's stack frames to be captured.

We should support ELASTIC_APM_ACTIVE, and add pair of methods to Tracer to control and report whether tracing is active. Modules should check this and avoid any overhead when the tracer is inactive.

vgo is a new proposed package versioning mechanism for Go. It's going to be included in Go 1.11 as an opt-in feature. We should look at including modules to support its use.

Rename from "contrib" to "mod" (short for "module", a la apache), and update middleware constructors to accept options rather than positional arguments where those parameters are optional (e.g. provide WithTracer rather than taking a *Tracer where nil means to use the default.)

Implement a tracing command wrapper for https://github.com/gomodule/redigo.

Add Go agent tests to https://github.com/elastic/apm-integration-testing

In some circumstances it might be useful to override the user-agent header, e.g. elastic/apm-server#816 (review)

See https://github.com/elastic/apm-dev/pull/221

Improve URL reconstruction in apmhttp, using the various standard and non-standard headers [0]. This could be pulled out as a separate package for others to reuse like [1], though we may want to vendor/internalise to keep things simple.

[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded
[1] https://github.com/watson/original-url

You can now enable or disable the tracer with the environment variable ELASTIC_APM_ACTIVE. We should consider supporting dynamically toggling on/off at runtime.

Add support to module/apmhttp for capturing outgoing HTTP requests as spans.

Feedback from @karmi: we docs could do with more advice on how request context ought to be propagated in order to trace spans within an HTTP transaction.

I'm currently removing the ability for specifying the framework in transaction payloads, because Go's web frameworks tend to be less exclusive of each other. i.e. you could have two web frameworks in the same application. When we can attach the framework info to the transaction, then we'll reinstate support.

This requires changes to the API and API server.

The APM server will be introducing a new API for accepting application metrics. Update the agent code to support the API, along with bridges for popular metrics libraries (Prometheus, go-metrics). Also provide builtin metrics, e.g. heap usage, CPU usage, GC timings.

PR: #94

The other APM agents call it "end", rather than "done". We will follow suit.

Also, we'll drop the time.Duration parameter from those methods, since in most cases developers will want to have Done/End calculate the duration. Instead, expose a Duration field on the types.

Implement a tracing wrapper for the Client interface of https://github.com/Shopify/sarama.

We should redact header/cookie/form values with keys matching some patterns (e.g. "secret", "password", etc.), or values matching some patterns (e.g. credit card or social security numbers). The patterns should also be extensible via environment variables.

As per elastic/apm-agent-nodejs#287

• data source name (DSN) parsing
• convenience package for registering the driver with apmsql

Implement lightweight OpenTracing API compatibility bridge.

Implement tracing for https://github.com/streadway/amqp.

This will probably have to be a collection of amqp.Channel method wrappers that:

• add context.Context
• create spans using said context
• add headers (when we support distributed tracing)
• call through to the underlying amqp.Channel

Provide configuration to enable capturing request bodies.

We're not currently wrapping the sql.Tx returned from BeginTx. By wrapping the returned Tx, we can observe how much time is spent beginning and committing/rolling back database transactions.

Please add support for /labstack/echo lite web framework to contrib modules.

In the intake API it's called start, but that's an offset from the transaction's timestamp. We should call the field Timestamp for consistency with the Transaction type, to tidy up the API.

We should introduce a changelog, incorporated into the elastic.co docs

Work is underway to add support for distributed tracing to the API/server: elastic/apm-server#913 / elastic/apm-server#968. We will need to make changes to the Go agent to support this too.

• net/http
• https://github.com/gin-gonic/gin
• https://github.com/gorilla/mux
• database/sql

Later:

• https://github.com/garyburd/redigo (issue: does not use context, but see gomodule/redigo#310)
• https://github.com/olivere/elastic
• https://github.com/grpc/grpc-go
• https://beego.me
• https://gobuffalo.io

All of the other agents have a default value of http://localhost:8200 for ELASTIC_APM_SERVER_URL, to simplify development/getting started. The Go agent currently has no default, and disables the transport if no URL is set. We should consider changing to be consistent.

Go 1.11 introduces the concept of "regions" and "tasks" to the runtime execution tracer. See https://medium.com/observability/debugging-latency-in-go-1-11-9f97a7910d68. We should automatically create tasks along with each transaction and span, so users can correlate execution trace data with transactions.

Service names must match the regular expression ^[a-zA-Z0-9 _-]+$

Implement tracing middleware for https://beego.me

We should report the entire time, including the middleware. That's the time that the user cares about.

We might also include a transaction mark that identifies the point at which the underlying handler completed, but it's unlikely to be significant overhead generally, so probably not that useful.

We should include process (and host?) CPU usage as part of the builtin metrics we report to APM server. Probably use elastic/gosigar.

Implement tracing middleware for https://gobuffalo.io

Provide a configurable pattern to ignore HTTP transactions with matching URLs.

If URLs in any transaction are longer than 1024 bytes, the APM server will reject the entire payload. We need to make sure that fields with limited lengths are trimmed.

We currently support setting tags on transactions. APM Server 6.5 (hopefully) will add support for span tags: elastic/apm-server#1156. We need to add this to the Go agent.

Add a Router type to apmhttprouter, along with a New method. The Router type will embed httprouter.Router but will shadow the route-adding methods, wrapping handlers on the way in.

The elasticapm.Transaction, Span, and Error types currently expose the underlying model types of the same name. This will pose a problem if/when the model changes. Instead of exposing those types directly, we should provide a stable API for adding context, setting error details, etc.

• elasticapm.Error
• elasticapm.Transaction
• elasticapm.Span
• introduce elasticapm.Context type/methods
• introduce stacktrace.Frame

elasticapm.Tracer.Service For now we'll continue to expose model.Service, as it's relatively simple and unlikely to change significantly, and we don't anticipate that modifying it will be a common thing anyway.

Assuming we're going to support OpenTracing, we'll need to be able to change the transaction or span's name after it has been started. This is necessary because OpenTracing's Span API includes a method for setting/changing the operation name.

Assuming we do that, there's a follow-on issue: we currently filter on transaction name (ELASTIC_APM_TRANSACTION_IGNORE_NAMES) when starting a transaction. Transactions with a matching name will be ignored. We can either not support that configuration for updated operation names, or perform an additional check when the transaction is ended.

Other agents are parsing SQL statements to format SQL spans like "SELECT FROM table". The Go agent currently just uses the first token (e.g. "SELECT"). We should follow suit.

Official page needs to be updated: https://www.elastic.co/solutions/apm

The APM server supports deflate and gzip compression. The Go agent should transparently gzip payloads that go over a given uncompressed size.

Fuzz-test the API to give some confidence that we prevent the sending of invalid payloads to the APM server.