The default processor config is set to batch:
processors:
batch:
spanmetrics:
metrics_exporter: prometheus
service:
pipelines:
traces:
receivers: [jaeger]
processors: [spanmetrics, batch]
To enable tail_sampling
, make sure your otel-collector-config.yml has the following configuration:
tail_sampling:
decision_wait: 10s
num_traces: 100
expected_new_traces_per_sec: 10
policies:
[
{
name: latency-policy,
type: latency,
latency: { threshold_ms: 5000 }
},
{
name: status-code-policy,
type: status_code,
status_code: { status_codes: [ ERROR, OK ] }
},
{
name: string-policy,
type: string_attribute,
string_attribute: { key: alarm, values: [ unexpected ] }
}
]
spanmetrics:
metrics_exporter: prometheus
service:
pipelines:
traces:
receivers: [jaeger]
processors: [spanmetrics, tail_sampling]
Also, use quarkus.opentelemetry.tracer.sampler=on
in Quarkus configuration.
curl -X 'POST' 'http://activity:80/api/send-request' -H 'trace-debug-id:local' -H 'accept: /' -H 'Content-Type: application/json' -d '{"uppercase": true,"reverse": true}'
(sum by (job, uri, status)(rate(http_server_requests_seconds_sum{job="hobby", method="POST", uri="/api/send-request"}[5m])))
or
(sum by (job, uri, status)(rate(http_server_requests_seconds_sum{job="activity", method="GET", uri="/activity"}[5m])))
If you would like to use the extra attribute for debugging, you can try the following command:
curl -H 'trace-debug-id:local' http://localhost:8080/activity/
You can simulate some load using hey:
hey -n 20 -c 10 -H 'trace-debug-id:local' http://localhost:8080/activity
sum(delta(cache_gets_total{result="hit",cache="activity"}[1m]))/sum(delta(cache_gets_total{cache="activity"}[1m]))