Giter Club home page Giter Club logo

aliyun-log-jaeger's Introduction

SLS Storage Plugin

This is the repository that contains object storage (Alibaba Could log service) plugin for Jaeger.

About

As a component of an observability/monitoring system, Jaeger is an essential source of data for development and operations students to locate and find problems and exceptions with the business system. As an SRE, we must ensure that the monitoring system lives longer than the system. Once the monitoring system is down before the business system, monitoring is entirely worthless. Monitoring is the last barrier for business exception analysis, and it is more sensitive to high availability and high performance than other systems.

The Alibaba Could log service(SLS) provides high performance, resilience, and freedom from operation and maintenance, allowing users to cope with surge traffic or inaccurate size assessment quickly, and the SLS service itself provides 99.9% availability and 11 out of 9 data reliability.

The Alibab Cloud log service ❤️ Jaeger

Image

Quick Start

You can follow the steps to use the plugin. We also provide an online demo to demonstrate the features of SLS Trace.

  1. Login Alibaba Cloud log service Console
  2. Create Project CrateProject
  3. Create Trace Instance CreateTraceInstance
  4. Modify Docker compose file ModifyConfigure
  5. Start Demo StartingDemo
  6. Having fun with Jaeger and SLS. 😀 HavingFunWithJaegerAndSLS

Build/Compile

In order to compile the plugin from source code you can use go build:

cd /path/to/jaeger-sls
go build

Start

Executing the following command to start jaeger with Tenon Plugin

export ACCESS_KEY_SECRET=""
export ACCESS_KEY_ID=""
export PROJECT=""
export ENDPOINT=""
export INSTANCE=""
GRPC_STORAGE_PLUGIN_BINARY="./jaeger-sls" SPAN_STORAGE_TYPE=grpc-plugin JAEGER_DISABLED=true GRPC_STORAGE_PLUGIN_LOG_LEVEL=DEBUG ./all-in-one

License

The SLS Storage gRPC Plugin for Jaeger is an MIT licensed open source project.

aliyun-log-jaeger's People

Contributors

badiib avatar black-adder avatar brucewu-fly avatar caniszczyk avatar dieterbe avatar douglas-reid avatar dstroot avatar ejwood79 avatar ekelleyv avatar hypnoce avatar ideepika avatar isaachier avatar javierprovecho avatar jnewmano avatar jpkrohling avatar jukylin avatar kxxoling avatar marckk avatar mh-park avatar neocn avatar objectiser avatar pavolloffay avatar qiansheng91 avatar rbtcollins avatar shabicheng avatar tiffon avatar vprithvi avatar wuman avatar xuxiaoahang2018 avatar yurishkuro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aliyun-log-jaeger's Issues

Agent 部署的作用 疑问?不部署agent, 依然写入sls trace 成功?

文档说明如下:
Agent 是一个监听在 UDP 端口上接收 span 数据的网络守护进程,它会将数据批量发送给 collector。它被设计成一个基础组件,部署到所有的宿主机上。Agent 将 client library 和 collector 解耦,为 client library 屏蔽了路由和发现 collector 的细节。

目前的client 使用 https://github.com/aliyun-sls/opentelemetry-go-provider-sls

问题1:
此client 查看源码 是通过 otlptrace.New 的方式,直接连接的是 sls 相关配置: 如: xxxx.cn-beijing.log.aliyuncs.com:xxx
并没有 和 agent 进行交互,

// 使用GRPC方式导出数据
traceSecureOption := otlptracegrpc.WithTLSCredentials(credentials.NewClientTLSFromCert(nil, ""))
if insecure {
	traceSecureOption = otlptracegrpc.WithInsecure()
}
traceExporter, err = otlptrace.New(context.Background(),
	otlptracegrpc.NewClient(otlptracegrpc.WithEndpoint(otlpEndpoint),
		traceSecureOption,
		otlptracegrpc.WithHeaders(headers),
		otlptracegrpc.WithCompressor(gzip.Name)))
if err != nil {
	return nil, nil, nil, err
}

如果agent 没运行,经测试,依然可以写入

那么 agent 解耦 client 和 collector 的作用 还有吗?
此处agent的作用又是什么呢?

请教一个问题

在java中怎么做数据埋点,一个服务可能要调用很多层的dubbo服务,这时候,怎么对每个服务做数据埋点呢?

jaegertracing/jaeger-agent镜像版本导致命令失效

docker安装jaeger-agent时,必须指定版本为 1.6 以前的,因为后续版本中 "/go/bin/agent-linux" 默认执行。
如果不加版本,将使用 lastest 版本,将会报错 “no command /go/bin/agent-linux”。
文件 aliyunlog-jaeger-docker-compose.yml 中也会出现同样的问题。

default

HTTP Error: Failed to convert log to span: strconv.ParseUint: parsing "null": invalid syntax

jaeger-query 报错

`

Status 500
Status text Internal Server Error
URL /api/traces
Query end=1600060399083000&limit=20&lookback=1h&maxDuration&minDuration&service=istio-ingressgateway&start=1600056799083000
Response body { "data": null, "total": 0, "limit": 0, "offset": 0, "errors": [ { "code": 500, "msg": "Failed to convert log to span: strconv.ParseUint: parsing "null": invalid syntax" } ] }

`

unknow command "/go/bin/agent-linux" for "jaeger-agent"

jaeger-alilog_jaeger-agent_1 | 2018-09-06T03:39:44.831447210Z unknown command "/go/bin/agent-linux" for "jaeger-agent"
jaeger-alilog_jaeger-agent_1 | 2018-09-06T03:39:44.833135209Z Error: unknown command "/go/bin/agent-linux" for "jaeger-agent"

用的aliyun docker swarm

No service dependencies found.

使用阿里日志,Dependencies页面提示No service dependencies found.
根据官方提示需要jaeger-analytics-flink
有没有基于阿里日志的jaeger-analytics-flink使用方法或者docker?

使用k8s 安装jaeger-agent 无法传送数据

1、k8s 内部服务使用jaeger-agent tcpdump抓包确定数据传送至agent,但是数据没有落入jaeger-collector 。
2、单独使用github 提供的jaeger-agent docker 启动 没有问题。
3、更换多个版本无效
启动命令:/go/bin/agent-linux --collector.host-port=<JAEGER_COLLECTOR_HOST>:14267

jaeger-agent日志(v 1.6的截图):
image

deployment 配置:
spec:
containers:
- name: jaeger-agent
image: jaegertracing/jaeger-agent:1.6
#command:
#- "/go/bin/agent-linux"
ports:
- containerPort: 5775
protocol: UDP
- containerPort: 6831
protocol: UDP
- containerPort: 6832
protocol: UDP
- containerPort: 5778
protocol: TCP
args: ["--collector.host-port=jaeger-collector:14267"]

Errors you may encounter when upgrading the library

(The purpose of this report is to alert aliyun/aliyun-log-jaeger to the possible problems when aliyun/aliyun-log-jaeger try to upgrade the following dependencies)

An error will happen when upgrading library _prometheus/client_golang:

github.com/prometheus/client_golang

-Latest Version: V1.7.1 (Latest commit fe7bd95 6 days ago )
Master branch
-Where did you use it:
https://github.com/aliyun/aliyun-log-jaeger/search?l=Go&q=prometheus%2Fclient_golang
-Detail:

github.com/prometheus/client_golang/go.mod

module github.com/prometheus/client_golang
go 1.11
require (
	github.com/beorn7/perks v1.0.1
	github.com/cespare/xxhash/v2 v2.1.1	
	…
)

github.com/prometheus/client_golang/prometheus/registry.go

package prometheus
import (
	"github.com/cespare/xxhash/v2"
	"github.com/golang/protobuf/proto"
	…
)

This problem was introduced since prometheus/client_golang v1.2.0(Latest commit 9a2ab94 on 16 Oct 2019). If you try to upgrade prometheus/client_golang to version v1.2.0 and above, you will get an error--- no package exists at "github.com/cespare/xxhash/v2"

I investigated the libraries (prometheus/client_golang >= v1.2.0) release information and found the root cause of this issue is that----

  1. These dependencies all added Go modules in the recent versions.

  2. They all comply with the specification of "Releasing Modules for v2 or higher" available in the Modules documentation. Quoting the specification:

A package that has migrated to Go Modules must include the major version in the import path to reference any v2+ modules. For example, Repo github.com/my/module migrated to Modules on version v3.x.y. Then this repo should declare its module path with MAJOR version suffix "/v3" (e.g., module github.com/my/module/v3), and its downstream project should use "github.com/my/module/v3/mypkg" to import this repo’s package.

  1. This "github.com/my/module/v3/mypkg" is not the physical path. So earlier versions of Go (including those that don't have minimal module awareness) plus all tooling (like dep, glide, govendor, etc) don't have minimal module awareness as of now and therefore don't handle import paths correctly See golang/dep#1962, golang/dep#2139.

Note: creating a new branch is not required. If instead you have been previously releasing on master and would prefer to tag v3.0.0 on master, that is a viable option. (However, be aware that introducing an incompatible API change in master can cause issues for non-modules users who issue a go get -u given the go tool is not aware of semver prior to Go 1.11 or when module mode is not enabled in Go 1.11+).
Pre-existing dependency management solutions such as dep currently can have problems consuming a v2+ module created in this way. See for example dep#1962.
https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Solution

1. Migrate to Go Modules.

Go Modules is the general trend of ecosystem, if you want a better upgrade package experience, migrating to Go Modules is a good choice.

Migrate to modules will be accompanied by the introduction of virtual paths(It was discussed above).

This "github.com/my/module/v3/mypkg" is not the physical path. So Go versions older than 1.9.7 and 1.10.3 plus all third-party dependency management tools (like dep, glide, govendor, etc) don't have minimal module awareness as of now and therefore don't handle import paths correctly.

Then the downstream projects might be negatively affected in their building if they are module-unaware (Go versions older than 1.9.7 and 1.10.3; Or use third-party dependency management tools, such as: Dep, glide, govendor…).

2. Maintaining v2+ libraries that use Go Modules in Vendor directories.

If aliyun/aliyun-log-jaeger want to keep using the dependency manage tools (like dep, glide, govendor, etc), and still want to upgrade the dependencies, can choose this fix strategy.
Manually download the dependencies into the vendor directory and do compatibility dispose(materialize the virtual path or delete the virtual part of the path). Avoid fetching the dependencies by virtual import paths. This may add some maintenance overhead compared to using modules.

As the import paths have different meanings between the projects adopting module repos and the non-module repos, materialize the virtual path is a better way to solve the issue, while ensuring compatibility with downstream module users. A textbook example provided by repo github.com/moby/moby is here:
https://github.com/moby/moby/blob/master/VENDORING.md
https://github.com/moby/moby/blob/master/vendor.conf
In the vendor directory, github.com/moby/moby adds the /vN subdirectory in the corresponding dependencies.
This will help more downstream module users to work well with your package.

3. Request upstream to do compatibility processing.

The prometheus/client_golang have 1039 module-unaware users in github, such as: AndreaGreco/mqtt_sensor_exporter, seekplum/plum_exporter, arl/monitoring…
https://github.com/search?q=prometheus%2Fclient_golang+filename%3Avendor.conf+filename%3Avendor.json+filename%3Aglide.toml+filename%3AGodep.toml+filename%3AGodep.json

Summary

You can make a choice when you meet this DM issues by balancing your own development schedules/mode against the affects on the downstream projects.

For this issue, Solution 1 can maximize your benefits and with minimal impacts to your downstream projects the ecosystem.

References

Do you plan to upgrade the libraries in near future?
Hope this issue report can help you ^_^
Thank you very much for your attention.

Best regards,
Kate

a simple demo for dependency

For our team needs, i write a simple dependency calculator.This is for small amount of span data, because we just started using jaeger.

I work on Jaeger UI; we'd love to hear any feedback or suggestions

Hello,

TLDR; I work on Jaeger UI, we'd love to hear any feedback or suggestions, or even to just chat about tracing!

I'm a maintainer on the Jaeger project, focused mostly on Jaeger UI.

I wanted to reach out to say we're thrilled that you've embraced Jaeger!

We're constantly striving to improve Jaeger, and we see the extensibility of the platform and UI as central to that.

We're very keen to learn what your experience with Jaeger has been like.

  • What pain-points did you encounter?
  • What's working well?
  • Any suggestions?

Any feedback would be fantastic 👍

We can also share our plans for new features in development, such as path-based dependency graphs, to gather your feedback while they're in development.

Please let me know what you think.

Best.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.