zilongtong / datakit Goto Github PK

DataKit is an open source, integrated data collection agent, which provides full platform (Linux/Windows/macOS) support and has comprehensive data collection capability, covering various scenarios such as host, container, middleware, tracing, logging and security inspection.

• Support collection of metrics, logging and tracing
• Fully support Kubernetes ecology
• Pipeline: Simple structured data extraction
• Supports third-party data import:
  • Telegraf
  • Prometheus
  • Statsd
  • Fluentd
  • Function
  • Tracing related(OpenTelemetry/DDTrace/Zipkin/Jaeger/Skywalking)

All DataKit changelog refers to here.

We can directly obtain the DataKit installation command from guance cloud. Most of the installation commands seems like that:

• Linux & Mac

DK_DATAWAY="https://openway.guance.com?token=<YOUR-TOKEN>" bash -c "$(curl -L https://static.guance.com/datakit/install.sh)"

• Windows

$env:DK_DATAWAY="https://openway.guance.com?token=<YOUR-TOKEN>";Set-ExecutionPolicy Bypass -scope Process -Force; Import-Module bitstransfer; start-bitstransfer -source https://static.guance.com/datakit/install.ps1 -destination .install.ps1; powershell .install.ps1;

• Kubernetes DaemonSet

For more documentations about DataKit installation, see here.

We also released the unstable DataKit, we can install via

• Linux & Mac

DK_DATAWAY="https://openway.guance.com?token=<YOUR-TOKEN>" bash -c "$(curl -L https://static.guance.com/datakit/community/install.sh)"

• Windows

$env:DK_DATAWAY="https://openway.guance.com?token=<YOUR-TOKEN>";Set-ExecutionPolicy Bypass -scope Process -Force; Import-Module bitstransfer; start-bitstransfer -source https://static.guance.com/datakit/community/install.ps1 -destination .install.ps1; powershell .install.ps1;

• Kubernetes DaemonSet

# We should use the unstable version yaml
wget https://static.guance.com/datakit/community/datakit.yaml

DataKit building relies on some external tools/libs, we must install them all before compile the source code.

We do not support build DataKit on Windows.

• Go-1.16.4+
• gcc-multilib: Used to build Oracle input(apt-get install gcc-multilib)
• tree: After building datakit, tree used to show all bianries(apt-get install tree)
• packr2: Used to package resources(mainly documents)
• goyacc: Used to build grammar for Pipleine(go get -u golang.org/x/tools/cmd/goyacc)
• Docker: Used to build DataKit image
• lint related:
  • gofumpt: Used to format go source code(go install mvdan.cc/gofumpt@latest)
  • golangci-lint 1.42.1
• eBPF related:
  • clang 10.0+
  • llvm 10.0+
  • apt install go-bindata
• Documentation exporting:
  • waque 1.13.1+

1. Clone

git clone https://github.com/DataFlux-cn/datakit.git

2. Building

cd datakit
make

If building ok, all binaries are generated under dist:

dist/
├── datakit-linux-amd64
│   ├── datakit             # DataKit main binary
│   └── externals      
│       ├── datakit-ebpf    # eBPF collector
│       ├── logfwd          # logfwd collector
│       └── oracle          # Oracle collector
└── local
    ├── installer-linux-amd64 # installer used fo Linux 
    └── version               # version descriptor

We can build all platforms(Linux/Mac/Windows) with following command:

make testing

We can use help command to see more usage of DataKit:

datakit help

# Or

./dist/datakit-linux-amd64/datakit help

Before contributing, check out some guideline of DataKit:

• Read architecure introduciton
• Read development guideline

For full documents of DataKit, see DataKit Doc Repository。