Giter Club home page Giter Club logo

vaero's Introduction

Vaero Logo

Go Contributions

View the documentation and quickstart guide online for usage information.

What is Vaero?

Vaero is a programmable high-performance data pipeline for log data. Collect, transform, and route your log data from any log source to any destination. Vaero helps you save on log storage costs, send data to different systems of analysis, normalize data to different formats, enrich data with more information, and control logs from all your vendors in one place.

Principles

  • Easy to program - Code your log pipelines in Python. No need to learn a config file format or new language.
  • Fast - Log pipelines are executed entirely in Go, with performance in mind.
  • Building blocks - Built-in transforms and source and destination integrations, so you don't have to reinvent the wheel.

Use Cases

  • Reduce log storage costs by filtering and routing logs
  • Centralize log management instead of configuring multiple sources
  • Adjust one system when you change vendors, not all your sources
  • Normalize log data between systems
  • Mask PII before storage
  • Improve overall log ETL performance

Code Log Pipelines in Python

from vaero.stream import Vaero

result = Vaero().source("http_server", port = 8080, endpoint = "/log") \
        .rename("hostname", "host") \
        .add("newfield", "Hello, world!") \
        .sink("s3")

Vaero.start()

Vaero's Python syntax is modern and easy to use. Log pipelines are executed with high-performance Go code.

Getting Started

Pipelines

Deployment

Adminstration

Reference

Configuration


Built and supported by Vaero

vaero's People

Contributors

bryantvaero avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.