Giter Club home page Giter Club logo

fluent-plugin-gcloud-pubsub's Introduction

fluent-plugin-gcloud-pubsub

Build Status Gem Version

Overview

Cloud Pub/Sub Input/Output plugin for Fluentd with gcloud gem

  • Publish BufferedOutput chunk into Cloud Pub/Sub with batch publishing
  • Pull messages from Cloud Pub/Sub

Preparation

  • Create a project on Google Developer Console
  • Add a topic of Cloud Pub/Sub to the project
  • Add a pull style subscription to the topic
  • Download your credential (json)

Publish messages

Use out_gcloud_pubsub.

Configuration

publish dummy json data like {"message": "dummy", "value": 0}\n{"message": "dummy", "value": 1}\n ....

<source>
  type dummy
  tag example.publish
  rate 100 
  auto_increment_key value
</source>

<match example.publish>
  type gcloud_pubsub
  project <YOUR PROJECT>
  topic <YOUR TOPIC>
  key <YOUR KEY>
  flush_interval 10
</match>

Pull messages

Use in_gcloud_pubsub.

Configuration

Pull json data from Cloud Pub/Sub

<source>
  type gcloud_pubsub
  tag example.pull
  project <YOUR PROJECT>
  topic <YOUR TOPIC>
  subscription <YOUR SUBSCRIPTION>
  key <YOUR KEY>
  max_messages 1000
  return_immediately true
  pull_interval 2
  format json
</source>

<match example.pull>
  type stdout
</match>

fluent-plugin-gcloud-pubsub's People

Contributors

mdoi avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.