yongkun Goto Github PK

camus

LinkedIn's Kafka to HDFS pipeline.

cassandra

Enhancement of adaptive data placement.

commons-io

Add resumable Tailer.

d3

A JavaScript visualization library for HTML and SVG.

elasticsearch

Open Source, Distributed, RESTful Search Engine

flume

I created a new release v0.10 based on 0.9.3 (cdh3u1-base), major changes include: 1) Masterless Ack, please check https://issues.apache.org/jira/browse/FLUME-640, which was supposed to be merged into 0.10 if Flume was not upgraded to NG; 2) Append to HDFS with new file rotation method.

flume-0.9.3-cdh3u0-rakuten

I created a new release v0.10 based on 0.9.3 (cdh3u0), major changes include: 1) Masterless Ack, please check https://issues.apache.org/jira/browse/FLUME-640, which was supposed to be merged into 0.10 if Flume was not upgraded to NG; 2) Append to HDFS with new file rotation method.

flume-ng

I propose "Transactional Multiplex Sink" which uses single channel to fanout data stream to different receivers in one transaction. It is expected to greatly reduce the resource usage.

grafana

Grafana - A Graphite & InfluxDB Dashboard and Graph Editor

iombench

A microbenchmark for storage systems or devices such as hard disk and flash SSD.

iombench-results

Benchmark results by iombench https://github.com/yongkun/iombench

kafka-python

Kafka protocol support in Python, with offset enhancement for consumer. User can pass offset as a parameter to consumer, with the following values: Previous (-1): position stores in zookeeper last time; CurrentBeginning (-2): current beginning offset (may not be 0 due to TTL); PreviousOrCurrentBeginning (-3): Get previous offset firstly, if not available, use current beginning; Latest (-4): Start from latest, like tail Any other value >= 0: actual offset, raise exception if the offset is invalid. Yongkun Wang and KiranRaj Mariyappa

kibana

Kibana

kinetic-java

Kinetic Java Library and Simulator.

markdown-here

Google Chrome, Firefox, and Thunderbird extension that lets you write email in Markdown and render it before sending.

nosql-counter-benchmark

A benchmark to measure the throughput of 'counter' operation for NoSQL system.

patches

patches for Flume, thrift, etc.

pmfs

Persistent Memory File System

redis

Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes

thrift

I created two release (thrift-0.6.2 and thrift-0.7.1) to fix some issues in early version (0.6.x, 0.7.x) which is used for Apache Flume. These issues have been fixed in later versions (0.8.x+) of original repo. Main change is the buffered input/output stream to improve the performance for on 0.6.x & 0.7.x, change on 0.7.x also includes 'oneway' fix (THRIFT-1447). Two new versions are created: thrift-0.6.2 and thrift-0.7.1.

thrift-reuse-conn

A simple example on how to reuse the thrift connection to transmit data bi-directionally.

ycsb

Yahoo! Cloud Serving Benchmark. I change to use Hector client for Cassandra, for connection pooling and better load balancing.

yongkun.github.io

yongkunwang.github.io