Topic: stream-processing Goto Github

👇 Here are 927 public repositories matching this topic...

airtai / faststream
stream-processing,FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.
Organization: airtai
Home Page: https://faststream.airt.ai/latest/
asyncio kafka python rabbitmq asyncapi distributed-systems faststream fastkafka propan nats

apache / hudi
stream-processing,Upserts, Deletes And Incremental Processing on Big Data.
Organization: apache
Home Page: https://hudi.apache.org/
hudi apachehudi datalake bigdata apachespark incremental-processing stream-processing data-integration apacheflink

arroyosystems / arroyo
stream-processing,Distributed stream processing engine in Rust
Organization: arroyosystems
Home Page: https://arroyo.dev
data dev-tools infrastructure kafka rust sql stream-processing data-stream-processing stream-processing-engine

bytewax / bytewax
stream-processing,Python Stream Processing
Organization: bytewax
Home Page: https://docs.bytewax.io/
python stream-processing rust data-engineering data-processing data-science dataflow machine-learning streaming-data

clj-commons / manifold
stream-processing,A compatibility layer for event-driven abstractions
Organization: clj-commons
stream-processing future clojure async hacktoberfest stream

confluentinc / ksql
stream-processing,The database purpose-built for stream processing applications.
Organization: confluentinc
Home Page: https://ksqldb.io
stream-processing real-time sql interactive kafka kafka-connect materialized-views ksqldb ksqldb-documentation ksqldb-tutorials

douban / dpark
stream-processing,Python clone of Spark, a MapReduce alike framework in Python
Organization: douban
bigdata mapreduce dpark stream-processing spark python

edp963 / wormhole
stream-processing,Wormhole is a SPaaS (Stream Processing as a Service) Platform
Organization: edp963
Home Page: https://edp963.github.io/wormhole
wormhole stream-processing spark-streaming

fluent / fluent-bit
stream-processing,Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
Organization: fluent
Home Page: https://fluentbit.io
fluentd c logging data-collector fluent-bit forwarder cloudnative stream-processing sql-queries metrics

francoispqt / gojay
stream-processing,high performance JSON encoder/decoder with stream API for Golang
User: francoispqt
go golang decoder encoder perfomance json stream-decoder stream-processing

gearpump / gearpump
stream-processing,Lightweight real-time big data streaming engine over Akka
Organization: gearpump
Home Page: https://gearpump.github.io/gearpump/
akka stream-processing bigdata scala

google-ai-edge / mediapipe
stream-processing,Cross-platform, customizable ML solutions for live and streaming media.
Organization: google-ai-edge
Home Page: https://mediapipe.dev
mediapipe c-plus-plus computer-vision deep-learning android video-processing audio-processing mobile-development machine-learning inference

halaxa / json-machine
stream-processing,Efficient, easy-to-use, and fast PHP JSON stream parser
User: halaxa
php stream-processing json-parser json-iterator json-stream parsing

hazelcast / hazelcast
stream-processing,Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
Organization: hazelcast
Home Page: https://www.hazelcast.com
java hazelcast in-memory big-data scalability distributed caching hacktoberfest stream-processing low-latency

hazelcast / hazelcast-jet
stream-processing,Distributed Stream and Batch Processing
Organization: hazelcast
Home Page: https://jet-start.sh
java big-data stream-processing batch-processing event-processing low-latency kafka cdc hacktoberfest

hstreamdb / hstream
stream-processing,HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.
Organization: hstreamdb
Home Page: https://hstream.io/
streaming database iot stream-processing haskell real-time sql hstreamdb scale distributed-systems

infinyon / fluvio
stream-processing,Lean and mean distributed stream processing system written in rust and web assembly.
Organization: infinyon
Home Page: https://www.fluvio.io/
cloud-native streaming rust real-time serverless stateful stream-processing webassembly data-integration data-flow

javascriptdata / danfojs
stream-processing,Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.
Organization: javascriptdata
Home Page: https://danfo.jsdata.org/
data-analytics data-science pandas data-manipulation tensors dataframe javascript data-analysis danfojs stream-data

lw-lin / streaming-readings
stream-processing,Streaming System 相关的论文读物
User: lw-lin
stream-processing streaming flink spark-streaming storm heron dataflow drizzle millwheel s4

madd86 / awesome-system-design
stream-processing,A curated list of awesome System Design (A.K.A. Distributed Systems) resources.
User: madd86
relational-database message-broker hadoop-ecosystem nosql interview distributed-systems microservices-architecture stream-processing microservices

manuzhang / awesome-streaming
stream-processing,a curated list of awesome streaming frameworks, applications, etc
User: manuzhang
Home Page: https://manuzhang.github.io/awesome-streaming
awesome-list stream-processing awesome list

materializeinc / materialize
stream-processing,The data warehouse for operational workloads.
Organization: materializeinc
Home Page: https://materialize.com
rust database sql streaming kafka distributed-systems postgresql-dialect materialized-view stream-processing postgresql

memgraph / memgraph
stream-processing,Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.
Organization: memgraph
Home Page: https://memgraph.com
graph-algorithms graph-database graph-analysis stream-processing streaming-data kafka-streams kafka cypher graph nosql

nerevu / riko
stream-processing,A Python stream processing engine modeled after Yahoo! Pipes
Organization: nerevu
stream-processing etl parallelism rss data asynchronous library cli functional-programming featured

nodefluent / kafka-streams
stream-processing,equivalent to kafka-streams :octopus: for nodejs :sparkles::turtle::rocket::sparkles:
Organization: nodefluent
Home Page: https://nodefluent.github.io/kafka-streams/
kafka streams stream-processing kafka-streams node nodejs big-data

numaproj / numaflow
stream-processing,Kubernetes-native platform to run massively parallel data/streaming jobs
Organization: numaproj
Home Page: https://numaflow.numaproj.io
kubernetes stream-processing data-processing k8s pipeline map-reduce hacktoberfest

online-ml / river
stream-processing,🌊 Online machine learning in Python
Organization: online-ml
Home Page: https://riverml.xyz
incremental-learning machine-learning python online-learning online-statistics data-science streaming online-machine-learning streaming-data concept-drift

oxnr / awesome-bigdata
stream-processing,A curated list of awesome big data frameworks, ressources and other awesomeness.
User: oxnr
Home Page: https://github.com/onurakpolat/awesome-bigdata
awesome bigdata data-warehouse data-analytics data-visualization data-stream database data-science data awesome-list

pathwaycom / pathway
stream-processing,Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Organization: pathwaycom
Home Page: https://pathway.com
batch-processing kafka pathway python streaming machine-learning-algorithms real-time data-analytics data-pipelines data-processing dataflow etl etl-framework iot-analytics rust stream-processing time-series-analysis

peerdb-io / peerdb
stream-processing,Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
Organization: peerdb-io
Home Page: https://peerdb.io
etl sql bigquery cloud-native distributed-systems kafka postgres postgresql realtime rust

pipeless-ai / pipeless
stream-processing,An open-source computer vision framework to build and deploy apps in minutes
Organization: pipeless-ai
Home Page: https://pipeless.ai
artificial-intelligence computer-vision multimedia multimedia-applications cloud deep-learning machine-learning object-detection video yolo

pipelinedb / pipelinedb
stream-processing,High-performance time-series aggregation for PostgreSQL
Organization: pipelinedb
Home Page: https://www.pipelinedb.com
realtime stream-processing analytics sql push postgresql pipelinedb time-series aggregation

quixio / quix-streams
stream-processing,100% Python stream processing with Streaming DataFrames
Organization: quixio
Home Page: https://docs.quix.io
kafka python stream-processing data-engineering data-science machine-learning real-time-data-processing streaming-data streaming-data-pipelines streaming-data-processing

redpanda-data / connect
stream-processing,Fancy stream processing made operationally mundane
Organization: redpanda-data
Home Page: https://docs.redpanda.com/redpanda-connect/about/
message-queue stream-processing streaming-data message-bus logs stream-processor cqrs event-sourcing go golang

redpanda-data / redpanda
stream-processing,Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
Organization: redpanda-data
Home Page: https://redpanda.com
kafka cpp kubernetes go streaming redpanda containers realtime event-driven microservices storage-engine stream-processing seastar

reugn / go-streams
stream-processing,A lightweight stream processing library for Go
User: reugn
Home Page: https://pkg.go.dev/github.com/reugn/go-streams
stream-processing pipeline etl kafka data-stream kafka-streams streams redis pulsar data-pipeline

risingwavelabs / risingwave
stream-processing,SQL stream processing, analytics, and management. We decouple storage and compute to offer instant failover, dynamic scaling, speedy bootstrapping, and efficient joins.
Organization: risingwavelabs
Home Page: https://www.risingwave.com/slack
database stream-processing cloud-native sql distributed-database rust serverless postgresql real-time postgres

robinhood / faust
stream-processing,Python Stream Processing
Organization: robinhood
asyncio distributed-systems kafka kafka-streams python stream-processing

siddhi-io / siddhi
stream-processing,Stream Processing and Complex Event Processing Engine
Organization: siddhi-io
Home Page: http://siddhi.io
cep cloud cloud-native cncf complex-event-processing library online-learning stream-processing streaming-data-processing

spring-cloud / spring-cloud-dataflow
stream-processing,A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
Organization: spring-cloud
Home Page: https://dataflow.spring.io
microservices-architecture orchestration stream-processing batch-processing predictive-analytics cloud-native datapipelines

spring-cloud / spring-cloud-stream
stream-processing,Framework for building Event-Driven Microservices
Organization: spring-cloud
Home Page: http://cloud.spring.io/spring-cloud-stream
microservices messaging event-driven-microservices stream-processing reactive cloud-native

threedotslabs / watermill
stream-processing,Building event-driven applications the easy way in Go.
Organization: threedotslabs
Home Page: https://watermill.io
watermill event-driven event-sourcing sagas events stream-processing reactive go golang rabbitmq

timeplus-io / proton
stream-processing,A streaming SQL engine, a fast and lightweight alternative to ksqlDB and Apache Flink, 🚀 powered by ClickHouse.
Organization: timeplus-io
Home Page: https://timeplus.com
clickhouse kakfa sql redpanda single-binary flink-alternative high-performance stream-processing analytics ksqldb-alternative

uhop / stream-json
stream-processing,The micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API.
User: uhop
streaming-json javascript-objects parse-json-files parser stream-components stream-processing

vectordotdev / vector
stream-processing,A high-performance observability data pipeline.
Organization: vectordotdev
Home Page: https://vector.dev
router logs metrics rust observability forwarder vector parser events stream-processing

ververica / flink-sql-cookbook
stream-processing,The Apache Flink SQL Cookbook is a curated collection of examples, patterns, and use cases of Apache Flink SQL. Many of the recipes are completely self-contained and can be run in Ververica Platform as is.
Organization: ververica
apache-flink flink flink-sql sql stream-processing

vladimirvivien / automi
stream-processing,A stream processing API for Go (alpha)
User: vladimirvivien
golang go streaming-api stream-processing data-stream

wallaroolabs / wally
stream-processing,Distributed Stream Processing
Organization: wallaroolabs
Home Page: https://www.wallaroolabs.com
stream-processing stream-processor stream-processing-engine pony-language python linux golang go framework api

yomorun / yomo
stream-processing,🦖 Stateful Serverless Framework for building Geo-distributed Edge AI Infra
Organization: yomorun
Home Page: https://yomo.run
edge-computing realtime quic stream-processing low-latency serverless distributed-cloud geodistributedsystems webassembly yomo

zhisheng17 / flink-learning
stream-processing,flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
User: zhisheng17
Home Page: http://www.54tianzhisheng.cn/tags/Flink/
flink kafka elasticsearch spark redis mysql rocketmq hbase rabbitmq stream-processing streaming clickhouse loki influxdb opentsdb