Topic: big-data Goto Github

Some thing interesting about big-data

👇 Here are 3925 public repositories matching this topic...

alibaba / fastjson2

big-data,🚄 FASTJSON2 is a Java JSON library with excellent performance.

Organization: alibaba

json fastjson json-parser jsonb java-json json-serialization json-path high-performance fastjson2 android

alibaba / graphscope

big-data,🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统

Organization: alibaba

Home Page: https://graphscope.io

graph graph-computation graph-neural-networks gremlin graph-analytics graph-data graph-computing analytics big-data data-science

alldatacenter / alldata

big-data,AllData数据中台开源项目，以数据平台为底座，以数据中台为桥梁，以机器学习平台为中层框架，以大模型应用为上游产品，提供全链路数字化解决方案。加入技术社区：https://docs.qq.com/doc/DVHlkSEtvVXVCdEFo

Organization: alldatacenter

Home Page: http://122.51.43.143:5173/ui_moat

flink griffin hudi iceberg kong paimon ranger tis datart dinky

amark / gun

big-data,An open source cybersecurity protocol for syncing decentralized graph data.

User: amark

Home Page: https://gun.eco/docs

machine-learning artificial-intelligence big-data blockchain p2p decentralized graph cryptography crypto offline-first

andkret / cookbook

big-data,The Data Engineering Cookbook

User: andkret

Home Page: https://learndataengineering.com/

data-engineer data-engineering big-data best-practices cookbook

apache / beam

big-data,Apache Beam is a unified programming model for Batch and Streaming data processing.

Organization: apache

Home Page: https://beam.apache.org/

python java big-data beam batch golang sql streaming

apache / calcite

big-data,Apache Calcite

Organization: apache

Home Page: https://calcite.apache.org/

geospatial calcite java big-data hadoop sql

apache / couchdb

big-data,Seamless multi-master syncing database with an intuitive HTTP/JSON API, designed for reliability

Organization: apache

Home Page: https://couchdb.apache.org/

content network-server http cloud erlang javascript couchdb big-data network-client database

apache / datafusion

big-data,Apache DataFusion SQL Query Engine

Organization: apache

Home Page: https://datafusion.apache.org/

arrow big-data dataframe datafusion olap python query-engine rust sql

apache / flink

big-data,Apache Flink

Organization: apache

scala java big-data flink python sql

apache / hive

big-data,Apache Hive

Organization: apache

Home Page: https://hive.apache.org/

java hive database sql apache big-data hadoop

apache / ignite

big-data,Apache Ignite

Organization: apache

Home Page: https://ignite.apache.org/

distributed-sql-database iot osgi network-client ignite data-management-platform big-data cloud database network-server

apache / incubator-hugegraph

big-data,A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)

Organization: apache

Home Page: https://hugegraph.apache.org

database graph-database graph gremlin big-data graphdb

apache / iotdb

big-data,Apache IoTDB

Organization: apache

Home Page: https://iotdb.apache.org/

timeseries iot big-data java database nosql tsdb

apache / predictionio

big-data,PredictionIO, a machine learning server for developers and ML engineers.

Organization: apache

Home Page: https://predictionio.apache.org

big-data predictionio scala

apache / spark

big-data,Apache Spark - A unified analytics engine for large-scale data processing

Organization: apache

Home Page: https://spark.apache.org/

python scala r java big-data jdbc sql spark

apache / zeppelin

big-data,Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

Organization: apache

Home Page: https://zeppelin.apache.org/

scala java big-data zeppelin javascript spark flink database nosql

arkime / arkime

big-data,Arkime is an open source, large scale, full packet capturing, indexing, and database system.

Organization: arkime

Home Page: https://arkime.com

network-monitoring pcap packet-capture nsm javascript c big-data security

binhnguyennus / awesome-scalability

big-data,The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

User: binhnguyennus

architecture awesome awesome-list backend big-data computer-science design-patterns devops distributed-systems interview interview-practice interview-questions lists machine-learning programming resources scalability system system-design web-development

big-data,A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Organization: catboost

Home Page: https://catboost.ai

machine-learning decision-trees gradient-boosting gbm gbdt python r kaggle gpu-computing catboost

clickhouse / clickhouse

big-data,ClickHouse® is a real-time analytics DBMS

Organization: clickhouse

Home Page: https://clickhouse.com

dbms olap analytics sql distributed-database big-data mpp clickhouse hacktoberfest cpp rust ai

crate / crate

big-data,CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.

Organization: crate

Home Page: https://cratedb.com/product

analytics big-data cratedb database dbms distributed distributed-database distributed-sql-database elasticsearch industrial-iot iot iot-analytics iot-database lucene olap postgresql sql time-series tsdb vector-database

cython / cython

big-data,The most widely used Python to C compiler

Organization: cython

Home Page: https://cython.org

python cython cpython cpython-extensions c cpp performance big-data

databricks / koalas

big-data,Koalas: pandas API on Apache Spark

Organization: databricks

big-data data-science dataframe mlflow pandas pydata spark

delta-io / delta

big-data,An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Organization: delta-io

Home Page: https://delta.io

acid analytics big-data delta-lake spark

donnemartin / data-science-ipython-notebooks

big-data,Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

User: donnemartin

python machine-learning deep-learning data-science big-data aws tensorflow theano caffe scikit-learn

feast-dev / feast

big-data,The Open Source Feature Store for Machine Learning

Organization: feast-dev

Home Page: https://feast.dev

machine-learning features ml big-data feature-store python mlops data-engineering data-science data-quality

featurebasedb / featurebase

big-data,A crazy fast analytical database, built on bitmaps. Perfect for ML applications. Learn more at: http://docs.featurebase.com/. Start a Docker instance: https://hub.docker.com/r/featurebasedb/featurebase

Organization: featurebasedb

Home Page: https://www.featurebase.com

go bitmap index pilosa big-data database sql

h2oai / h2o-3

big-data,H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Organization: h2oai

Home Page: http://h2o.ai

h2o machine-learning data-science deep-learning big-data ensemble-learning gbm random-forest naive-bayes pca

hazelcast / hazelcast

big-data,Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

Organization: hazelcast

Home Page: https://www.hazelcast.com

big-data caching data-in-motion data-insights distributed distributed-computing distributed-systems hacktoberfest hazelcast in-memory java low-latency real-time scalability stream-processing

heibaiying / bigdata-notes

big-data,大数据入门指南 :star:

User: heibaiying

hadoop hdfs yarn mapreduce hive spark storm hbase scala kafka

jostmey / nakedtensor

big-data,Bare bone examples of machine learning in TensorFlow

User: jostmey

tensorflow tensorflow-tutorials distributed-computing simple big-data linear-regression tensorflow-examples tensorflow-exercises

microsoft / synapseml

big-data,Simple and Distributed Machine Learning

Organization: microsoft

Home Page: http://aka.ms/spark

spark pyspark azure scala microsoft ml machine-learning databricks cognitive-services lightgbm

moataz-elmesmary / data-science-roadmap

big-data,Data Science Roadmap from A to Z

User: moataz-elmesmary

Home Page: https://www.linkedin.com/posts/moatazelmesmary_github-moataz-elmesmarydata-science-roadmap-activity-6974293994960769024-yCdi?utm_source=share&utm_medium=member_desktop

data-analysis data-engineering data-science data-visualization deep-learning machine-learning mathematics probability python sql

pachyderm / pachyderm

big-data,Data-Centric Pipelines and Data Versioning

Organization: pachyderm

Home Page: https://www.pachyderm.com/

go pachyderm docker analytics big-data containers distributed-systems kubernetes data-science data-analysis

paradedb / paradedb

big-data,Postgres for Search and Analytics

Organization: paradedb

Home Page: https://paradedb.com

elasticsearch sparse-vector hybrid-search database bm25 full-text-search analytics olap htap big-data

prestodb / presto

big-data,The official home of the Presto distributed SQL query engine for big data

Organization: prestodb

Home Page: http://prestodb.io

big-data data hadoop hive java lakehouse presto query sql

provectus / kafka-ui

big-data,Open-Source Web UI for Apache Kafka Management

Organization: provectus

kafka-ui kafka-brokers kafka kafka-streams kafka-client opensource kafka-connect kafka-producer streams big-data

questdb / questdb

big-data,QuestDB is an open source time-series database for fast ingest and SQL queries

Organization: questdb

Home Page: https://questdb.io

time-series low-latency database sql postgres grafana simd questdb tsdb java

quickwit-oss / quickwit

big-data,Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.

Organization: quickwit-oss

Home Page: https://quickwit.io

rust log-management logs tantivy cloud-native open-source big-data cloud-storage distributed-tracing search-engine

risingwavelabs / risingwave

big-data,SQL stream processing, analytics, and management. We decouple storage and compute to offer instant failover, dynamic scaling, speedy bootstrapping, and efficient joins.

Organization: risingwavelabs

Home Page: https://www.risingwave.com/slack

database stream-processing cloud-native sql distributed-database rust serverless postgresql real-time postgres

rom1504 / img2dataset

big-data,Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

User: rom1504

deep-learning dataset big-data image multimodal image-dataset download-images

starrocks / starrocks

big-data,StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.

Organization: starrocks

Home Page: https://starrocks.io

database olap sql analytics big-data realtime-database vectorized distributed-database real-time-analytics mpp

tangbc / vue-virtual-scroll-list

big-data,⚡️A vue component support big amount data list with high render performance and efficient.

User: tangbc

Home Page: https://tangbc.github.io/vue-virtual-scroll-list

virtual-list big-data infinite-scroll

trinodb / trino

big-data,Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Organization: trinodb

Home Page: https://trino.io

java presto hive hadoop big-data sql prestodb database databases distributed-systems

tschellenbach / stream-framework

big-data,Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

User: tschellenbach

Home Page: https://getstream.io/

activity-stream cassandra feed news news-feed big-data redis activity-feed