marin-ma Goto Github PK

arrow-1

Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication. Languages currently supported include C, C++, Java, JavaScript, Python, and Ruby.

gluten

hudi

Upserts, Deletes And Incremental Processing on Big Data.

oap

Optimized Analytics Package for Spark Platform

qat-zstd-plugin

qatzip

Compression Library accelerated by Intel® QuickAssist Technology

ray

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

raydp

RayDP: Distributed data processing library that provides simple APIs for running Spark on Ray and integrating Spark with distributed deep learning and machine learning frameworks.

spark

Mirror of Apache Spark

spark-sql-perf

substrait

A cross platform way to express data transformation, relational algebra, standardized record expression and plans.

velox-oap

A new C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.