Topic: spark-sql Goto Github

👇 Here are 712 public repositories matching this topic...

airbnb / airbnb-spark-thrift
spark-sql,A library for loadling Thrift data into Spark SQL
Organization: airbnb
spark spark-streaming spark-sql thrift

almond-sh / almond
spark-sql,A Scala kernel for Jupyter
Organization: almond-sh
Home Page: https://almond.sh
jupyter scala jupyter-notebook repl jupyter-kernels spark spark-sql

apache / incubator-gluten
spark-sql,Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
Organization: apache
Home Page: https://gluten.apache.org/
clickhouse simd spark-sql vectorization velox arrow

apache / kyuubi
spark-sql,Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
Organization: apache
Home Page: https://kyuubi.apache.org/
spark hive sql thrift jdbc spark-sql data-lake hadoop kubernetes hacktoberfest

bluishglc / bdp
spark-sql,A prototype project of big data platform, the source codes of the book Big Data Platform Architecture and Prototype
User: bluishglc
Home Page: https://laurence.blog.csdn.net/article/details/106851739
bigdata prototype quickstart spark spark-streaming spark-sql demo oozie redis kafka

chabane / bigdata-playground
spark-sql,A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
User: chabane
docker spark-sql scala kafka hbase parquet avro nodejs angular graphql

cuebook / cuelake
spark-sql,Use SQL to build ELT pipelines on a data lakehouse.
Organization: cuebook
Home Page: https://cuelake.cuebook.ai
apache-iceberg delta lakehouse datalake data-lake elt etl data-engineering data-integration data-ingestion

databricks / learningsparkv2
spark-sql,This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
Organization: databricks
Home Page: https://learning.oreilly.com/library/view/learning-spark-2nd/9781492050032/
apache-spark spark structured-streaming spark-sql spark-mllib mllib mlflow delta-lake

dbiir / paraflow
spark-sql,A real-time analytical system for ID-associated data
Organization: dbiir
Home Page: https://dbiir.github.io/paraflow/
presto spark-sql parquet orc hadoop kafka

dotnet / spark
spark-sql,.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
Organization: dotnet
Home Page: https://dot.net/spark
spark csharp dotnet analytics bigdata spark-streaming spark-sql machine-learning fsharp dotnet-core

getredash / redash
spark-sql,Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
Organization: getredash
Home Page: http://redash.io/
analytics athena bi bigquery business-intelligence dashboard databricks hacktoberfest javascript mysql postgresql python redash redshift spark spark-sql visualization

groda / big_data
spark-sql,Tutorials on Big Data essentials: Hadoop, MapReduce, Spark.
User: groda
big-data bigdata spark spark-sql docker mapreduce mapreduce-bash pyspark hadoop testdfsio

hablapps / sparkoptics
spark-sql,Optics for Spark DataFrames
Organization: hablapps
spark-sql spark optics scala dataframe dataframes

harryprince / awesome-sparklyr
spark-sql,An awesome sparklyr related package collection
User: harryprince
sparklyr awesome apache-spark big-data machine-learning dbi r-stats r spark-sql

harryprince / geospark
spark-sql,bring sf to spark in production
User: harryprince
Home Page: https://github.com/harryprince/geospark/wiki
large-scale-spatial-analysis spark-sql r gis spatial-queries spatial-analysis apache-spark sparklyr-extension

huangyueranbbc / sparkdemo
spark-sql, spark全示例代码(java、scala) Spark most full instance code DEMO (java、scala)
User: huangyueranbbc
spark hadoop bigdata operator sparkjava spark-sql spark-streaming sparkfun-products sparkline sparkp

iobruno / data-engineering-zoomcamp
spark-sql,Data Engineering examples covering Airflow and Mage for workflows; dbt for BigQuery, Redshift, ClickHouse; Spark and Kafka for Batch/Streaming Processing
User: iobruno
pyspark spark spark-sql kafka kafka-streams ksqldb workflow-orchestration dbt-bigquery dbt-postgres dbt-clickhouse

izhangzhihao / real-time-data-warehouse
spark-sql,Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi
User: izhangzhihao
flink data-warehouse real-time-data-warehouse data-warehousing flink-sql debezium kafka elasticsearch delta-lake cdc

jaceklaskowski / spark-workshop
spark-sql,Apache Spark™ and Scala Workshops
User: jaceklaskowski
Home Page: https://jaceklaskowski.github.io/spark-workshop/
workshop spark apache-spark spark-sql spark-workshops spark-mllib spark-structured-streaming

japila-books / spark-sql-internals
spark-sql,The Internals of Spark SQL
Organization: japila-books
Home Page: https://books.japila.pl/spark-sql-internals
apache-spark spark-sql internals spark mkdocs-material book

kaantas / spark-twitter-sentiment-analysis
spark-sql,Sentiment Analysis of a Twitter Topic with Spark Structured Streaming
User: kaantas
twitter twitter-api twitter-sentiment-analysis apache-spark apache-kafka python pyspark afinn pykafka spark-sql

kevinschaich / pyspark-cheatsheet
spark-sql,🐍 Quick reference guide to common patterns & functions in PySpark.
User: kevinschaich
Home Page: https://spark.apache.org/docs/latest/api/python/pyspark.sql.html
pyspark pyspark-tutorial cheatsheet cheat cheatsheets reference references documentation docs data-science

learningjournal / spark-streaming-in-python
spark-sql,Apache Spark 3 - Structured Streaming Course Material
User: learningjournal
Home Page: https://www.learningjournal.guru
apache-spark spark-streaming spark-sql pyspark python big-data bigdata data-lake

learningjournal / spark-streaming-in-scala
spark-sql,Apache Spark 3 - Structured Streaming Course Material
User: learningjournal
Home Page: https://www.learningjournal.guru
apache-spark spark-streaming spark-sql scala spark big-data bigdata datalake

learningjournal / sparkprogramminginscala
spark-sql,Apache Spark Course Material
User: learningjournal
Home Page: https://www.learningjournal.guru
apache-spark spark spark-sql spark-scala big-data bigdata datalake data-lake scala

martandsingh / apachespark
spark-sql,This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
User: martandsingh
apachespark data-analysis data-engineering database databricks datalake deltalake etl-pipeline hadoop hive

mayur2810 / sope
spark-sql,Apache Spark ETL Utilities
User: mayur2810
Home Page: https://mayur2810.github.io/sope
spark-sql spark etl framework yaml transformer scala dsl

mc2-project / opaque-sql
spark-sql,An encrypted data analytics platform
Organization: mc2-project
security privacy machine-learning analytics spark spark-sql enclave

microsoft / data-accelerator
spark-sql,Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
Organization: microsoft
spark spark-streaming spark-sql sparksql streaming-data streaming servicefabric nodejs docker hdinsight

microsoft / mcw-big-data-analytics-and-visualization
spark-sql,MCW Big data analytics and visualization
Organization: microsoft
machine-learning azure-data-factory hdinsight spark-sql database-administrator power-bi

minio / spark-select
spark-sql,A library for Spark DataFrame using MinIO Select API
Organization: minio
select bigdata spark spark-sql amazon-s3 parquet-files pyspark sbt minio

oeljeklaus-you / useractionanalyzeplatform
spark-sql,电商用户行为分析大数据平台
User: oeljeklaus-you
spark java hadoop sparkjava spark-sql accumulator kyro

polomarcus / spark-structured-streaming-examples
spark-sql,Spark Structured Streaming / Kafka / Cassandra / Elastic
User: polomarcus
cassandra kafka spark spark-sql structured-streaming

qbeast-io / qbeast-spark
spark-sql,Qbeast-spark: DataSource enabling multi-dimensional indexing and efficient data sampling. Big Data, free from the unnecessary!
Organization: qbeast-io
Home Page: https://qbeast.io/qbeast-our-tech/
spark scala big-data sampling datasource data-lakehouse spark-sql

qubole / sparklens
spark-sql,Qubole Sparklens tool for performance tuning Apache Spark
Organization: qubole
Home Page: http://sparklens.qubole.com
spark scala simulation scheduler scheduling performance performance-analysis performance-metrics performance-tuning performance-visualization

sharpdata / sharpetl
spark-sql,Write ETL using your favorite SQL dialects
Organization: sharpdata
Home Page: https://sharpdata.github.io/SharpETL/
etl etl-framework hive scala spark spark-sql sql datawarehouse data-warehouse bigdata

sjyttkl / spark_learning
spark-sql,尚硅谷大数据Spark-2019版最新 Spark 学习
User: sjyttkl
Home Page: https://www.bilibili.com/video/av62992342/?p=67
spark spark-sql spark-core

spider-123-eng / spark
spark-sql,Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .
User: spider-123-eng
Home Page: https://github.com/Re1tReddy/Spark
spark-streaming-data streaming consumer spark-transformations spark-to-cassandra-connection spark-kafka-integration spark-dataframes spark-joins spark-hive-context spark-jdbc-connection

spirom / spark-data-sources
spark-sql,Developing Spark External Data Sources using the V2 API
User: spirom
spark data-sources spark-sql

streamnative / awesome-pulsar
spark-sql,A curated list of Pulsar tools, integrations and resources.
Organization: streamnative
apache-bookkeeper apache-flink apache-kafka apache-pulsar apache-spark apache-storm elastic-beats grafana-dashboard messaging prometheus pub-sub spark spark-sql spark-structured-streaming

streamnative / pulsar-spark
spark-sql,Spark Connector to read and write with Pulsar
Organization: streamnative
apache-pulsar apache-spark spark structured-streaming spark-sql data-processing data-science batch-processing stream-processing flink

sunwu51 / bigdatatutorial
spark-sql,bigdatatutorial
User: sunwu51
Home Page: https://sunwu51.github.io/bigdatatutorial
hadoop spark spark-sql spark-streaming postgresql greenplum bigdata

tejasjbansal / helthcare-system
spark-sql,Data cleaning, pre-processing, and Analytics on a Health care data using Spark and Python.
User: tejasjbansal
big-data big-data-projects hive python spark spark-sql sqoop

thomas-george-t / movies-analytics-in-spark-and-scala
spark-sql,Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.
User: thomas-george-t
spark spark-sql rdd shell-script big-data dataframes movielens-data-analysis big-data-analytics big-data-projects spark-programs

vim89 / datapipelines-essentials-python
spark-sql,Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
User: vim89
apache-spark spark spark-sql python python3 pyspark etl etl-pipeline etl-framework etl-components

wangj1106 / recommendmoteur
spark-sql,电影推荐系统、电影推荐引擎、使用Spark完成的电影推荐引擎
User: wangj1106
recommendation-engine recommender-system recommendation movies als spark spark-streaming spark-sql flume kafka

wathon / data_engineering_with_python-track-datacamp
spark-sql,Data Engineer with Python lecture notes from #datacamp.
User: wathon
data-engineering python datacamp-track shell sql scala aws-boto pyspark spark-sql mongodb

xiaogp / recsys_spark
spark-sql,Spark SQL 实现 ItemCF，UserCF，Swing，推荐系统，推荐算法，协同过滤
User: xiaogp
collaborative-filtering recommender-system spark-sql

zsvoboda / ngods
spark-sql,New generation opensource data stack
User: zsvoboda
analytics data data-pipeline iceberg jdbc presto prestodb prestosql python scala

zsvoboda / ngods-stocks
spark-sql,New Generation Opensource Data Stack Demo
User: zsvoboda
cube dagster datahub dbt iceberg metabase python spark spark-sql trino