Apache Spark Reference Architecture

This repository is intended as a collaboration place around Apache Spark Reference Architectures. These Reference Architectures serves as a blueprint for composing capabilities for data, workload, and access management into system architecture for Apache Spark cluster deployments.

Reference architectures can vary from generic template reference architectures that are broadly applicable to specific use cases reference architectures that very tied to a specific domain. In this effort our goal is to collaborate towards multi-domain reference architectures that gravitates to the middle of this continuum (e.g. ETL, Streaming ETL, Deep Learning, etc)

Reference Architectures

Streaming ETL

Data Pipelines

Deep Learning