mouhamed-jinja Goto Github PK

airflow-pg-integration-with-orm

airflow-spark-postgres-integration

The simplest way to set up a cluster environment that includes Spark, Airflow, and Postgres.

awake-sleep-yawn-detection-ai-model

blogapi

BlogAPI - A RESTful API built with FastAPI and PostgreSQL for managing blog post data, including authentication, authorization, validation, and error handling functionality.

cnn

hadoop-docker-spark-sql-hive-data-integration-and-warehousing-project

This project leverages Hadoop, Spark, SQL, and Hive for efficient data integration, transformation, warehousing, and analytics. It provides a comprehensive solution for managing and analyzing large datasets.

kafka-simple-python-consumer-producer

kafka cluster with KRaft, and simple python producer and consumer

kafka-spark-streaming-integration

Integrate Kafka with Zookeeper with spark, to build simple streaming application, and the producer is simple python application

lab-7-etl-on-a-dataset-by-using-aws-glue

lab1-aws-data-engineering-75653

postgres-database-replication

Discover the magic of database replication! This GitHub project showcases PostgreSQL and Docker Compose to optimize high-load reads. Explore the setup with one master and three followers for efficient, high-performance databases

postgres-s3-data-migration-with-airflow

Airflow Data Migration Project: A comprehensive Airflow project demonstrating data migration from PostgreSQL to AWS S3. Leverage the power of Airflow's operators, connections, and hooks to build robust and scalable data pipelines. Ideal for data engineering enthusiasts looking to learn and implement Airflow in real-world scenarios

postgresblend-data-pipeline

"PostgresBlend Data Pipeline" is a comprehensive data integration solution designed to seamlessly merge diverse data sources into a unified PostgreSQL Data Warehouse. This project streamlines the process of integrating data from CSVs, JSON, Parquet, and MySQL databases, utilizing Apache Spark for efficient transformation and organization.

python-airflow-postgres-docker-dwh

This repository contains Apache Airflow Directed Acyclic Graphs (DAGs) and associated scripts for orchestrating an Extract, Transform, Load (ETL) workflow. The workflow is designed to extract data from a source, perform transformations, and load it into a data warehouse.

sign_language_recognition

Implemented a NN project for Sign Language Recognition using CNN. Dataset preprocessing, data augmentation, and model training with Adam optimizer. Evaluated performance using confusion matrix. Expertise in CNN-based image classification.

spark-kafka-cassandra-streaming-pipeline

an end-to-end data streaming pipeline, seamlessly integrating Python, Kafka, Spark Streaming, Docker, and Airflow. Effortlessly process, transmit, and analyze real-time data with this comprehensive project, designed for efficient and scalable streaming applications

text-summarization-using-nlp-fastapi-sqlite

This project demonstrates my NLP and web development skills using FastAPI to create a text summarization application. Users can summarize text and save results to SQLite. The application is containerized with Docker for easy deployment. I gained experience with NLP, RESTful APIs with FastAPI, and containerizing apps.