Shubham Kumar Jaiswal's Projects
This Project demonstrates the Technology shift in Automobile Firm to resolve the data engineering challenge of manual data ops. AWS Cloud Services implemented here as: S3 bucket for lake storage incoming batches, Lambda Python Script for automating the validation function call and Glue Crawler to generate relational table with successful testing.
The objective of this project is to build a flexible semi-Automated Data Pipeline using PySpark for KCC Analytics Department to provide a easiness of ETL ecosystem with scalable workloads, so they can solely focus on BI Reporting and valuable decision making.
In this project,Code as a Pipeline implemented as ETL process to extract news from well known Media and transformed to dataframe with Vandersentiment Analyzer then delivered to End user through email notifier and loaded data seperately into CSV file locally.
Config files for my GitHub profile.