dkmalungu / python_ml Goto Github PK

As a seasoned Data Engineer with expertise in designing and implementing robust data solutions, my GitHub profile showcases my contributions to various data-centric projects. With proficiency in PostgreSQL, Microsoft SQL Server, ClickHouse, and Snowflake, I have developed efficient ETL pipelines, designed scalable databases, and leveraged big data technologies such as Apache Spark for data processing and analysis. My repositories demonstrate my skills in Python and Scala, along with experience in utilizing cloud platforms like GCP and AWS for data storage and processing. I prioritize clean code, documentation, and collaborative development, as evidenced by my active use of Git and GitHub for version control and code collaboration. Explore my repositories to discover my problem-solving abilities, data modeling expertise, and passion for continuous learning in the field of data engineering.

Skill Summary:

• Data Engineering: Strong expertise in designing and implementing data solutions, including data platforms, ETL pipelines, and database management. Proficient in PostgreSQL, Microsoft SQL Server, ClickHouse, and Snowflake.
• Programming Languages: Highly skilled in Python and proficient in Scala. Experienced in utilizing programming languages for data manipulation, transformation, and analysis.
• Big Data Technologies: Knowledgeable in Apache Spark for large-scale data processing and analysis. Familiarity with Apache Airflow for data pipeline orchestration.
• Cloud Platforms: Proficient in working with Google Cloud Platform (GCP) and Amazon Web Services (AWS) for data storage, processing, and deployment.
• Business Intelligence: Skilled in using tools like MetaBase for data visualization and creating insightful reports and dashboards.
• Data Modeling and Warehousing: Well-versed in data modeling principles and experienced in building and optimizing databases for efficient data storage and retrieval.
• Version Control: Proficient in Git and GitHub for collaborative development and version control.
• Software Engineering: Knowledgeable in software engineering practices, including code refactoring, code quality evaluation, and CI/CD pipelines. Experienced in using tools like Prefect for data pipeline orchestration.
• Documentation and Testing: Strong experience in documenting data solutions, writing technical documentation, and implementing testing strategies to ensure data quality and reliability.
• Problem Solving: Excellent problem-solving skills with the ability to analyze complex data challenges and provide innovative and effective solutions.
• Communication and Leadership: Demonstrated leadership skills, including leading team meetings, workshops, and agile project management. Strong communication skills to collaborate effectively with cross-functional teams and stakeholders.
• Continuous Learning: Committed to staying updated with the latest technologies, tools, and trends in the field of data engineering and actively seeking opportunities for continuous learning and professional development.

Programing Languages Summary

• Python: Proficient in Python programming language with extensive experience in data engineering, ETL pipeline development, data mining and analysis, and creating reusable libraries. Contributions include building scalable ETL pipelines using Python, Pyspark, and SQL, as well as developing desktop applications and chatbot systems.
• SQL: Strong command of SQL for data manipulation, administration, and optimization. Skilled in working with PostgreSQL, Microsoft SQL Server, and Snowflake databases. Experience includes database configuration, optimization, and implementation of backup and restoration scripts using SQL.
• Scala: Experienced in using Scala programming language, particularly in the context of data engineering. Familiarity with dbt (data build tool) and building data transformations following the Kimball approach.
• Bash: Proficient in Bash scripting for automating tasks, database maintenance, and implementation of database backup and restoration scripts. Skilled in using Bash together with Nagios and ELK tools for database maintenance and administration.
• TensorFlow: Knowledgeable in TensorFlow, an open-source machine learning framework. Acquired certification in TensorFlow development from the Google Mobile Academy. Applied TensorFlow in building a chatbot system as a Full Stack Software Engineer.
• PyQt: Skilled in using PyQt framework for building smart meter water systems. Led a team in integrating and achieving bi-directional communication between sensors and the back-office system using PyQt.