Python Developer

Role Overview: As a Python Developer at Data Unveil, you will be responsible for developing and optimizing ETL workflows and data pipelines using Python and PySpark. You will work on writing and optimizing SQL queries for Oracle and MySQL databases, utilizing PySpark for distributed data processing, and integrating AWS services like S3, Lambda, EC2, RDS, Glue, etc. Your role will involve implementing data ingestion, transformation, and validation processes to support business analytics and reporting. Collaboration with data teams, debugging and optimizing Spark jobs and database queries, ensuring data integrity, security, and performance optimization are key aspects of this role. Key Responsibilities: - Develop and optimize ETL workflows and data pipelines using Python and PySpark. - Write and optimize SQL queries for Oracle and MySQL databases. - Use PySpark for distributed data processing and large-scale transformations. - Implement data ingestion, transformation, and validation processes. - Integrate AWS services like S3, Lambda, EC2, RDS, Glue, etc. - Ensure data integrity, security, and performance optimization. - Collaborate with data teams to understand requirements. - Debug and optimize Spark jobs and database queries. - Use version control (Git) and participate in code reviews. Qualifications Required: - Proficient in Python, Pyspark, Linux, AWS services such as S3, Lambda, EC2, RDS, etc., Git, CI/CD practices. - Strong experience with Oracle, MySQL, and data extraction, transformation, and loading (ETL) processes.,