Data Engineer

As a Senior Data Engineer, you will be responsible for designing and optimizing large-scale data pipelines using Databricks and the AWS ecosystem. Your primary role will involve working with technologies such as PySpark, Delta Lake, MLflow, Glue, S3, Lambda, Redshift, EMR, and more. Your expertise in Databricks and AWS with at least 7 years of experience is mandatory for this role. **Key Responsibilities:** - Design and optimize large-scale data pipelines using Databricks and AWS technologies - Utilize Databricks features such as PySpark, Delta Lake, and MLflow - Work with AWS services including Glue, S3, Lambda, Redshift, and EMR - Develop and maintain Data Lake solutions - Implement ETL pipelines and ensure data quality and integrity - Collaborate with cross-functional teams to deploy CI/CD pipelines - Proficient in Python and SQL for data manipulation and analysis **Qualifications Required:** - 7+ years of experience working with Databricks and AWS - Strong knowledge of Databricks technologies such as PySpark, Delta Lake, and MLflow - Hands-on experience with AWS services like Glue, S3, Lambda, Redshift, and EMR - Proficiency in building and optimizing ETL pipelines - Familiarity with Data Lake architecture and design principles - Experience in implementing CI/CD pipelines for automated deployments Feel free to drop your resume at anindita.goswami@prohrstrategies.com or directly message for more information if you are interested or know someone suitable for this role.,