Pyspark Data Engineer

Krazy Mantra HR Solutions Pvt. Ltd

5 - 7 years

3 - 6 Lacs

hyderabad pune

Posted:2 days ago| Platform:

Apply

Skills Required

hive scala pyspark hadoop python continuous integration confluence ci/cd apache pig data architecture control-m sql git data modeling spark jenkins big data hbase jira github oozie airflow data processing data engineering scrum gitlab agile sqoop

Work Mode

Work from Office

Job Type

Full Time

Job Description

We are looking for a skilled Pyspark Data Engineer with 5 to 7 years of experience, located in Pune and Hyderabad. The ideal candidate will have expertise in Pyspark, Scala, Hive, Airflow, Control M, CICD, Python, Hadoop, Data Modelling, and experience with Jira, Confluence, GitHub, or other similar Agile Scrum technologies.

Roles and Responsibility

Design and develop scalable data pipelines using Pyspark and Airflow.
Collaborate with cross-functional teams to identify and prioritize project requirements.
Develop and maintain large-scale data systems using Hadoop and Data Modelling techniques.
Troubleshoot and resolve complex technical issues related to data processing and analysis.
Implement CI/CD pipelines using tools like Jenkins or GitLab CI/CD.
Work closely with stakeholders to understand business needs and provide data-driven insights.

Job Requirements

Strong proficiency in Pyspark, Scala, Hive, Airflow, Control M, CICD, Python, Hadoop, and Data Modelling.
Experience with Agile Scrum methodologies and tools like Jira, Confluence, or GitHub.
Excellent problem-solving skills and the ability to work in a fast-paced environment.
Strong communication and collaboration skills.
Ability to design and implement scalable data architectures.
Experience with version control systems like Git.
Notice period: Immediate - 45 days.
Full-time graduation is required.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.