Pyspark Lead

Krazy Mantra HR Solutions Pvt. Ltd

5 - 8 years

5 - 9 Lacs

bengaluru

Posted:-1 days ago| Platform:

Apply

Skills Required

hive airflow pyspark data bricks data modeling python scala data processing emr data engineering docker pipeline data quality java spark kafka linux jenkins troubleshooting hadoop sqoop big data aws

Work Mode

Work from Office

Job Type

Full Time

Job Description

We are looking for a skilled PySpark Lead with 5 to 8 years of experience, based in Bangalore. The ideal candidate will have expertise in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.

Roles and Responsibility

Design and develop scalable data pipelines using PySpark and other relevant technologies.
Collaborate with cross-functional teams to identify and prioritize project requirements.
Develop and maintain large-scale data systems, ensuring high performance and reliability.
Troubleshoot and resolve complex technical issues related to data processing and analysis.
Implement automated testing and validation frameworks to ensure data quality and integrity.
Work closely with stakeholders to understand business needs and provide technical solutions.

Job Requirements

Strong proficiency in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.
Experience with big data processing and analysis is highly desirable.
Strong problem-solving skills and attention to detail are essential.
Excellent communication and collaboration skills are required.
Ability to work in a fast-paced environment and meet deadlines.
A Bachelor's or Master's degree in Computer Science or a related field (BE, B.Tech, ME, M.Tech) is preferred.
Notice period: 0 to 30 days.
Specific domain / industry: N/A.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.