Pyspark Lead

Krazy Mantra HR Solutions Pvt. Ltd

6 - 8 years

5 - 9 Lacs

bengaluru

Posted:3 days ago| Platform:

Apply

Skills Required

pyspark sql design process etl data engineering hive continuous integration scala ci/cd emr java data modeling spark hadoop big data snowflake python airflow data processing pipeline data bricks data quality kafka troubleshooting sqoop illustrator

Work Mode

Work from Office

Job Type

Full Time

Job Description

We are looking for a skilled PySpark Lead with 6-8 years of experience to join our team in Bangalore. The ideal candidate will have expertise in SQL, PySpark (or Spark with Scala), and Design Aspects - ETL/ Data Engineering Pipeline.

Roles and Responsibility

Design and develop scalable data pipelines using PySpark and other relevant technologies.
Collaborate with cross-functional teams to identify and prioritize project requirements.
Develop and maintain large-scale data systems, ensuring high performance and reliability.
Troubleshoot and resolve complex technical issues related to data processing and analysis.
Implement automated testing and validation frameworks to ensure data quality and integrity.
Work closely with stakeholders to understand business needs and provide technical solutions.

Job Requirements

Strong proficiency in SQL, PySpark (or Spark with Scala), and Design Aspects - ETL/ Data Engineering Pipeline.
Experience with Cloud understanding, CI / CD, Data Modelling, Airflow, Kafka, EMR/ Databricks, Snowflake, Hive, and other relevant technologies.
Excellent problem-solving skills and the ability to analyze complex technical issues.
Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.
Ability to design and implement scalable data systems with high performance and reliability.
Education: Be, B.TECH, ME, M.TECH.
Notice Period: Immediate joiners.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.