Posted:Just now|
Platform:
Work from Office
Full Time
Job Summary
We are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.
Key Responsibilities
Develop and optimize scalable PySpark applications on Databricks.
Work with AWS services (S3, EMy, Lambda, Dlue) for cloud-native data processing.
Integrate streaming and batch data sources, especially using Kafka.
Tune Spark jobs for performance, memory, and compute efficiency.
Collaborate with DevOps, product, and analytics teams on delivery and deployment.
Ensure data governance, lineage, and quality compliance across all pipelines.
Required Skills
3 6 years of hands-on development in PySpark.
Experience with Databricks and performance tuning using Spark UI.
Strong understanding of AWS services, Kafka, and distributed data processing.
Proficient in partitioning, caching, join optimization, and resource configuration.
Familiarity with data formats like Parquet, Avro, and OyC.
Exposure to orchestration tools (Airflow, Databricks Workflows).
Scala experience is a strong plus.
Algoleap Technologies
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
Practice Java coding challenges to boost your skills
Start Practicing Java Now
hyderabad
1.5 - 2.0 Lacs P.A.
hyderabad, telangana, india
Salary: Not disclosed
bengaluru
30.0 - 35.0 Lacs P.A.
pune, maharashtra, india
Salary: Not disclosed
Chennai, Tamil Nadu, India
4.0 - 7.0 Lacs P.A.
Gurugram, Chennai, Bengaluru
20.0 - 25.0 Lacs P.A.
Bengaluru
30.0 - 35.0 Lacs P.A.
Kolkata
4.0 - 8.0 Lacs P.A.
Chennai
8.0 - 12.0 Lacs P.A.
Gurugram
8.0 - 12.0 Lacs P.A.
