PySpark Developer-12 Months Contract

PySpark Developer

PySpark Developers

• Experience: 4-10 years (Mandatory)

• Qualification: BE / B. Tech

• Location: Pune

• Contract Duration: 12 Months

• Salary: Up to 24 LPA

Key Responsibilities

• Design, develop, and optimize

  ETL/ELT pipelines using PySpark.

• Ingest, clean, transform, and process structured/semi-structured/unstructured data.
• Work with large-scale datasets on distributed computing platforms (Hadoop, Spark, Databricks).
• Integrate data from multiple sources, including Delta Lake, Data Lake, RDBMS, APIs.
• Ensure high-performance, scalability, and reliability of data pipelines.
• Collaborate with data engineers, analysts, and data scientists to deliver business-ready datasets.
• Implement monitoring, logging, and error-handling for data pipelines.
• Contribute to data quality frameworks and best practices.

Required Skills & Experience

• Bachelors degree in computer science, computer engineering or similar.
• Excellent communication skills.
• 4-10 years of experience in data engineering with strong hands-on expertise in

  PySpark

  .
• Proficiency in

  Python programming for data manipulation and pipeline development.

• Strong knowledge of

  Spark SQL, DataFrames, and RDDs.

• Experience with big data ecosystems Hadoop, Hive, or Databricks.
• Good understanding of SQL and relational databases (PostgreSQL, MySQL, Oracle).

• Familiarity with cloud platforms (AWS, Azure, GCP) and services (S3, ADLS, BigQuery).

• Experience with version control (Git) and CI/CD tools.

• Strong debugging, performance optimization, and troubleshooting skills.
• Experience with

  containerization (Docker, Kubernetes)

  .

Please share the following details along with the most updated resume to geeta.negi@compunnel.com if you are interested in the opportunity:

• Total Experience
• Relevant experience
• Current CTC
• Expected CTC
• Notice Period (Last working day if you are serving the notice period)
• Current Location
• SKILL 1 RATING OUT OF 5
• SKILL 2 RATING OUT OF 5
• SKILL 3 RATING OUT OF 5

(Mention the skill)