Data Engineer (Python + Pyspark + AWS)-Hybrid/Mumbai/Contract

Strong Experience in PySpark

o Hands-on expertise in building scalable data pipelines using PySpark.

o Proficiency in using Spark SQL, DataFrame, and RDD APIs to implement complex

business logic.

 Proficient Programming Skills

o Solid coding skills in Python (preferred), with strong fundamentals in data structures,

algorithms, and software engineering principles.

PUBLIC

 Data Pipeline Development

o Proven experience designing, developing, and maintaining batch and streaming data

pipelines.

o Understanding of ETL/ELT processes and best practices for data transformation, data

quality, and performance optimization.

 Knowledge of Modern Data Engineering Ecosystem

o Familiarity with the current data engineering landscape including distributed data

processing, storage systems, and workflow orchestration tools.

o Tools/technologies could include Apache Airflow, dbt, Delta Lake, etc.

 Cloud Data Platform Experience (Preferably AWS)

o Experience working with cloud services such as:

 AWS S3 (data lake)

 AWS Glue / EMR (for Spark workloads)

 AWS Lambda, Step Functions, or similar for orchestration/integration

 Redshift or other cloud data warehouses

 Spark API Expertise for Business Logic Implementation

o Ability to choose and apply the right Spark APIs (DataFrame, Dataset, RDD) for

performance-efficient implementation of business logic at scale.