Data Engineer

Location:

Work Mode:

Experience Required:

Notice Period:

Databricks Engineer

Key Responsibilities

• Develop, optimize, and maintain

  PySpark

  scripts and data pipelines in

  Databricks

  .
• Work with

  AWS cloud services

  such as S3, Lambda, EMR, Glue, IAM, and CloudWatch.
• Perform data transformations, data ingestion, and ETL operations.
  
• Design and implement scalable data solutions on Databricks.
  
• Collaborate with cross-functional teams to understand data requirements.
  
• Ensure data quality, performance tuning, and debugging of data workflows.
  
• Manage cluster configurations and monitor performance within Databricks.
  
• Follow best practices for coding, version control, and CI/CD.
  

Required Skills

• Mandatory:

• Databricks
  
• Python
  
• PySpark
  
• AWS
  
• Strong understanding of distributed computing frameworks.
  
• Experience with Spark optimization techniques.
  
• Hands-on experience with data lakes, data warehouses, and ETL concepts.
  
• Good knowledge of SQL.
  
• Familiarity with Git, CI/CD pipelines is a plus.
  

Good to Have

• Experience with Delta Lake.
  
• Knowledge of Airflow or other orchestration tools.
  
• Exposure to DevOps practices.