Data Engineer ( AWS & Databricks)

Data Engineer

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL processes using Databricks and AWS services
  
• Hands-on development using

  Python

  ,

  Scala

  , and

  SQL

• Orchestrate workflows and manage job scheduling
  
• Optimize and monitor

  Databricks jobs

  for performance and reliability
• Work with large-scale datasets and implement best practices for data ingestion and processing
  
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
  
• Root cause analysis and performance tuning
  
• Document data engineering processes and developer guides
  
• Work in an Agile environment using tools like

  JIRA

• Batch ETL experience is essential; real-time pipeline experience is a plus
  
• Exposure to

  Google Cloud Platform (GCP)

  is a plus

Required Skills & Experience:

• 3+ years of

  hands-on experience with Databricks

• 5+ years of

  experience with AWS data engineering services

• Strong proficiency in

  Python

  and

  Scala

• Experience with

  ETL development

  and

  SQL

• Knowledge of

  workflow orchestration tools

• Experience with

  monitoring

  ,

  tuning

  , and

  debugging Databricks jobs

• Ability to perform root cause analysis and improve data quality/processes
  
• Experience working in Agile teams and using

  JIRA