Data Engineer

Job Title: Data Engineer

Experience Required:

Location:

Data Engineer

Key Responsibilities:

• Analyze and understand source data structures, formats, and business requirements.
• Design and develop efficient

  data pipelines

  on

  Databricks

  using

  PySpark

  to extract, transform, and load (ETL) data into

  target Delta tables

  .
• Optimize and automate data workflows for improved performance and scalability.
• Ensure

  data accuracy, consistency, and integrity

  across systems.
• Collaborate with cross-functional teams including data analysts, data scientists, and business stakeholders to support data-driven initiatives.
• Implement

  best coding practices

  and ensure the quality and maintainability of the data engineering solutions.
• Monitor and troubleshoot production data issues, ensuring timely resolution.

Required Skills & Qualifications:

• Minimum

  4 years of hands-on experience

  as a Data Engineer.
• Strong proficiency in

  Python

  programming and

  PySpark

  .
• Experience with

  Databricks

  and working with

  Delta Lake tables

  .
• Solid understanding of

  data modeling

  ,

  ETL processes

  , and

  data warehousing concepts

  .
• Experience in analyzing and working with large datasets from multiple data sources.
• Good understanding of

  SQL

  and

  performance optimization

  techniques.
• Strong analytical and problem-solving skills with attention to detail.
• Excellent communication and collaboration abilities.

Nice to Have:

• Experience with

  Azure Data Lake

  ,

  AWS

  , or

  GCP

  .
• Familiarity with

  CI/CD

  for data pipelines.
• Knowledge of

  data governance

  and

  data quality frameworks

  .