Azure Databricks Engineer

Azure Databricks Engineer

Key Responsibilities

• Design and build scalable and robust data pipelines

  leveraging

  Azure Databricks, PySpark, and Spark SQL

  for efficient data processing.

• Integrate data from various structured and unstructured data sources

  using Azure data services such as

  Azure Data Factory, Azure Data Lake Storage (ADLS), and Azure Synapse Analytics

  .

• Develop and maintain comprehensive ETL/ELT processes

  for the ingestion, transformation, and storage of large datasets.

• Collaborate effectively with data scientists, data analysts, and other engineers

  to understand requirements and deliver high-quality data products and solutions that drive business value.

• Monitor, troubleshoot, and optimize existing data pipelines

  to ensure peak performance, reliability, and cost-efficiency.
• Ensure

  data quality, governance, and security compliance

  are rigorously upheld in all developed solutions, adhering to industry best practices and regulatory standards.
• Actively

  participate in architectural decisions and cloud data solutioning

  , contributing insights and expertise to shape our data platform strategy.

Required Skills

• Data Engineering Experience

  : 5+ years of experience in data engineering or related fields, demonstrating a strong foundation in data manipulation and processing.

• Azure Databricks & Apache Spark

  : Strong hands-on experience with

  Azure Databricks

  and deep expertise in

  Apache Spark

  for big data processing.

• Programming Proficiency

  : Proficiency in

  Python (PySpark)

  ,

  SQL

  , and proven experience with

  performance tuning techniques

  for data pipelines and queries.

• Azure Data Services

  : Extensive experience with core Azure data services including

  Azure Data Factory, Azure Data Lake Storage (ADLS)

  , and

  Azure Synapse Analytics

  .

• Data Concepts

  : Solid understanding of

  data modeling, data warehousing, and data lakes

  concepts and their application in enterprise environments.

• DevOps & Version Control

  : Familiarity with

  DevOps practices, CI/CD pipelines

  , and

  version control systems (e.g., Git)

  for collaborative development and automated deployments.

Qualification

• Any Graduate or Above

Experience

• 5 to 15 Years

Notice Period

• Only Immediate

Contact: