Job
Description
Job Summary We are seeking a highly skilled and experienced Databricks Developer to join our data engineering team. The ideal candidate will have over 4 years of experience working with Databricks on Azure, along with a strong background in data pipelines, and performance optimization. The candidate will be responsible for developing scalable data processing solutions, ensuring data quality, and enabling advanced analytics initiatives. Key Responsibilities Design, develop, and optimize data pipelines and ETL processes using Azure Databricks (PySpark /Spark SQL). Collaborate with data architects, analysts, and other developers to deliver data solutions aligned with business requirements. Perform data wrangling, cleansing, transformation, and aggregation from multiple sources. Implement and maintain data lake and data warehouse solutions using Azure services (ADLS, Synapse, Delta Lake). Monitor pipeline performance, troubleshoot issues, and ensure data integrity and reliability. Use Delta Lake for building robust and scalable data architectures. Develop and maintain CI/CD pipelines for Databricks workflows and jobs. Participate in code reviews, unit testing, and documentation of data processes. Required Skills & Experience 4+ years of hands-on experience with Databricks on Azure. Strong expertise in PySpark, Spark SQL, and Delta Lake. Solid understanding of Azure Data Services: Azure Data Lake Storage (ADLS), Azure Data Factory (ADF) Proficiency in Python for data processing tasks. Experience with data ingestion from various sources (on-prem, cloud, APIs). Knowledge of data modeling, data governance, and performance tuning. Familiarity with CI/CD tools (Azure DevOps, Git) and job orchestration in Databricks. Strong problem-solving skills and ability to work independently or as part of a team. Show more Show less
