Databricks Data Engineer

Job Description

We are seeking a skilled Data bricks Data Engineer to join our dynamic team The ideal candidate will have strong expertise in PySpark, Spark SQL, and a comprehensive understanding of the Databricks data engineering ecosystem You will play a critical role in designing and implementing data pipelines, optimizing data workflows, and ensuring data integrity Key Responsibilities: Design, develop, and maintain scalable data pipelines using Databricks Utilize PySpark and Spark SQL to process large datasets and perform data transformations Collaborate with data scientists and analysts to understand data requirements and deliver efficient solutions Optimize existing data workflows for performance and reliability Monitor data pipeline performance and troubleshoot issues as they arise Implement best practices for data governance and security Participate in code reviews and contribute to team knowledge sharing Roles & Responsibilities Key Skills and Qualifications: Proven experience as a Data Engineer with a focus on Databricks Strong proficiency in PySpark and Spark SQL Deep understanding of the Data bricks ecosystem, Databricks SQL Experience with ETL processes, and data warehousing concepts Knowledge of data visualization tools and frameworks is a plus Strong problem-solving skills and the ability to work independently Excellent communication skills and ability to collaborate with cross-functional teams