Databricks Developer

As a Databricks Developer at PalTech, you will play a crucial role in designing, developing, and maintaining highly scalable and efficient data pipelines using Databricks, PySpark, and other related technologies. Your primary responsibility will involve processing large scale datasets and collaborating with cross-functional teams to design and implement data engineering and analytics solutions. Key Responsibilities: - Design, develop, and maintain highly scalable and efficient data pipelines using Databricks, PySpark, and related technologies - Collaborate with cross-functional teams to design and implement data engineering and analytics solutions - Optimize Databricks notebooks and implement solutions for data quality and reconciliation requirements - Utilize Unity Catalog and Metastore, delta lake, DLT pipelines, and Databricks SQL for data processing - Ensure high data quality and compliance by following data governance frameworks and best practices - Apply data modelling techniques and work proficiently in PySpark or SparkSQL Qualifications Required: - Expertise in using Unity Catalog and Metastore - Extensive experience in optimizing Databricks notebooks - Expertise in delta lake and DLT pipelines - Experience in using Databricks SQL - Strong understanding of data governance frameworks, policies, and best practices - Proficiency in PySpark or SparkSQL Please note that the above job description is for a full-time on-site role located in Hyderabad.,