Sr. Software Engineer - Azure Databricks

Role Overview: YASH Technologies is seeking a Senior Databricks Developer to join their team in Pune or Bangalore. As a part of the team, you will be responsible for designing, developing, and optimizing data pipelines using Databricks, PySpark, Delta Lake, and other Azure components. You will work on new pipeline development, data modeling, production code optimization, and troubleshooting data issues to ensure scalability, performance, and efficiency. Key Responsibilities: - Design, develop, and optimize data pipelines using Databricks, PySpark, Delta Lake, and related Azure components. - Work on pipeline development, data modeling, and production code optimization to ensure scalability and performance. - Build new pipelines from scratch, enhance existing ones, and maintain them effectively. - Apply strong understanding of ETL design principles, data modeling concepts, and data governance standards. - Collaborate within a scrum team, taking ownership of assigned stories and delivering high-quality, production-ready code independently. - Demonstrate proficiency in PySpark, SQL, Delta Lake, Unity Catalog, and Databricks Workflows with a solid understanding of logic and data flow. - Utilize Azure environments and tools like ADLS, ADF, and Synapse as applicable. - Contribute inputs to architecture and design discussions, primarily focusing on hands-on development. - Troubleshoot data issues, perform root-cause analysis, and optimize performance for existing pipelines. Qualification Required: - 3-5 years of experience in data engineering, preferably with Azure Databricks. - Strong technical expertise in PySpark, SQL, and Delta Lake. - Familiarity with Unity Catalog, data governance, and DevOps practices (Git, CI/CD). - Ability to work independently as well as collaboratively within an agile delivery team. - Excellent problem-solving, debugging, and communication skills.,