Data Production Support Engineer

As a Data Platform Support Engineer, your main responsibility will be to ensure the smooth execution of data pipelines and prompt resolution of issues to uphold business continuity. You will play a crucial role in maintaining the health and reliability of data systems by conducting root cause analysis, implementing proactive measures, and minimizing disruptions. Your key accountabilities will include monitoring and managing the performance of Azure Data Factory pipelines, Databricks workflows, and SQL databases to guarantee seamless data processing. You will troubleshoot and resolve production incidents in Azure-based data pipelines, conduct root cause analysis, and implement preventive measures. Additionally, you will oversee and optimize the performance of Databricks notebooks and clusters to support efficient data transformations and analytics. It will be essential for you to ensure the reliability and scalability of data integration workflows by utilizing Azure-native monitoring tools and alerts. Collaborating with development teams to deploy and support new Azure Data Factory pipelines, SQL scripts, and Databricks jobs into production will also be part of your responsibilities. Maintaining compliance with data governance, security, and backup policies across the Azure platform is crucial. Furthermore, you will need to coordinate with stakeholders to provide clear updates on production incidents, resolutions, and performance improvements. Planning and executing disaster recovery and failover strategies for Azure Data Factory, Databricks, and SQL components is essential to ensure business continuity. Documenting operational processes, troubleshooting steps, and best practices for the Azure platform will be necessary to build a comprehensive knowledge base. Your technical skills should include expertise in Azure Data Factory and Databricks, proficiency in SQL, experience in monitoring and alerting using Azure Monitor and Log Analytics, strong incident management skills, knowledge of data governance and security standards, experience in process improvement, and proficiency in documentation. In summary, as a Data Platform Support Engineer, you will play a critical role in maintaining the health and reliability of data systems, ensuring seamless data processing, and implementing proactive measures to minimize disruptions. Your expertise in Azure Data Factory, Databricks, SQL, monitoring and alerting, incident management, data governance, process improvement, and documentation will be key to your success in this role.,