Job
Description
As a Lead Data Engineer, your role involves architecting and guiding the design and implementation of a global data handling and synchronization platform. You will provide technical leadership to a small team of data engineers and advise on master data management (MDM) best practices to ensure compliance with global data residency and privacy requirements such as GDPR. **Key Responsibilities:** - Lead the design and implementation of data pipelines for global and regional data synchronization using technologies like Azure SQL, Data Lake, Data Factory, and PySpark. - Define the data architecture and drive MDM strategies to maintain consistent, high-quality data across different regions. - Develop and enforce standards for secure handling of Personally Identifiable Information (PII) and non-PII data to ensure compliance with GDPR and other regulations. Guide and mentor data engineers, reviewing their code and solutions to ensure adherence to best practices. - Collaborate with software architects, DevOps teams, and business stakeholders to integrate data flows with application logic and deployment pipelines. - Oversee monitoring, alerting, and documentation for data processes within existing frameworks. - Provide technical guidance on data partitioning, replication, schema evolution, and data governance. **Qualifications Required:** - 8+ years of experience as a Data Engineer, with at least 2 years in a technical leadership or lead role. - Proficiency in Pyspark optimization and expertise in the Microsoft Azure data stack (Azure SQL, Data Lake, Data Factory, PySpark) and distributed data architectures. - Proven experience with Master Data Management (MDM) and cross-region data synchronization. - Familiarity with data privacy, security, and compliance standards such as GDPR. - Proficiency in Python, SQL, and ETL tools. - Strong leadership, problem-solving, and communication skills. **Preferred Qualifications:** - Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures. - Knowledge of Continuous Integration/Continuous Deployment (CI/CD) and infrastructure-as-code tools like Azure DevOps, ARM/Bicep, and Terraform. As a Lead Data Engineer, your role involves architecting and guiding the design and implementation of a global data handling and synchronization platform. You will provide technical leadership to a small team of data engineers and advise on master data management (MDM) best practices to ensure compliance with global data residency and privacy requirements such as GDPR. **Key Responsibilities:** - Lead the design and implementation of data pipelines for global and regional data synchronization using technologies like Azure SQL, Data Lake, Data Factory, and PySpark. - Define the data architecture and drive MDM strategies to maintain consistent, high-quality data across different regions. - Develop and enforce standards for secure handling of Personally Identifiable Information (PII) and non-PII data to ensure compliance with GDPR and other regulations. Guide and mentor data engineers, reviewing their code and solutions to ensure adherence to best practices. - Collaborate with software architects, DevOps teams, and business stakeholders to integrate data flows with application logic and deployment pipelines. - Oversee monitoring, alerting, and documentation for data processes within existing frameworks. - Provide technical guidance on data partitioning, replication, schema evolution, and data governance. **Qualifications Required:** - 8+ years of experience as a Data Engineer, with at least 2 years in a technical leadership or lead role. - Proficiency in Pyspark optimization and expertise in the Microsoft Azure data stack (Azure SQL, Data Lake, Data Factory, PySpark) and distributed data architectures. - Proven experience with Master Data Management (MDM) and cross-region data synchronization. - Familiarity with data privacy, security, and compliance standards such as GDPR. - Proficiency in Python, SQL, and ETL tools. - Strong leadership, problem-solving, and communication skills. **Preferred Qualifications:** - Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures. - Knowledge of Continuous Integration/Continuous Deployment (CI/CD) and infrastructure-as-code tools like Azure DevOps, ARM/Bicep, and Terraform.
