Job
Description
You are a Lead Data Engineer responsible for architecting and guiding the design and implementation of a global data handling and synchronization platform. In this role, you will provide technical leadership to a small team of data engineers and advise on master data management (MDM) best practices to ensure compliance with global data residency and privacy requirements like GDPR. **Key Responsibilities:** - Lead the design and implementation of data pipelines for global and regional data synchronization using Azure SQL, Data Lake, Data Factory, PySpark, etc. - Define the data architecture and drive MDM strategies to maintain consistent, high-quality data across regions. - Develop and enforce standards for secure handling of PII and non-PII data, ensuring compliance with GDPR and other regulations. - Guide and mentor data engineers, review code and solutions to ensure best practices are followed. - Collaborate with software architects, DevOps, and business stakeholders to integrate data flows with application logic and deployment pipelines. - Oversee monitoring, alerting, and documentation for data processes within the existing frameworks. - Provide technical guidance on data partitioning, replication, schema evolution, and data governance. **Qualifications Required:** - 8+ years of experience as a Data Engineer, with at least 2 years in a technical leadership or lead role. - Experience with Pyspark optimization. - Strong expertise in Microsoft Azure data stack (Azure SQL, Data Lake, Data Factory, PySpark) and distributed data architectures. - Proven experience with Master Data Management (MDM) and cross-region data synchronization. - Familiarity with data privacy, security, and compliance (GDPR, etc.). - Proficiency in Python, SQL, and ETL tools. - Strong leadership, problem-solving, and communication skills. **Preferred Qualifications:** - Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures. - Knowledge of CI/CD and infrastructure-as-code (Azure DevOps, ARM/Bicep, Terraform).,
