Job
Description
We are seeking a Senior Data Engineer with a minimum of 5 years of experience in data architecture, data engineering, or related fields. The ideal candidate will have a proven track record of leading data projects, mentoring junior team members, and innovating to solve complex business problems. This role requires hands-on expertise in cloud platforms (Azure or GCP), programming, and designing data pipelines, as well as experience with data governance and CI/CD processes. Required Skills and Qualifications: Minimum of 5 years of experience in data architecture, data engineering, or related fields. Strong proficiency in Python and/or Java programming languages. Hands-on experience with GCP or Azure cloud platforms. Knowledge of data governance tools such as Azure Purview and Collibra. Expertise in designing and implementing batch, streaming, and event-driven data pipelines. In-depth knowledge of data lake and data warehouse architectures, particularly Delta Lake and BigQuery. Experience working in Agile or Scrum environments, with a focus on iterative delivery and adapting to evolving requirements. Excellent problem-solving, analytical, and critical thinking skills. Strong attention to detail with an emphasis on data accuracy and process integrity. Soft Skills: Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders. Collaborative mindset and ability to work effectively within cross-functional teams. Ability to mentor and guide junior team members. Key Responsibilities: Lead the design, development, and optimization of data pipelines using batch, streaming, and event-driven architectures (e.g., Azure Event Hubs, GCP Pub/Sub, Apache Kafka). Architect and implement data lake and data warehouse solutions, including Delta Lake and BigQuery. Implement and manage tools for data cataloging, metadata management, and access control (e.g., Azure Purview, Collibra). Develop, deploy, and manage CI/CD pipelines using tools like Azure DevOps or Jenkins. Strategize and deliver data-driven solutions to solve complex business challenges. Mentor junior team members and provide leadership on data engineering best practices. Collaborate with cross-functional teams and communicate effectively with stakeholders. Ensure compliance with data privacy regulations (e.g., GDPR, CCPA) and related industry standards.
