Data Engineer - OPS

Location:

Experience:

Role Overview:

We are looking for a highly skilled Manager/Tech Lead, to lead and manage a team of 15 Data Engineers. This position requires a strong technical background, exceptional team management skills, and expertise in data engineering tools and techniques. The role involves end-to-end ownership of data pipelines, integration, and delivery, with a focus on cost and project optimization.

Key Responsibilities:

1. Team Management:

• Manage and mentor a team of 15 Data Engineers, ensuring professional growth and skill enhancement.
• Allocate tasks effectively, monitor performance, and address any issues proactively.

1. Technical Leadership:

• Drive the development and optimization of data pipelines using Databricks, Data Lakes, Azure Data Factory, Delta Live Tables, Python, SQL, and PySpark.
• Implement robust data integration, data quality, and data observability frameworks.

1. Project & Delivery Management:

• Collaborate daily with clients, production teams, and delivery teams to align on goals and resolve challenges.
• Ensure timely and efficient project delivery, maintaining high standards of quality and performance.

1. Cost & Process Optimization:

• Identify opportunities for cost optimization in existing projects.
• Streamline workflows to improve efficiency and reduce redundancies.

1. Stakeholder Management:

• Act as a liaison between technical teams and stakeholders, ensuring clear communication of requirements and deliverables.
• Manage expectations and provide updates on project progress and risks.

1. Tech Migration & Innovation:

• Lead technology migration initiatives to ensure the adoption of modern tools and platforms.
• Stay updated with industry trends and recommend best practices for data engineering.

Skills & Qualifications:

• Mandatory Technical Skills:

• Databricks, Data Lakes, Azure Data Factory, Delta Live Tables, Python, SQL, PySpark.
• Expertise in building and managing scalable data pipelines and integrations.
• Strong understanding of data quality and observability frameworks.

• Management Skills:

• Proven experience in leading and managing large teams (15+ members).
• Strong project management and delivery skills.
• Excellent stakeholder management and communication abilities.

• Other Requirements:

• Experience in driving delivery for tech teams and managing daily client interactions.
• Knowledge of cost optimization and tech migration best practices.