Data Engineering Manager

Job Description

Team Leadership & Management: Lead, mentor, and develop a team of data engineers. Foster a collaborative and innovative team environment. Conduct performance evaluations and support professional growth. Data Engineering & Architecture: Architect and implement scalable data solutions using Azure Databricks and Snowflake. Design, build, and maintain robust data pipelines with a solid understanding of ETL/ELT processes. Optimize data workflows for performance, reliability, and scalability. Solution Architecture: Architect comprehensive data solutions tailored to business needs. Lead the design and implementation of data warehouses, ensuring alignment with organizational objectives. Collaborate with stakeholders to define and refine data requirements and solutions. AI Integration: Work alongside data scientists and AI specialists to integrate machine learning models into data pipelines. Implement AI-driven solutions to enhance data processing and analytics capabilities. Engineering Project Management: Manage data engineering projects from inception to completion, ensuring timely delivery and adherence to project goals. Utilize project management methodologies to track progress, allocate resources, and mitigate risks. Coordinate with stakeholders to define project requirements and objectives. Infrastructure as Code & Automation: Implement and manage infrastructure using Terraform. Develop and maintain CI/CD pipelines to automate deployments and ensure continuous integration and delivery of data solutions. Quality Assurance & Best Practices: Establish and enforce data engineering best practices and standards. Ensure data quality, security, and compliance across all data initiatives. Conduct code reviews and ensure adherence to coding standards. Collaboration & Communication: Work closely with data analysts, business intelligence teams, and other stakeholders to understand data needs and deliver solutions. Communicate technical concepts and project statuses effectively to non-technical stakeholders. Undergraduate degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent experience. Experience: 8+ years of overall experience in data engineering. 2+ years of experience in Managing Data Engineering Teams. Proven experience with Azure Databricks and Snowflake. Solid experience in designing data solutions for data warehouses. Hands-on experience with Terraform for infrastructure as code. Strong knowledge of CI/CD tools and practices. Experience integrating AI and machine learning models into data pipelines. Technical Skills: Proficiency in Spark, Scala, Python, SQL, and Databricks. Proven Unix scripting and SQL skills. Strong understanding of SQL and database management. Familiarity with data warehousing, ETL/ELT processes, and big data technologies. Experience with cloud platforms, preferably Microsoft Azure. Project Management: Proven ability to manage multiple projects simultaneously. Familiarity with project management tools (e.g., Jira, Trello, Asana,Rally). Strong organizational and time-management skills. Soft Skills: Excellent leadership and team management abilities. Ability to work collaboratively in a fast-paced environment. Proven ability to perform with minimal supervision. Solid work prioritization, planning, and organizational skills. Leadership qualities including being proactive, thoughtful, thorough, decisive, and flexible. Role & responsibilities Preferred candidate profile