Mgr - Data Engineer ( DBT / Matillion )

Job Description

Data Solution Design and Development: o Architect, design, and implement end-to-end data pipelines and systems that handle large-scale, complex datasets. o Ensure optimal system architecture for performance, scalability, and reliability. o Evaluate and integrate new technologies to enhance existing solutions. o Implement best practices in ETL/ELT processes, data integration, and data warehousing. 2. Project Leadership and Delivery: o Lead technical project execution, ensuring timelines and deliverables are met with high quality. o Collaborate with cross-functional teams to align business goals with technical solutions. o Act as the primary point of contact for clients, translating business requirements into actionable technical strategies. 3. Team Leadership and Development: o Manage, mentor, and grow a team of 5 to 7 data engineers. o Conduct code reviews, validations, and provide feedback to ensure adherence to technical standards. o Provide technical guidance and foster an environment of continuous learning, innovation, and collaboration. 4. Optimization and Performance Tuning: o Analyze and optimize existing data workflows for performance and cost-efficiency. o Troubleshoot and resolve complex technical issues within data systems. 5. Adaptability and Innovation: o Embrace a consulting mindset with the ability to quickly learn and adopt new tools, technologies, and frameworks. o Identify opportunities for innovation and implement cutting-edge technologies in data engineering. o Exhibit a "figure it out" attitude, taking ownership and accountability for challenges and solutions. 6. Client Collaboration: o Engage with stakeholders to understand requirements and ensure alignment throughout the project lifecycle. o Present technical concepts and designs to both technical and non-technical audiences. o Communicate effectively with stakeholders to ensure alignment on project goals, timelines, and deliverables. 7. Learning and Adaptability: o Stay updated with emerging data technologies, frameworks, and tools. o Actively explore and integrate new technologies to improve existing workflows and solutions. 8. Internal Initiatives and Eminence Building: o Drive internal initiatives to improve processes, frameworks, and methodologies. o Contribute to the organizations eminence by developing thought leadership, sharing best practices, and participating in knowledge-sharing activities. Qualifications Education: o Bachelor’s or Master’s degree in computer science, Data Engineering, or a related field. o Certifications in cloud platforms such as Snowflake Snowpro, Azure Data Engineer is a plus. Experience: o 7 to 10 years of experience in data engineering with hands-on expertise in data pipeline development, architecture, and system optimization. o Proven track record in leading data engineering teams and managing end-to-end project delivery. Technical Skills: o Expertise in programming languages such as Python, Scala, or Java. o Proficiency in designing and delivering data pipelines in Cloud Data Warehouses (e.g., Snowflake, Redshift), using various ETL/ELT tools such as Matillion, dbt, Striim, etc. o Solid understanding of database systems (relational and NoSQL) and data modeling techniques. o Hands-on experience of 2+ years in designing and developing data integration solutions using Matillion and/or dbt. o Strong knowledge of data engineering and integration frameworks. o Expertise in architecting data solutions. o Successfully implemented at least two end-to-end projects with multiple transformation layers. o Good grasp of coding standards, with the ability to define standards and testing strategies for projects. o Proficiency in working with cloud platforms (AWS, Azure, GCP) and associated data services. o Enthusiastic about working in Agile methodology. o Possess a comprehensive understanding of the DevOps process including GitHub integration and CI/CD pipelines. o Experience working with containerization (Docker), and orchestration tools (such as Airflow, Control-M). Soft Skills: o Exceptional problem-solving and analytical skills. o Strong communication and interpersonal skills to manage client relationships and team dynamics. o Ability to thrive in a consulting environment, quickly adapting to new challenges and domains. o