Data Engineering Lead

Job Description

Data Engineering Lead | 7+ Years | Remote | Work Timings: 1:00 PM to 10:00 PM Or 2:00 PM to 11:00 PM | Contract Duration- 3 Months + Job Description: Provide technical leadership on data engineering (acquisition, transformation, distribution), across multiple projects and squads Guide, nurture, and coach a team of data engineers, architects and database administrators working in an agile, fast- paced environment Drive adoption and evolution of frameworks and products based on best- in-class data management technologies, such as ETL/ELT tools, open-source frameworks, workflow management tools, etc. Provide suggestions and recommendation on how to improve the overall data engineering architecture towards a scalable, cost effective ecosystem Your Deliverables: Design, implement, and manage data pipelines using our customer data platform Treasure Data. Develop, test, and maintain custom workflows. Monitor and troubleshoot Airflow workflows to ensure optimal performance and reliability. Collaborate with data engineers and data scientists to understand data requirements and translate them into scalable workflows. Ensure data quality and integrity across all workflows. Document processes, workflows, and configurations. Interact with Redshift on a regular basis ensuring pipelines are functioning as expected Your Qualities: Inherent curiosity and empathy for customer success Obsessive about solving customer problems Think long and Act short Collaborative with your peers, partners and your team. Excited about the mission and milestones not titles and hierarchies Nurture an environment of experimentation and learning Your Expertise: Excellent understanding of enterprise data warehousing principles of ETL/ELT and the tools/technologies used to implement them Proven experience with Treasure Data, including designing and maintaining complex workflows. Strong knowledge of SQL programming and databases Strong knowledge of DigDag files