Data Engineering lead

Job Description

Job Title: Senior Data Engineering Lead Location: [ Remote] Experience: 7+ Years Employment Type: [Contract] Company: Kooe Private Limited Position Overview: Kooe Private Limited is seeking an experienced Senior Data Engineering Lead to provide technical leadership and hands-on expertise in data engineering , focusing on the acquisition, transformation, and distribution of data. You will guide a dynamic team of data engineers, architects, and database administrators, working across multiple projects and squads in an agile, fast-paced environment. This role offers the opportunity to drive the adoption and evolution of frameworks and tools, enhancing the scalability and cost-effectiveness of the overall data architecture. You’ll collaborate closely with cross-functional teams and play a key role in designing, implementing, and optimizing data pipelines. Key Responsibilities: Technical Leadership: Provide guidance and mentorship to a team of data engineers, architects, and database administrators, fostering collaboration and growth. Data Pipeline Design & Management: Lead the design, implementation, and management of data pipelines using the Treasure Data platform. Workflow Development: Develop, test, and maintain custom workflows, ensuring they meet business needs and are optimized for performance. Monitoring & Troubleshooting: Monitor and troubleshoot Airflow workflows to ensure high availability, reliability, and performance. Collaboration: Work closely with data engineers and data scientists to understand data requirements and translate them into scalable workflows. Data Integrity: Ensure data quality and integrity across all workflows, meeting operational standards. Documentation: Document all processes, workflows, and configurations for transparency and knowledge sharing. Cloud Interaction: Regularly interact with Amazon Redshift to ensure proper functioning of data pipelines and workflow execution. Required Expertise: Data Engineering: Excellent understanding of enterprise data warehousing principles, ETL/ELT processes, and related tools and technologies. Treasure Data Experience: Proven experience with Treasure Data , including designing and maintaining complex workflows and data pipelines. SQL & Presto SQL: Strong knowledge of SQL programming and Presto SQL optimization techniques. Airflow & Workflow Orchestration: Experience with Apache Airflow or similar orchestration tools, including workflow monitoring and troubleshooting. Python & Relational Databases: Experience with Python for data transformation and managing relational databases. Cloud & AWS: Familiarity with cloud platforms (e.g., AWS ) and their respective data services. Version Control & Collaboration: Experience using Git for version control and excellent collaboration skills to work with both business and technical teams. Your Qualities: Customer-Centric: Inherently curious and empathetic towards customer success. Problem-Solver: Obsessive about solving customer problems. Collaborative: Eager to collaborate across teams and share knowledge. Mission-Focused: Excited about the mission and milestones, not titles or hierarchies. Adaptable: Willing to experiment, learn, and innovate in a fast-paced environment. Preferred Qualifications: Mentoring & Coaching: Proven experience mentoring and coaching junior engineers and fostering a learning environment. Leadership Communication: Comfortable communicating with senior leadership to provide updates and discuss technical challenges. Show more Show less