Job
Description
Technical Leadership: Architect and design scalable, efficient, and reliable data pipelines and platforms and Lead the adoption and best practices for Date Engineering tools. Ensure high standards for data quality, reliability, governance, and security. Drive the teams technical roadmap, identifying areas for innovation and optimization People & Project Leadership: Lead and mentor a team of data engineers, fostering growth, ownership, and excellence. Manage sprint planning, code reviews, and technical delivery in an agile environment. Collaborate with Product, Analytics, and Data Science teams to align on data needs and priorities. Own end-to-end delivery of projects, ensuring scalability, performance, and sustainability. Requirements 8\u201310 years of experience in data engineering or backend engineering with significant data pipeline responsibilities. Expertise in Python for data engineering tasks, including data transformation and orchestration. Hands-on experience with Kafka for real-time streaming and event-driven architectures. Proven experience with Airflow or equivalent scheduling/orchestration tools. Deep knowledge of Databricks and Apache Spark for distributed data processing. Proficient in SQL, with strong understanding of relational and distributed databases (e.g., Snowflake, Redshift, BigQuery). Experience in designing data architectures on cloud platforms (AWS, Azure, or GCP). Solid understanding of data modeling, data warehousing, and data governance. Experience in managing cost optimization and performance in cloud-based data platforms. Familiarity with MLOps or data products serving ML pipelines.
