Associate Director Data Engineering

As an accomplished Associate Director of Data Engineering, you will play a crucial role in leading and scaling next-generation data platforms. Your deep technical expertise and strategic vision will drive the design, development, and operation of high-performance, high-velocity real-time platforms, pipelines, and AI-enabled systems. **Key Responsibilities:** - **Leadership & Mentorship:** Lead and mentor a team of data engineers, fostering technical excellence, innovation, and collaboration. - **Architecture & Roadmap:** Own the technical vision and roadmap for the data engineering function, ensuring alignment with business and AI-driven priorities. - **Platform Engineering:** Design, develop, and maintain real-time and batch data platforms with a focus on scalability, performance, and cost-efficiency. - **ETL/ELT & Data Warehousing:** Architect ETL/ELT processes, streaming pipelines, and modern data lake/warehouse solutions (e.g., Redshift, Snowflake). - **Streaming & Big Data:** Build and optimize Spark and Structured Spark Streaming pipelines to support large-scale, low-latency data use cases. - **AI/ML & GenAI Enablement:** Partner with Data Science and AI teams to enable advanced AI/ML, GenAI, and MLOps use cases; experience with agentic systems and LLM workflows is a plus. - **Modern Data Stack:** Drive adoption of modern orchestration and streaming tools (e.g., Airflow, Kafka) and implement best practices in engineering and operations. - **Governance & Trust:** Champion data observability, lineage, quality, and privacy initiatives to ensure trusted data across the organization. - **Reliability & Security:** Ensure platforms meet enterprise standards of reliability, scalability, security, and compliance across real-time and batch workloads. **Qualification Required:** - 10+ years of experience in data engineering with proven leadership in scaling enterprise-grade platforms. - Strong hands-on expertise with Spark, Structured Spark Streaming, and modern data architectures. - Experience architecting real-time streaming pipelines and cloud-based data warehouses/lakes. - Deep understanding of data modeling, governance, security, and observability practices. - Exposure to AI/ML, Generative AI, and MLOps; familiarity with vector databases, RAG frameworks, and intelligent agents is an advantage. - Solid grasp of data architecture, system design, and performance optimization at scale. - Strong knowledge of modern data stack tools (Airflow, Kafka, etc.). - Demonstrated ability to lead cross-functional teams, define technical strategy, and deliver business impact. - Experience working in an agile development environment and managing sprint-based delivery cycles.,