Manager, Engineering

Role Overview:

We are seeking an experienced and strategic Data Engineering Manager to lead our team. The ideal candidate will have a strong background in big data architecture, cloud-native services, and team leadership. You will be responsible for building and optimizing our data platforms, ensuring data security and privacy, and integrating emerging technologies like AI/LLMs to enhance our analytics capabilities. This role requires a blend of technical expertise, leadership skills, and a forward-thinking mindset to drive our data initiatives.Responsibilities:

• Lead, mentor, and grow a team of talented data engineers. Provide technical guidance and conduct code reviews to ensure best practices.
• Architect and develop scalable and robust data pipelines using a blend of big data frameworks (e.g., Spark, Kafka, Flink) and cloud-native services (AWS) to support security analytics use cases.
• Drive CI/CD best practices, infrastructure automation, and performance tuning across distributed environments.
• Evaluate and pilot the use of AI/LLM technologies in data pipelines for tasks like anomaly detection, metadata enrichment, and automation.
• Ensure data security and privacy compliance across all data platforms and processes.
• Evaluate and integrate LLM-based automation and AI-enhanced observability into engineering workflows.
• Collaborate with data scientists, product managers, and business leaders to understand data needs and deliver solutions that drive business value.
• Oversee the design and implementation of various databases, including NoSQL databases like HBase and Cassandra.

What Were Looking For (Minimum Qualifications)

• 10 to 15 years of experience in big data architecture and engineering, including deep proficiency with the AWS cloud platform, with at least 3-5 years in a leadership or management role.
• Expertise in distributed systems and frameworks such as Apache Spark, Scala, Kafka, and Flink, with experience building production-grade data pipelines.
• Strong programming skills in Java for building scalable data applications.
• Hands-on experience with ETL tools and orchestration systems.
• Solid understanding of data modeling across both relational (PostgreSQL, MySQL) and NoSQL (HBase) databases, as well as performance tuning.
• Demonstrated experience with AWS services including Lambda functions, AWS Step Functions, and CloudFormation (CF).
• Strong interpersonal and communication skills to effectively lead a team and collaborate with a diverse group of stakeholders.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

What Will Make You Stand Out (Preferred Qualifications)

• Experience integrating AI/ML or LLM frameworks (e.g., LangChain, LlamaIndex) into data workflows.
• Experience implementing CI/CD pipelines with Kubernetes, Docker, and Terraform.
• Knowledge of modern data warehousing (e.g., BigQuery, Snowflake) and data governance principles (GDPR, HIPAA).