Data Engineering Manager

Work Type:

Location-

Compensation:

Job Overview

Key Responsibilities

Data Integration & Pipeline Development

• Develop and maintain end-to-end ETL/ELT workflows integrating data from SQL Server, SAP HANA, Oracle, AWS Redshift, cloud storage, and APIs.
• Design scalable batch and real-time data pipelines with strong error handling and performance optimization.
• Implement workflow orchestration patterns supporting enterprise-grade data operations.
  

Dataiku DSS Administration & Automation

• Manage Dataiku environments including licenses, credentials, projects, and automation nodes.
• Automate pipelines, API integrations, and recurring tasks using Python, SQL, and Dataiku recipes/plugins.
• Build reusable components for auditing, user activity tracking, and operational reporting.
  

AI & Analytics Enablement

• Prepare, cleanse, and transform datasets for AI/ML model development in collaboration with data scientists.
• Implement data validation, monitoring, and reconciliation frameworks to ensure accuracy and reduce manual effort.
• Build interactive dashboards and KPI visualizations to support business decision-making.
  

Must-Have Qualifications

• 5–8 years of experience in data integration, data engineering, or AI/analytics development.
• Strong hands-on experience with Dataiku DSS (development + administration).
• Expertise in SQL and Python (Pandas, PySpark) for data transformation and automation.
• Experience with Snowflake, AWS (S3, Redshift), and cloud-based data pipelines.
• Solid understanding of ETL design patterns, batch scheduling, and workflow orchestration.
• Working knowledge of API automation, data validation, and reconciliation reporting.
• Experience collaborating with data science and business teams.
• Strong analytical, problem-solving, and documentation skills.
  

Nice-to-Have Qualifications

• Certifications: Dataiku Core Designer / ML Practitioner, AWS Solutions Architect, Azure Fundamentals.
• Exposure to DevOps tooling for data operations.
• Experience in dashboarding/BI tools (Tableau, Power BI, etc.).