Senior Data Engineer - Dataiku

Key Responsibilities:

• Dataiku Leadership:

• Drive data engineering projects with a strong emphasis on utilizing Dataiku for data preparation, analysis, visualization, and building end-to-end data solutions.

• Data Pipeline Development:

• Design, develop, and optimize robust, scalable data pipelines supporting business intelligence and advanced analytics initiatives. Build and maintain ETL/ELT processes across diverse data sources.

• Data Modeling & Architecture:

• Apply data modeling best practices (dimensional, star schema, etc.) to create scalable and efficient database designs with a focus on performance and integrity.

• ETL/ELT Expertise:

• Build and manage ETL pipelines using tools such as Dataiku, Apache Airflow, Talend, or SSIS to ensure efficient, reliable, and high-quality data flow.

• Gen AI Integration:

• Explore and integrate Generative AI solutions leveraging LLM Mesh or similar frameworks to enhance platform capabilities.

• Programming & Scripting:

• Use Python and SQL extensively for data manipulation, automation, and the development of custom data applications.

• Cloud Platform Deployment:

• Deploy and manage data solutions on cloud platforms such as AWS or Azure, using services like S3, EC2, Azure Data Lake, or Synapse Analytics.

• Data Quality & Governance:

• Ensure high-quality, well-governed data through integration, monitoring, and adherence to governance standards and best practices.

• Collaboration & Mentorship:

• Work closely with data scientists, analysts, and business teams to gather requirements and deliver impactful data products. Mentor junior team members as needed.

• Performance Optimization:

• Monitor and enhance data pipeline and system performance to meet evolving business and technical needs.

Required Skills & Experience:

• Demonstrated expertise in using Dataiku for data pipeline development and visualization
• Strong hands-on experience with data modeling techniques and best practices
• Proficient in building and managing ETL/ELT processes using tools like Dataiku, Apache Airflow, Talend, or SSIS
• Familiarity with LLM Mesh or related Gen AI integration frameworks
• Strong programming skills in Python and SQL for scripting, analysis, and automation
• Hands-on experience deploying data solutions on AWS or Azure cloud platforms
• Solid understanding of Generative AI concepts and their application in data workflows
• Excellent problem-solving and analytical thinking skills
• Effective communication and collaboration abilities

Bonus Points (Nice to Have):

• Experience with big data technologies such as Spark, Hadoop, or Snowflake
• Knowledge of data governance and security frameworks
• Experience with MLOps practices and tooling
• Contributions to open-source projects in the data or AI domain

Education:

• Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related quantitative field