Data Engineer

Role Overview: As an AI Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure crucial for AI and machine learning projects. Your role will involve bridging traditional data engineering with specific AI requirements to ensure high-quality data preparation and efficient data flow into AI and GenAI applications. This is a full-time on-site position based at the office in Infopark, Kochi. Key Responsibilities: - Build, test, and maintain scalable data pipelines for AI and machine learning workflows. - Develop and manage architected data solutions to support generative and predictive AI use cases. - Automate data acquisition, transformation, integration, cleansing, and validation from structured and unstructured sources. - Collaborate with data scientists, AI/ML engineers, and business teams to understand requirements and ensure model readiness. - Optimise ETL/ELT processes for scalability, reliability, and performance. - Manage data quality frameworks, monitor pipelines, and address data drift, schema changes, or pipeline failures. - Deploy and track real-time and batch pipelines supporting AI model inference and training. - Implement security, privacy, and compliance procedures for all AI data operations. - Document infrastructure, data flows, and operational playbooks related to AI solutions. Qualifications Required: - Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field. - Strong expertise with data pipeline orchestration tools like Apache Airflow, Luigi, Prefect. - Proficiency in SQL, Python, and experience working with big data frameworks such as Spark and Hadoop. - Familiarity with ML/AI frameworks like TensorFlow, PyTorch, Scikit-learn and MLOps practices. - Experience with cloud data solutions/platforms such as AWS, GCP, Azure. - In-depth understanding of data modelling, storage, governance, and performance optimisation. - Ability to manage both batch and streaming data processes and work with unstructured data such as images and text. - Excellent troubleshooting, analytical, and communication skills.,