Data Engineer

Role Overview: As an AI Data Engineer, your primary responsibility will be to design, develop, and maintain data pipelines and infrastructure crucial for AI and machine learning projects. You will play a vital role in ensuring that models are trained on high-quality, well-prepared data and that data flows efficiently from diverse sources into AI and GenAI applications. This full-time on-site job is based at our office in Infopark, Kochi. Key Responsibilities: - Build, test, and maintain scalable data pipelines for AI and machine learning workflows. - Develop and manage architected data solutions such as warehouses, lakes, and streaming platforms to support generative and predictive AI use cases. - Automate data acquisition, transformation, integration, cleansing, and validation from structured and unstructured sources. - Collaborate with data scientists, AI/ML engineers, and business teams to understand requirements, provision data assets, and ensure model readiness. - Optimize ETL/ELT processes for scalability, reliability, and performance. - Manage data quality frameworks, monitor pipelines, and address data drift, schema changes, or pipeline failures. - Deploy and track real-time and batch pipelines supporting AI model inference and training. - Implement security, privacy, and compliance procedures for all AI data operations. - Document infrastructure, data flows, and operational playbooks related to AI solutions. Qualification Required: - Bachelors or Masters degree in Computer Science, Data Engineering, or a related field. - Strong expertise with data pipeline orchestration tools like Apache Airflow, Luigi, Prefect. - Proficiency in SQL, Python, and experience working with big data frameworks such as Spark, Hadoop. - Familiarity with ML/AI frameworks like TensorFlow, PyTorch, Scikit-learn, and MLOps practices. - Experience with cloud data solutions/platforms such as AWS, GCP, Azure. - In-depth understanding of data modeling, storage, governance, and performance optimization. - Ability to manage both batch and streaming data processes and work with unstructured data like images and text. - Excellent troubleshooting, analytical, and communication skills.,