AI Data Engineer

As a Data Engineer at Capgemini Invent, your role will involve designing, developing, and maintaining scalable data pipelines to support ML models. You will collaborate closely with data scientists and ML engineers to ensure high-quality data inputs for ML applications. Additionally, you will conduct exploratory data analysis, manage datasets of various data types, and optimize data workflows for efficiency and performance. Your key responsibilities will include: - Designing, developing, and maintaining scalable data pipelines for ML models. - Performing data preprocessing, cleansing, and labeling to ensure high-quality data inputs. - Conducting exploratory data analysis to uncover insights and identify data patterns. - Collaborating with data scientists and ML engineers to align data pipeline requirements with model development needs. - Creating and managing datasets comprising text, image, audio, and video data for ML applications. - Implementing best practices for data management, focusing on data integrity, consistency, and security. - Optimizing data workflows and processing pipelines for enhanced efficiency and performance. - Utilizing cloud-based data storage and processing solutions as necessary. - Keeping up to date with industry trends and technologies to enhance data engineering processes. - Providing technical support and guidance to junior data engineers and team members. Qualifications required for this role: - Bachelors or Masters degree in Computer Science, Engineering, or a related field. - 5+ years of experience in data engineering, with a focus on building data pipelines and preprocessing data. - Strong proficiency in programming languages such as Python, Java, or Scala. - Hands-on experience with data processing frameworks and tools like Apache Spark, Hadoop, or similar. - Proficiency in SQL and practical experience with relational and NoSQL databases. - Experience with data visualization and EDA tools like Pandas, Matplotlib, or Tableau. - Familiarity with ML and AI concepts, particularly in the context of data preparation and pipelines. - Experience with text, image, audio, and video data management, including labeling and cleansing. - Exposure to EdgeAI applications and their unique data processing requirements (preferred). - Strong problem-solving skills and the ability to work both independently and collaboratively.,