Data Engineer

Job Description

• Design, implement, and maintain data pipelines for processing large datasets, ensuring data availability, quality, and efficiency for machine learning model training and inference.
• Collaborate with data scientists to streamline the deployment of machine learning models, ensuring scalability, performance, and reliability in production environments.
• Develop and optimize ETL (Extract, Transform, Load) processes, ensuring data flow from various sources into structured data storage systems.
• Ensure effective model monitoring, versioning, and logging to track performance and metrics in a production setting.
• Ensure data security, integrity, and compliance with data governance policies.
• Perform troubleshooting and root cause analysis on production-level machine learning systems.
• Skills: Glue, Pyspark, AWS Services, Strong in SQL; Nice to have : Redshift, Knowledge of SAS Dataset