Senior Data Engineer

Job Description

Architect of Solutions: Lead the design, development, and enhancement of scalable ML Ops/Data pipelines and Data Products on cloud-native platforms. Technical Expertise: Demonstrate your expertise in Python, Pyspark, Docker, Kubernetes, and AWS ecosystems to deliver exceptional solutions. Collaborative Spirit: Work hand-in-hand with global and diverse Agile teams, from data to design, to overcome technical data challenges. Innovate & Inspire: Stay ahead of the curve by integrating the latest industry trends and innovations into your work such as GenAI. Essential Skills/Experience A proactive mindset and enthusiasm for Agile environments. Strong hands-on experience with cloud providers and services. Experience in performance tuning SQL and ML Ops data pipelines. Extensive experience in troubleshooting data issues, analyzing end-to-end data pipelines, and working with users in resolving issues. Masterful debugging and testing skills to ensure excellence in execution. Inspiring communication abilities that elevate team collaboration. Experience of structured, semi-structured (XML, JSON), and unstructured data handling including extraction and ingestion via web-scraping and FTP/SFTP. Production experience delivering CI/CD pipelines (Github, Jenkins, DataOps.Live). Cloud DevOps Engineer who can develop, test, and maintain CICD Pipeline using Terraform, cloud formation. Remain up to date with the latest technologies, like GenAI / AI platforms and FAIR scoring to improve outcomes.