Sr. Data Engineer (PySpark, Azure Databricks)

We are seeking a highly skilled Senior Data Engineer with extensive experience in designing, building, and optimizing high-volume data pipelines.

This is a fully remote job opportunity!

Responsibilities

• Adhere to coding and Numerator technology standards 
• Build suitable automation test suites within Azure DevOps 
• Maintain and update automation test suites as required 
• Carry out manual testing, load testing, exploratory testing as required 
• Work closely with Business Analysts and Senior Data Developers to consistently achieve sprint goals 
• Assist in estimation of sprint-by-sprint stories and tasks 
• Pro-actively take a responsible approach to product delivery

Requirements

• 7-10 years of experience in data engineering roles
• Good

  Python skills

• Experience working with

  Microsoft Azure Cloud

• Experience in Agile methodologies (Scrum/Kanban)
• Experience with Apache Spark,

  PySpark, Databricks

• Experience working with Devops pipeline, preferably

  Azure DevOps

Preferred Qualifications

• Bachelor's or master's degree in computer science, Information Technology, Data Science, or a related field.
• Experience working in a support focused role 
• Knowledge/experience in AI ML techniques
• Certification in relevant Data Engineering discipline or related fields.