Data Engineer

Responsibilities

• Working with the client and the team to determine data processing and access needs and how best to deliver them.
• Creation and support of highly available data pipelines and storage solutions (e.g. data warehouses).
• Delivery of software using agile (pair programming, TDD, CI/CD, etc) practices.
• Automation of data infrastructure and deployments.
• Advocating agile practices to the client organisation.
• Mentoring client team members.
• Helping improve the team and clients data capabilities.

Required Skills

• Significant experience working on data pipelines, platforms and projects.
• Experience in developing solutions in one or more of the main Cloud providers (AWS, GCP, Azure).
• Strong background in at least one programming language, Python or Scala preferable.
• Working with a variety of databases, data warehouses and distributed file systems in production environments.
• Strong experience in SQL and data modelling.
• Exposure with data processing at scale.
• Knowledge about GDPR and data security.
• Working with source control and infrastructure as code (Git/Github, Terraform, ansible etc.) and CI/CD tools.
• Understanding and being able to support data scientists or business intelligence teams - in data prep, mentoring and support functions.
• Strong collaboration and communication skills, including the ability to communicate effectively while working remotely with globally distributed