Data Engineer

Responsibilities

• Design, build, monitor, and support large-scale data processing pipelines.
• Support, mentor, and pair with other members of the team to advance our team's capabilities and capacity.
• Help Blis explore and exploit new data streams to innovate and support commercial and technical growth.
• Work closely with Product and be comfortable with taking, making, and delivering against fast-paced decisions to delight our customers. This ideal candidate will be comfortable with fast feature delivery with a robust engineered follow-up.
  

Requirements

• 5+ years direct experience delivering robust, performant data pipelines within the constraints of direct SLA's and commercial financial footprints.
• Proven experience in architecting, developing, and maintaining Apache Druid and Imply platforms, with a focus on DevOps practices and large-scale system re-architecture.
• Mastery of building Pipelines in GCP, maximising the use of native and native supporting technologies e. g. Apache Airflow.
• Mastery of Python for data and computational tasks with fluency in data cleansing, validation, and composition techniques.
• Hands-on implementation and architectural familiarity with all forms of data sourcing i. e streaming data, relational and non-relational databases, and distributed processing technologies (e. g. Spark).
• Fluency with all appropriate Python libraries typical of data science e. g. pandas, scikit-learn, scipy, numpy, MLlib, and/or other machine learning and statistical libraries.
• Advanced knowledge of cloud-based services, specifically GCP.
• Excellent working understanding of server-side Linux.
• Professional in managing and updating tasks, ensuring appropriate levels of documentation, testing, and assurance around their solutions.
  

Desired

• Experience optimizing both code and config in Spark, Hive, or similar tools.
• Practical experience working with relational databases, including advanced operations such as partitioning and indexing.
• Knowledge and experience with tools like AWS Athena or Google BigQuery to solve data-centric problems.
• Understanding and ability to innovate, apply, and optimize complex algorithms and statistical techniques to large data structures.
• Experience with Python Notebooks, such as Jupyter, Zeppelin, or Google Datalab, to analyze, prototype, and visualize data and algorithmic output.