Job Description

Role & responsibilities A good understanding of data warehousing concepts, including data warehouse technical architectures, infrastructure components, ETL/ELT and reporting/analytic tools Azure experience, including expert understanding of Azure Data Lakes Experience in the extraction of data from SQL servers Must have experience in at least two programming languages among Java, Scala or Python Experience in wider financial services or insurance sector is preferred. High level of accuracy and attention to detail and the ability to simplify problems into component parts and deal with them systematically Understanding of Meta-data driven data ingestion using Kafka Experience in coaching / mentoring less experienced engineers through technical problem solving Strong customer / value focus Driven to deliver business value Desirable but not essential - understanding of Postgres databases Desirable but not essential - experience of data visualization tools (QlikSense etc) Preferred candidate profile Degree, MSc or PhD in a numerate or scientific discipline (e.g. Computer Science, Physics, Mathematics etc) 4+ years of experience in developing scalable solutions using big data technologies Hands-on development experience on Hadoop, spark, Hive, H-Base, Kafka Experience in ELK stack Experience in NoSQL databases like Cassandra, MongoDB etc Cloud experience (Azure(preferred) or AWS) Experience in application, data, and infrastructure architecture disciplines Translate, load, and exhibit unrelated data sets in various formats and sources like JSON, text files, Kafka queues, and log data. Proven design, coding, testing, and debugging skills and experience. Experience in end-to-end release of highly reliable applications including design, development, and testing. Understanding of Linux OS core principles, performance and tuning A solid understanding of database design and data manipulation Experience in building data processing, ETL and data pipelining workflows Perks and benefits