Lead - Data Engineering (AWS)

Annalect is currently seeking a data engineering lead to join our technology team. In this role you will be building data pipelines and developing data set processes. We are looking for people who have a shared passion for technology, design & development, data, and fusing these disciplines together to build cool things. In this role, you will lead teams working on one or more software and data products in the Annalect Engineering Team. You will participate in technical architecture, design and development of software products as well as research and evaluation of new technical solutions.  You will also help us drive the vision forward for data engineering projects by helping us extend and improve standards, and by building high performing, collaborative, enthusiastic teams.

• Managing delivery of multiple complex simultaneous development projects from design through to release
• Capable of understanding and contributing to the technical solution from design through to code level
• Working closely with Project Managers to produce accurate delivery estimates
• Hands on development and maintenance of CRM database and datalake solutions
• Writing technical specifications, participating in code reviews and meetings, and providing updates on project progress
• Liaising with key stakeholders, client, and other Code departments to facilitate the transition of solution from analysis through to design and delivery
• Attend client meetings if required to discuss and demonstrate solutions at both technical and business level
• System integration of multiple products to build multichannel communications solutions to meet client requirements
• General development and support programming involving investigation of code and data issues.
• Providing training to internal & external personnel
• Ready to learn new skills and technologies required for the job

• Strong design and development experience on datalake and database solutions using AWS services
• Experience in delivery of complex data projects
• Strong Data Modelling, Database & Data Warehouse analysis and design skills
• Experience in ETL & Database design and development
• Data manipulation, exploration, analysis, and data integrity (experience of manipulating data of varied sources and targets like RDBMS, XML, JSON and Flat Files)
• Demonstrated ability to understand and discuss complex technical concepts.
• Knowledge of software development processes including Agile, Lean, Waterfall & V-model
• Commercial experience in technical delivery of projects
• Integration of 3rd party products, such as Adobe Campaign, Acoustic Campaign Automation & Salesforce Marketing Cloud to solutions to enable multi-channel communications

Essential Skills:

• AWS Glue with PySpark/Python
• AWS Lambda with Python
• AWS Step functions or Apache Airflow
• Amazon S3
• Apache Kafka
• Amazon Redshift and Redshift Spectrum
• Data Analysis and Data Modelling
• Other AWS Services: IAM, CloudWatch, Secrets Manager, Parameter Store
• Experience in development of Datalake on AWS
• Experience in specifying deployment packages on AWS
• Advanced SQL knowledge