- Design, build and optimise data pipelines and ETL processes in Azure Databricks ensuring high performance, reliability, and scalability.
- Implement best practices for data ingestion, transformation, and cleansing to ensure data quality and integrity.
- Work within clients best practice guidelines as set out by the Data Engineering Lead
- Work with data modellers and testers to ensure pipelines are implemented correctly.
- Collaborate as part of a cross-functional team to understand business requirements and translate them into technical solutions.
Role Requirements
- Strong Data Engineer with experience in Financial Services
- Knowledge of and experience building data pipelines in Azure Databricks
- Demonstrate a continual desire to implement strategic or optimal solutions and where possible, avoid workarounds or short term tactical solutions
- Work within an Agile team
Experience/Skillset
- 8+ years experience in data engineering
- Good skills in SQL, Python and PySpark
- Good knowledge of Azure Databricks (understanding of delta tables, Apache Spark, Unity Catalog)
- Experience writing, optimizing, and analyzing SQL and PySpark code, with a robust capability to interpret complex data requirements and architect solutions
- Good knowledge of SDLC
- Familiar with Agile/Scrum ways of working
- Strong verbal and written communication skills
- Ability to manage multiple priorities and deliver to tight deadlines
WHY JOIN CAPCO
You will work on engaging projects with some of the largest banks in the world, on projects that will transform the financial services industry. We offer: A work culture focused on innovation and creating lasting value for our clients and employees Ongoing learning opportunities to help you acquire new skills or deepen existing expertise A flat, non-hierarchical structure that will enable you to work with senior partners and directly with clients A diverse, inclusive, meritocratic culture
We offer:
- A work culture focused on innovation and creating lasting value for our clients and employees
- Ongoing learning opportunities to help you acquire new skills or deepen existing expertise
- A flat, non-hierarchical structure that will enable you to work with senior partners and directly with clients