Data Engineer

Job Description

You will be joining the newly formed AI, Data Analytics team, primarily responsible as a Data Engineer leading various projects within the new Data Platform team. The new team is focused on driving increased value from the data InvestCloud captures to enable a smarter financial future for our clients, in particular focused on enhanced intelligence. Ensuring we have fit-for-purpose modern capabilities is a key goal for the team. Key Responsibilities Assist in the Design, development, and maintenance of scalable data pipelines to support diverse analytics and machine learning needs. Manage data architectures for reliability, scalability, and performance. Support data integration solutions from our data partners, including ETL/ELT processes, ensuring seamless data flow across platforms. Collaborate with Data Scientists, Analysts, and Product Teams to define and support data requirements. Manage and maintain data platforms such as Oracle, Snowflake, and/or Databricks, ensuring high availability and performance, whilst optimizing for cost. Ensure data security and compliance with company policies and relevant regulations. Monitor and troubleshoot data systems to identify and resolve performance issues. Develop and maintain datasets and data pipelines to support Machine Learning model training and deployment Analyze large datasets to identify patterns, trends, and insights that can inform business decisions. Work with 3rd party providers of Data and Data Platform products to evaluate and implement solutions achieving Investclouds business objectives. Required Skills Bachelors or Masters degree in Computer Science, Engineering, or a related field, or equivalent practical experience. Minimum of 4 years of professional experience in data engineering or a related role. Proficiency in database technologies, including Oracle and PostgreSQL. Hands-on experience with Snowflake and/or Databricks, with a solid understanding of their ecosystems. Experience with programming languages such as Python or SQL. Familiarity with ETL/ELT tools and data integration frameworks. Experience with cloud platforms such as AWS, GCP, or Azure. Familiarity with containerization and CI/CD tools (e.g., Docker, Git). Strong problem-solving skills and the ability to handle complex datasets. Good communication skills to collaborate with global technical and non-technical stakeholders. Knowledge of data preprocessing, feature engineering, and model evaluation metrics Excellent proficiency in English Ability to work in a fast-paced environment across multiple projects simultaneously Ability to collaborate effectively as a team player, fostering a culture of open communication and mutual respect. Preferred skills Knowledge of data warehousing and data lake architectures. Familiarity with governance frameworks for data management and security. Familiarity with Machine Learning frameworks (TensorFlow, PyTorch, Scikit-learn) and LLM frameworks (e.g. Langchain)