Data Engineer

Transform and process source data into data products and enable egression to other platforms
Develop efficient Data Lake frameworks using Scala, PySpark, or Python, with robust data quality and security controls.Create reusable libraries and config driven pipelines for enterprise wide ingestion and integrationEnsure compliance with data governance and security standards using tools like AWS DataZone, Lake Formation, AWS Glue and MWAACommunicate effectively with internal stakeholders and contribute to team knowledge sharingExperience with CI/CD tools such as GitHub, GitHub Actions, ArgoCD, and automated deployment pipelines.Experience with observability tools like Cloudwatch, Prometheus, or GrafanaExposure to AI/ML technologies, with the ability to integrate or leverage AI models to enhance data processing, automation, or system intelligenceExperience using DevSecOps methodologies and best practice to automate deployments and testingDemonstrable in Python, SQL, Spark, and Scala for distributed data processing.Demonstrable experience with AWS Glue, Lambda, Redshift, Athena, Kinesis, and EMRExposure to AWS Marketplace, API Gateway, and data monetisation strategies.Performance tuning of ETL jobs and query optimisation for cost efficiencyDesign and develop next generation data platforms and pipeline solutions.Build and maintain backend services and integrations using Java Spring Boot, ensuring performance, scalability, and security.Collaborate with engineering peers to co create and contribute to shared practices and frameworksApply modern coding standards to deliver high quality, reliable solutions that address core business needs and reduce risk.Strong experience in Java, Spring Boot, and RESTful API development