This is a remote position.
Shuru is an AI-native technology and engineering company that partners with businesses to build and scale high-impact digital products, platforms, and AI-driven solutions. We work with fast-growing organizations across industries, helping them solve complex technology challenges through strong engineering, product, data, and AI capabilities.
Senior Data Engineer
1. Pipeline Development
-
Build, test, and maintain data pipelines on Azure Databricks for the assigned business line, using PySpark and SQL.
-
Implement Bronze, Silver, and Gold layer transformations using Delta Lake, following the platform’s established lakehouse standards.
-
Ingest structured and semi-structured data from sources such as MariaDB, Azure Event Hubs, APIs, and file systems.
2. Data Modelling
-
Design and build data models (dimensional models, Gold-layer tables) for the business line’s use cases, in line with patterns set by the Lead Data Engineer.
-
Flag where a business line’s data model needs a shared, cross-line solution rather than a local one, and raise it to the Lead Data Engineer.
3. Quality & Reliability
-
Implement automated testing, data validation, and pipeline monitoring for everything owned.
-
Respond to and troubleshoot pipeline incidents and data quality issues for the assigned business line, escalating where needed.
-
Follow the platform’s cataloguing, lineage, and metadata practices so the data produced is discoverable and trustworthy.
4. Collaboration & Delivery
-
Work directly with the Forward Deployed Data Lead to turn scoped business requirements into build plans and delivery timelines.
-
Partner with analysts and, where relevant, data scientists and AI engineers consuming the pipelines built, to make sure outputs meet their needs.
-
Participate in code reviews and platform standards discussions led by the Lead Data Engineer.
5. Compliance
- Ensure pipelines and datasets meet data security, privacy, and regulatory requirements (e.g., GDPR and PII handling) relevant to the business line.
Requirements
-
5–8 years of experience in data engineering, including hands-on work with cloud data platforms.
-
Experience building pipelines on Azure Databricks, Delta Lake, or comparable lakehouse/cloud-native tools.
-
Exposure to version control (Git), Agile methodologies, and CI/CD practices.
-
Experience in fintech, broking, or trading platforms is a plus, not required.
-
Bachelor’s degree in Computer Science, Engineering, or a related field.
Benefits
-
Work on challenging, large-scale engineering and data problems.
-
Build modern platforms using cutting-edge cloud and data technologies.
-
Work closely with experienced engineering, product, and technology leaders.
-
Opportunity to take ownership of architecture and key technical decisions.
-
Remote-first and collaborative work environment.
-
Strong opportunities for learning, leadership, and career growth.