Data Engineer

As an experienced Data Engineer at Regnology, you will play a crucial role in designing ingestion pipelines, optimizing query performance, and ensuring data quality, governance, and cost efficiency at scale. Your responsibilities will include: - **Migration Strategy & Execution**: - Design and implement data ingestion pipelines to extract data from Oracle into GCS/Iceberg. - Migrate and modernize existing Oracle schemas, partitions, and materialized views into Iceberg tables. - Define CDC (Change Data Capture) strategies using custom ETL. - **Data Lakehouse Architecture**: - Configure and optimize Trino clusters (coordinator/worker, Helm charts, autoscaling). - Design partitioning, compaction, and clustering strategies for Iceberg tables. - Implement schema evolution, time-travel, and versioning capabilities. - **Performance & Cost Optimization**: - Benchmark Trino query performance vs Oracle workloads. - Tune Trino/Iceberg for large-scale analytical queries, minimizing query latency and storage costs. - **Data Quality, Metadata & Governance**: - Integrate Iceberg datasets with metadata/catalog services (Postgre/Hive Metastore, or Glue). - Ensure compliance with governance, observability, and lineage requirements. - Define and enforce standards for unit testing, regression testing, and data validation. - **Collaboration & Delivery**: - Support existing reporting workloads (regulatory reporting, DWH) during and after migration. - Document architecture, migration steps, and provide knowledge transfer. **Why you should decide on us**: - Lets grow together, join a market-leading SaaS company - our agile character and culture of innovation enable you to design our future. - We provide you with the opportunity to take on responsibility and participate in international projects. - In addition to our buddy-program, we offer numerous individual and wide-ranging training opportunities during which you can explore technical and functional areas. - Our internal mobility initiative encourages colleagues to transfer cross-functionally to gain experience and promotes knowledge sharing. - We are proud of our positive working atmosphere characterized by a supportive team across various locations and countries and transparent communication across all levels. - Together we're better - meet your colleagues at our numerous team events. **Qualifications Required**: - 5+ years of experience. - Prior experience migrating financial/regulatory datasets. - Experience with Regulatory Reporting or similar enterprise workloads. - Familiarity with large-scale performance benchmarking and cost modeling. **Required Skills & Experience**: - **Core Expertise**: - Strong hands-on experience with Trino/Presto, Apache Iceberg, and Oracle SQL/PLSQL. - Proven experience with data lakehouse migrations at scale (50 TB+). - Proficiency in Parquet formats. - **Programming & Tools**: - Solid coding skills in Java, Scala, or Python for ETL/ELT pipeline development. - Experience with orchestration (Spark). - Familiarity with CDC tools, JDBC connectors, or custom ingestion frameworks. - **Cloud & DevOps**: - Strong background in GCP (preferred) or AWS/Azure cloud ecosystems. - Experience with Kubernetes, Docker, Helm charts for deploying Trino workers. - Knowledge of CI/CD pipelines and observability tools. - **Soft Skills**: - Strong problem-solving mindset with the ability to manage dependencies and shifting scopes. - Clear documentation and stakeholder communication skills. - Ability to work in tight delivery timelines with global teams. Regnology is a leading international provider of innovative regulatory, risk, and supervisory technology solutions, serving over 7,000 financial services firms with reporting solutions globally. The company offers a positive working atmosphere, numerous training opportunities, and promotes knowledge sharing among colleagues. If this challenging opportunity excites you, apply now at [Regnology Careers](https://www.regnology.net).,