Big Data Engineer

Essential functions

• Develop and own the migration roadmap, including phased transition from on-prem to cloud while minimizing business disruption.

• Oversee data migration strategies (bulk historical loads, incremental sync, and cutover).

• Define and enforce coding standards, CI/CD pipelines, and automated testing for data pipelines.

• Partner with Data Architects, Cloud Engineers, and Security teams to align platform design with enterprise standards.

Qualifications

Mandatory

• Proven experience leading data engineering teams, including distributed teams across multiple geographies and time zones.

• Effective in managing cross-team collaboration with architects, product managers, and operations.

• Scala and Python

• Apache Spark (batch & streaming) must!

• Deep knowledge of HDFS internals and migration strategies.

• Experience with Apache Iceberg (or similar table formats like Delta Lake / Apache Hudi) for schema evolution, ACID transactions, and time travel.

• Running Spark and/or Flink jobs on Kubernetes (e.g., Spark-on-K8s operator, Flink-on-K8s).

• Experience with distributed blob storages like Ceph or AWS S3 and similar

• Building ingestion, transformation, and enrichment pipelines for large-scale datasets.

• Infrastructure-as-Code (Terraform, Helm) for provisioning data infrastructure.

• Strong communication skills

Would be a plus

Nice to have:

• Experience with Apache Flink

• Apple experience preferred (to enable him/her to get up to speed on our tooling set quickly and more independently)

We offer

• Opportunity to work on bleeding-edge projects


• Work with a highly motivated and dedicated team


• Competitive salary


• Flexible schedule


• Benefits package - medical insurance, sports


• Corporate social events


• Professional development opportunities


• Well-equipped office