Lead Data Engineer - Java / Scala

Job Description

- Bachelor's degree in Computer Science, Information Technology, or a related field (Master's preferred). Proven experience as a Data Engineer, with a focus on cloud migrations, preferably on GCP with 8 to 10 + yrs. exp Strong programming skills in languages Java, or Scala. Strong expertise in building and maintaining data pipelines and data warehouses. Develop a comprehensive migration strategy for our data infrastructure and workloads to GCP. Proficiency in GCP services, including Big Query, Dataproc, Pub/Sub and Cloud Storage. Migrate data processing jobs using GCP services. Experience with data modelling, ETL processes, and data transformation. Lead the migration of existing EMR pipelines to GCP, including ETL processes, data ingestion, and transformation workflows. Experience with Data Catalogue migration & updating the partitions on day-to-day basis Ensure data pipelines are optimized for performance and cost-effectiveness. Troubleshoot and resolve data pipeline issues. Monitor and optimize data processing and storage costs. Excellent problem-solving and analytical skills. Effective leadership and team management abilities. Strong communication and collaboration skills.