Job
Description
As a senior data pipeline engineer at Agoda, you will be working on distributed systems that span multiple data centers, thousands of servers, and process hundreds of billions of messages a day. Your role will focus on ensuring data quality, integrity, and accuracy while designing new scalable systems to handle the increasing data volume. You will have the opportunity to manage projects with a small team, enhancing your ownership and leadership skills. Your responsibilities will include building, administering, and scaling data pipelines, developing and expanding existing frameworks used by teams, managing data ingestion into multiple systems, building tools to monitor data accuracy SLAs, exploring new technologies to improve data quality and processes, and developing quality software through design reviews, code reviews, and test-driven development. In this role, you will: - Build, administer, and scale data pipelines processing hundreds of billions of messages a day across multiple data centers - Develop and expand existing frameworks used by Teams throughout Agoda for producing messages to the data pipeline - Manage data ingestion into multiple systems such as Hadoop, ElasticSearch, and other Distributed Systems - Build tools to monitor high data accuracy SLAs for the data pipeline - Explore new technologies to enhance data quality, processes, and data flow - Develop quality software through design review, code reviews, and test-driven development To succeed in this role, you will need: - A Bachelor's degree in Computer Science, Information Systems, Computer Engineering, or a related field - 8+ years of industry experience, preferably at a tech company - Good knowledge of data architecture principles - Operational experience debugging production issues - Strong coding skills in multiple programming languages like Golang, Java, Scala, Python, C#, etc. - Detail-oriented mindset with excellent communication skills in technical English - Understanding of Kafka and experience as a Kafka Administrator - Experience with data ingestion from Kafka into systems like Hadoop, ElasticSearch, and other Distributed Systems - Strong systems administration skills in Linux - Previous work on or contribution to Open Source Projects Agoda is an Equal Opportunity Employer. Your application will be kept on file for future vacancies, and you can request to have your details removed at any time. Agoda does not accept third-party resumes, and they are not responsible for any fees related to unsolicited resumes. As a senior data pipeline engineer at Agoda, you will be working on distributed systems that span multiple data centers, thousands of servers, and process hundreds of billions of messages a day. Your role will focus on ensuring data quality, integrity, and accuracy while designing new scalable systems to handle the increasing data volume. You will have the opportunity to manage projects with a small team, enhancing your ownership and leadership skills. Your responsibilities will include building, administering, and scaling data pipelines, developing and expanding existing frameworks used by teams, managing data ingestion into multiple systems, building tools to monitor data accuracy SLAs, exploring new technologies to improve data quality and processes, and developing quality software through design reviews, code reviews, and test-driven development. In this role, you will: - Build, administer, and scale data pipelines processing hundreds of billions of messages a day across multiple data centers - Develop and expand existing frameworks used by Teams throughout Agoda for producing messages to the data pipeline - Manage data ingestion into multiple systems such as Hadoop, ElasticSearch, and other Distributed Systems - Build tools to monitor high data accuracy SLAs for the data pipeline - Explore new technologies to enhance data quality, processes, and data flow - Develop quality software through design review, code reviews, and test-driven development To succeed in this role, you will need: - A Bachelor's degree in Computer Science, Information Systems, Computer Engineering, or a related field - 8+ years of industry experience, preferably at a tech company - Good knowledge of data architecture principles - Operational experience debugging production issues - Strong coding skills in multiple programming languages like Golang, Java, Scala, Python, C#, etc. - Detail-oriented mindset with excellent communication skills in technical English - Understanding of Kafka and experience as a Kafka Administrator - Experience with data ingestion from Kafka into systems like Hadoop, ElasticSearch, and other Distributed Systems - Strong systems administration skills in Linux - Previous work on or contribution to Open Source Projects Agoda is an Equal Opportunity Employer. Your application will be kept on file for future vacancies, and you can request to have your details removed at any time. Agoda does not accept third-party resumes
