Hadoop Big data engineer

Responsibilities

• Onboarding and updating data deliveries on our Hadoop Datalake
  • Developing frameworks to process and deploy new data delivery methods or file formats (JSON, XML, CSV, ProtoBuf, Avro, Parquet, etc) • Creating views and tables for our users with optimal performance for their use case • Experience of connecting and interfacing with systems via APIs • Support both Development and Operations teams in quickly solving complex issues that are hindering the team and/or stakeholders • Coordinating with source teams producing data and also consumers using this data for their use cases • Supporting in the design, building, and testing of new services and extending existing services on our platform • Team collaboration on complex challenges, code reviews and testing new functionalities • Evaluating stakeholder requirements and guiding the innovation team towards the most suitable solution architecture • Identifying common development patterns and ensuring generic solutions and architecture are applied where possible • Take the design lead for complex requirements that go beyond the standard work done by the team • Continue to move the teams towards using more automation and efficient ways of working, to improve quality, eliminate manual mistakes and be more predictable with our deliveries

Education, Experience, and Licensing Requirements:

Domain