Senior Data Engineer

We are hiring for -

Role: Senior Data Engineer

Experience Range: 3

Qualification:

experience in the desired field

Roles & Responsibilities:

• As a Senior Data Engineer, you manage and develop the solutions in close alignment with various business and Spoke stakeholders. You are responsible for the implementation of the IT governance guidelines. 
• Collaborate with the Spokes Data Scientists, Data Analysts, and Business Analysts, when relevant.

Tasks

• Create and manage data pipeline architecture for data ingestion, pipeline setup and data curation
• Experience working with and creating cloud data solutions 
• Assemble large, complex data sets that meet functional/non-functional business requirements
• Implement the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Pyspark, SQL and AWS big data-technologies
• Build analytics tools that use the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
• Manipulate data at scale: getting data in a ready-to-use state in close alignment with various business and Spoke stakeholders

Must Have:

Advanced knowledge:

• ETL
• Data Lake, Data Warehouse, RDS architectures knowledge
• Python, SQL (Any other OOP language is also valuable)
• Pyspark (preferably) or Spark Knowledge
• Object-oriented programming, Clean Code and good documentation skills
• AWS: S3, Athena, Lambda, Glue, IAM, SQS, EC2, Quicksight, and etc.                                       
• Git                                                                                                                                         
• Data Analysis & Visualization

Optional:.

• AWS CDK Cloud Development Kit
• CI/CD knowledge