AWS Data Engineer

Desired Competencies (Technical/Behavioral Competency)

• Must-Have • Good hands on experience in Python programming and Pyspark • Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift) • Required skill set- SQL, Airflow • Must Have Experience with snowflake or Hive or AWS S3 • Responsibility Will be accountable for build/test complex data pipelines (batch and near real time) • Expectation – Readable documentation of all the components being develop • Experience in writing SQLs and stored procedures • Working experience with RDBMS (Oracle / Teradata)
• Good-to-Have • Good understanding of Data warehouse • Familiarize in ETL tools like Informatica • Experience working with NoSQL database like DynamoDB or MongoDB .

SN Responsibility of / Expectations from the Role

1 Create and maintain optimal data pipeline architecture

2 Assemble large, complex data sets that meet functional / non-functional business requirements.

3 Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

4 Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.