Senior Data Engineer

Role & responsibilities

Top 3 skills which is mandatory :

Must have skills:- Data Engineer, SQL, DBT, Azure & Databricks

Job Duties:

Your role

As a Data Engineer, your main responsibilities will involve:

• Strong experience working in projects in DBT.
  • The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects.• Analyze data models and derive logical conclusions.• Processes modeling.• Hands on development and monitoring of the Azure cloud Platform and various associated components for data ingestion, transformation and processing.• Effectively diagnose, isolate, and resolve complex problems pertaining to data infrastructure, including performance tuning and optimization.• Designs and writes programs according to functional and non-functional requirements.• Develops and maintains technical documentation.• Follows established configuration/change control processes.

Minimum Skills Required: As a Big Data Developer / Data Engineer, your skills and experience will ideally include:

• University level education or equivalent
  • 2+ years of experience working as a Data Engineer• Knowledge of design patterns• Ability to write complex SQL queries• A good understanding of distributed computation• DBT• GIT• Experience with Azure cloud.• Experience with Databricks.

Nice to have:

• Python
  • 1+ Experience in functional programming (preferably Scala - big plus)• 1+ years of experience in Apache Spark 2.0 (big plus)• Experience in R programming• Experience with Hadoop technologies (i.e. Sqoop, Hive, Flume etc.)• Continuous Integration / Continuous Deployment (Azure DevOps)• Experience in data exploration using notebooks (Databricks notebooks).• Experience in automation of regular tasks(e.g. data refreshes) using bash and python scripting."