Data Operations Engineer (Azure Data Factory)

Role & responsibilities

•       Experience in ADF

•       Experience in SQL Server

•       Ready to work in Shift (We have 24 X 7 Shift)

•       Maintaining and monitoring of Data Pipelines

•       Ability to understand the issues with pipelines and perform root cause analysis

•       Monitor data quality pillars like uniqueness, completeness, correctness, timeliness,

consistency, availability

•       Good documenting skills for capturing issue logs, sequence of events, root cause

analysis and short term, long term fix details

•       Updating wiki page for each pipeline enhancement or change.

•       Assisting partner teams on information related to data issues so as to arrive at issue

resolution

•       Send regular updates on status of open issues, resolved issues

•       Periodic review of Wiki pages and project documentation to ensure correctness of

information and data quality rules

•       Updating ADO daily including discussion threads at user story level, tracking efforts

against well-defined tasks, closure of tasks

•       Reporting on day-to-day progress to tech lead

•       Proper Shift hand over to next support engineer.

Secondary Skill set

•       Experience in ADB/Synapse/SQL/Cosmos/Scope scripting 

•       Knowledge of MSBI tech stack

•       Perform data validation/quality assurance analysis around datasets

•       Evaluate existing data quality rules and update them based on source field level or

schema changes

•       Working Knowledge on MSBI technology stack

•       Refer to existing PBI dashboards and monitor regularly for trends, anomalies and take

necessary action

Preferred candidate profile