Data Quality & Automation Engineer

Key Skills:

Roles and Responsibilities:

• Convert manual test cases in Azure DevOps into automated validation scripts using Python (VS Code).
• Design, implement, and optimize SQL validation queries for large-scale data reconciliation.
• Maintain and enhance existing Python-based data validation frameworks for source-to-target reconciliation across Oracle, SQL Server, and Databricks.
• Manage and execute scenario-based data validations, including record counts, null checks, data accuracy, and business rule validations.
• Perform cross-system data validation for both batch and streaming data flows.
• Automate execution flows, structured logging, exception handling, and report generation (CSV, text, dashboards).
• Debug and maintain framework scripts, manage environment setup (Oracle Instant Client, ODBC drivers), and troubleshoot connectivity or runtime issues.
• Develop and maintain unit and smoke tests for Python scripts and SQL queries to ensure framework reliability.
• Collaborate with business and technical teams to define validation requirements, establish quality thresholds, and interpret validation results.
• Manage Git repositories, ensuring proper branching strategies, merges, and version control best practices.

Skills Required:,

• Strong experience in Azure Cloud services and data engineering ecosystems.
• Proficiency in Python programming, especially for automation and data validation.
• Hands-on experience with SQL and database performance optimization.
• Familiarity with Azure DevOps for pipeline automation and CI/CD integration.
• Experience in data reconciliation, data validation frameworks, and ETL testing.
• Knowledge of Git, version control, and collaborative development workflows.
• Excellent analytical, problem-solving, and communication skills.

Education: