Senior Data Engineer (Data Quality & Governance Engineer)- Remote, India

Job Summary

Responsibilities

• Design and enforce data contracts across Raw → Clean → Modeled zones.
• Define schema versioning policies, KPI logic, and metadata rules across business-critical datasets.
• Implement row-level and statistical validations using Great Expectations or Deequ.
• Create audit-ready QA tables to track failed checks, schema mismatches, and data regressions.
• Track end-to-end lineage and KPI evolution using OpenMetadata (with Glue/S3/Athena).
• Auto-classify columns as PII, derived, or forecast-driving fields using AWS Glue Tags/Scripts.
• Provide CTAS-based Athena queries for building QA dashboards and UAT verifications.
• Build BI-ready, QA-approved datasets for downstream tools like Superset and Power BI.
• Establish reusable profiling and validation dashboards for data quality and business teams.
• Collaborate with engineers, QA, and business SMEs to finalize data validation logic.
  

Requirements

Essential Skills:

Job

• Hands-on experience with AWS Glue (Jobs, Crawlers, Catalog), S3, and Athena.
• Strong foundation in data contracts, quality enforcement, and schema versioning.
• Expertise in using Deequ or Great Expectations for anomaly detection and data validation.
• Familiarity with OpenMetadata, Amundsen, or custom metadata tracking solutions.
• Ability to tag and manage sensitive data fields (e.g., PII, model inputs, derived KPIs).
• Strong SQL with Athena (CTEs, CTAS, filters, aggregations).
• Experience building QA dashboards in Superset, Streamlit, or similar BI tools.
  

Personal

• Excellent collaboration and communication with QA, architects, and business teams.
• Self-driven with attention to detail in schema accuracy and metadata enrichment.
• Ability to translate KPIs and quality rules into validation logic.
• Proactive in surfacing data regressions and audit issues before they reach production.
• High ownership mindset with a strong data compliance and governance attitude.
  

Preferred Skills

Job

• Implementation of statistical QA techniques like z-score anomalies or entropy thresholds.
• Experience handling rejected record logs, schema drift validations, and data reconciliation.
• Awareness of AWS cost optimization techniques in Glue and Athena.
  

Personal

• Proactive, ownership-driven mindset with a collaborative approach.
• Strong communication and collaboration skills.
• Strong problem-solving skills with attention to detail.
• Have the ability to work under stringent deadlines and demanding client conditions.
• Strong analytical and problem-solving skills.
• Ability to work in fast-paced, delivery-focused environments.
• Should have strong mentoring and documentation skills.
• Ability to take end-to-end ownership of QA validation modules.
  

Other Relevant Information

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Minimum 9+ years of experience in data engineering & architecture.
  

Benefits

• This role offers the flexibility of working remotely in India.