Data Operations Engineer

Job Description

Managing data Operations - from data ingestion to the final delivery of the Product to our clients/internal stakeholders. The process includes handling RWD, unstructured healthcare data, and other associated datasets. Needs to coordinate with the vendors and work with the Data Engineering team to load the data into the warehouse. Work with stakeholders including the Product, Data, and Design teams to assist with data[1]related technical issues and support their needs. Meet the delivery SLAs to load/deliver the data and make it available for internal and external stakeholders Should assess structural modifications, volume consistency, fill rate consistency, and comprehensive data Investigate production problems to identify the underlying causes and suggest fixes Collaborate with cross-functional teams to troubleshoot data problems and provide solutions. Assess the integrity of internal data sources and processes, work with project managers, client service managers, and other internal teams to address problems, implement preventative measures, and enhance procedures. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, etc. Shall keep themselves and the team updated about market events that will affect the data quality in the next few refreshes. What makes you a great fit: 3-5 years of Hands-on experience working with healthcare data types, including RWD (real-world data), claims, EHR, unstructured patient data, etc Hands-on experience in critical problem-solving, writing advanced database queries, and SQL utilities to perform data extraction, data analysis, and working with the data engineering team. Proficiency in Data modeling, validation, quality check, and data engineering concepts. Experience working with big-data processing technologies using - databricks, dbt, S3, Delta lake, Deequ, Griffin, Snowflake, BigQuery. Familiarity with version control technologies, and CI/CD systems. Data Quality Assurance Responsibilities and Duties Write and review QA plans. Validate all incoming and outgoing data Meet the quality SLAs Run test cases, identify issues, and work with the team to resolve them Identify any customer-impacting issues and communicate them beforehand What makes you a great fit: 2-3 years of Hands-on experience working with analytical data Should be able to run the automated, semi-automated, manual tests Experience working with healthcare data types, including RWD (real-world data), claims, EHR, unstructured patient data, etc is a plus. Hands-on experience in critical problem-solving, writing advanced database queries, and SQL utilities to perform data extraction, data analysis, and working with the data engineering team. Experience working with QA frameworks like Great Expectations