Job
Description
Job Description Senior Data Engineer - Quality Engineering Experience Range: 7-10 Years Location: Bangalore (Hybrid Mode) Resillion, a leading quality engineering company with offices around the world, is seeking a talented Data Engineer to join our growing India team. In this role, you will play a critical part in building and maintaining the data infrastructure that supports our AI-powered testing tools and analytics solutions. You will have technical responsibility for the entire data lifecycle, from data acquisition and ingestion to storage, processing, and analysis. Responsibilities: Collaborate with GenAI engineers, Software Engineers, Test automation engineers and other stakeholders within Resillion to understand data needs and translate them into technical solutions, including designing data pipelines for training & deploying models and data pre-processing for AI, including generative AI applications. Design, implement, and advise on data migration testing strategies and data quality assurance strategies for Resillion customers, ensuring a smooth transition of data to new customer systems. Design, develop, and implement scalable data pipelines using cloud-based data engineering tools and technologies, with a focus on both Microsoft Azure solutions (e.g., Azure Data Factory, Azure Databricks) and Google Cloud Platform (GCP) solutions (e.g., Google Cloud Dataflow, Google Cloud Dataproc). Write efficient and maintainable code to extract, transform, and load data from various sources, leveraging your expertise in Azure Data Lake Storage and other relevant Azure services, as well as Google Cloud Storage and other relevant GCP services. Build and manage data warehouses and data lakes for quality engineering data, utilizing your knowledge of Azure Synapse Analytics or similar technologies, and Google BigQuery or similar technologies. Develop and implement data quality checks and monitoring procedures to ensure data integrity using Azure Data Catalog or other appropriate tools, as well as Google Cloud Data Catalog or other appropriate tools. Automate data engineering tasks and workflows using Azure automation tools and GCP automation tools (e.g., Cloud Functions, Cloud Composer). Set up quality intelligence dashboards for quality assurance data using Microsoft Power BI to provide stakeholders with clear and actionable insights. Stay up-to-date with the latest data engineering tools and technologies, including advancements in AI, Machine Learning (MLOps), and generative AI for data processing in both the Azure and GCP environments. Advise on and implement test data generation strategies and solutions for various testing needs. Qualifications Minimum 7+ years of experience in data engineering or a related field Proven experience in designing, developing, and deploying data pipelines
