Senior Data Engineer

7 - 10 years

9 - 12 Lacs

Posted:4 hours ago| Platform: Naukri logo

Apply

Work Mode

Work from Office

Job Type

Full Time

Job Description

Job Description

Senior Data Engineer - Quality Engineering

Experience Range: 7-10 Years

Location: Bangalore (Hybrid Mode)

Resillion, a leading quality engineering company with offices around the world, is seeking a talented Data Engineer to join our growing India team. In this role, you will play a critical part in building and maintaining the data infrastructure that supports our AI-powered testing tools and analytics solutions. You will have technical responsibility for the entire data lifecycle, from data acquisition and ingestion to storage, processing, and analysis.
Responsibilities:
  • Collaborate with GenAI engineers, Software Engineers, Test automation engineers and other stakeholders within Resillion to understand data needs and translate them into technical solutions, including designing data pipelines for training & deploying models and data pre-processing for AI, including generative AI applications.
  • Design, implement, and advise on data migration testing strategies and data quality assurance strategies for Resillion customers, ensuring a smooth transition of data to new customer systems.
  • Design, develop, and implement scalable data pipelines using cloud-based data engineering tools and technologies, with a focus on both Microsoft Azure solutions (e.g., Azure Data Factory, Azure Databricks) and Google Cloud Platform (GCP) solutions (e.g., Google Cloud Dataflow, Google Cloud Dataproc).
  • Write efficient and maintainable code to extract, transform, and load data from various sources, leveraging your expertise in Azure Data Lake Storage and other relevant Azure services, as well as Google Cloud Storage and other relevant GCP services.
  • Build and manage data warehouses and data lakes for quality engineering data, utilizing your knowledge of Azure Synapse Analytics or similar technologies, and Google BigQuery or similar technologies.
  • Develop and implement data quality checks and monitoring procedures to ensure data integrity using Azure Data Catalog or other appropriate tools, as well as Google Cloud Data Catalog or other appropriate tools.
  • Automate data engineering tasks and workflows using Azure automation tools and GCP automation tools (e.g., Cloud Functions, Cloud Composer).
  • Set up quality intelligence dashboards for quality assurance data using Microsoft Power BI to provide stakeholders with clear and actionable insights.
  • Stay up-to-date with the latest data engineering tools and technologies, including advancements in AI, Machine Learning (MLOps), and generative AI for data processing in both the Azure and GCP environments.
  • Advise on and implement test data generation strategies and solutions for various testing needs.

Qualifications
  • Minimum 7+ years of experience in data engineering or a related field
  • Proven experience in designing, developing, and deploying data pipelines
  • Strong understanding of data warehousing, data lakes, and data modelling concepts
  • Expertise in SQL and scripting languages (e.g., Python)
  • Experience with cloud platforms (specifically Microsoft Azure and Google Cloud Platform) and their data engineering services (Azure Data Factory, Databricks, Data Lake Storage, Synapse Analytics and Google Cloud Dataflow, Dataproc, Cloud Storage, BigQuery, etc.)
  • Experience working with data formats commonly used in software development (e.g., logs, code repositories) is a plus
  • Experience with data migration strategies and data migration testing
  • Experience in defining and implementing data testing strategies and data quality assurance processes, particularly in data migration projects.
  • Experience in advising on and implementing test data generation techniques and tools.
  • Experience with MLOps practices and tools for deploying and monitoring machine learning models.
  • Experience with generative AI concepts and their application in data processing and analysis.
  • Experience with Microsoft Power BI for data visualization
  • Excellent communication and collaboration skills
  • Ability to work independently and as part of a team

Mock Interview

Practice Video Interview with JobPe AI

Start Machine Learning Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You