Sr. Data Engineer - Pune

As a Data Engineer in Pune, your responsibilities will include designing, implementing, and optimizing end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. You will be developing data pipelines to extract and transform data in near real-time using cloud-native technologies. Implementing data validation and quality checks to ensure accuracy and consistency will also be part of your role. Monitoring system performance, troubleshooting issues, and implementing optimizations to enhance reliability and efficiency will be crucial tasks. Collaboration with business users, analysts, and other stakeholders to understand data requirements and deliver tailored solutions is an essential aspect of this position. Documentation of technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system documentation will be expected. Providing technical guidance and support to team members and stakeholders as needed will also be a key responsibility. Desirable competencies for this role include having 8+ years of work experience, proficiency in writing complex SQL queries on MPP systems such as Snowflake or Redshift, experience in Databricks and Delta tables, data engineering experience with Spark, Scala, or Python, familiarity with the Microsoft Azure stack including Azure Storage Accounts, Data Factory, and Databricks, experience in Azure DevOps and CI/CD pipelines, working knowledge of Python, and being comfortable participating in 2-week sprint development cycles.,