Senior Data Engineer II

3 - 8 years

5 - 10 Lacs

Posted:1 hour ago| Platform: Naukri logo

Apply

Work Mode

Work from Office

Job Type

Full Time

Job Description

We are looking for a skilled Data Engineer to join our growing team. You will play a pivotal role in designing, building, and maintaining our data infrastructure and pipelines. Working closely with data analysts, data scientists, and various business stakeholders, you ll ensure data is reliable, efficient, and accessible in a scalable manner.

Key Responsibilities:

  • Design, build, and maintain end-to-end data pipelines using Airflow (including managed services like Amazon MWAA) to orchestrate, schedule, and monitor batchstreaming workflows.
  • Implement DAGs (Directed Acyclic Graphs) with retry logic, error handling, and alerting to ensure data quality and pipeline reliability.
  • Integrate data from various sources using Airbyte for ingestion and dbt for transformations in a scalable and modular fashion.
  • Collaborate with Data Analysts and Data Scientists to implement transformations and business logic, ensuring data is analytics-ready.
  • Design and implement efficient data models for both structured and semi-structured data in AWS S3 (data lake) and Snowflake (data warehouse).
  • Ensure data schemas and transformations support advanced analytics, BI reporting, and machine learning use cases.
  • Ingestion
  • Utilize AWS Lake Formation APIs and best practices to maintain data security, access controls, and compliance.
  • Work closely with IT security to establish robust encryption standards, audit trails, and identityrole-based access.
  • Optimize AWS Athena queries and configurations (e.g., data partitioning) for performance and cost efficiency.
  • Monitor and tune Airflow DAGs, Snowflake queries, and data transformations to improve throughput and reliability.
  • Partner with cross-functional teams, including DevOps, Platform Engineering, and Data Science, to ensure seamless integration of data workflows and systems.
  • Communicate technical solutions effectively to non-technical stakeholders and leadership, translating requirements into actionable tasks.
  • Participate in architecture reviews, code reviews, and troubleshooting sessions to ensure quality and alignment with best practices.
  • Remain current with emerging trends in data engineering, orchestration tools (Airflow, MWAA), and cloud services (AWS, Snowflake).

Qualifications & Skills:

Education & Experience

  • Bachelor s or Master s degree in Computer Science, Engineering, or a related field.
  • 3+ years of experience as a Data Engineer or in a similar role working with cloud-based data platforms.
  • Technical Skills
  • Cloud & Orchestration:
  • Airflow (self-managed or managed services like Amazon MWAA) for workflow orchestration, DAG development, and scheduling.
  • Familiarity with best practices for Airflow DAG structure, dependency management, and error handling.
  • AWS Expertise:
  • Hands-on experience with AWS Lake Formation, S3, Athena, and related services (e.g., Lambda, Glue, IAM).
  • Snowflake:
  • Proficient in setting up data warehouses, configuring security, and optimizing queries on Snowflake.
  • Data & Transformation:
  • Experience with Airbyte or similar tools for data ingestion.
  • dbt or other SQL-based transformation frameworks for modular data processing.
  • Programming:
  • Proficiency in Python andor JavaScala for building data pipelines and custom integrations.
  • Query Languages:
  • Advanced knowledge of SQL for data manipulation and analysis.

Soft Skills

  • Strong problem-solving and analytical abilities.
  • Excellent communication and collaboration skills, able to effectively work in cross-functional teams.
  • Ability to operate in a fast-paced, agile environment and manage multiple priorities simultaneously.
  • AWS certifications (e.g., AWS Certified Data Analytics Specialty, AWS Certified Solutions Architect) are a plus.
  • Experience with CICD pipelines, containerization (Docker, Kubernetes), and infrastructure as code (Terraform, CloudFormation) is beneficial.
  • Familiarity with DevOps best practices for managing Airflow environments (e.g., version control for DAGs, automated testing, monitoring).

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Alter Domus logo
Alter Domus

Financial Services

Luxembourg Watermael-Boitsfort

RecommendedJobs for You

bengaluru, karnataka, india

bengaluru, karnataka, india

bengaluru, karnataka, india