Good co India logo

Lead Data Engineer

Good co India

5 - 10 years

25 - 40 Lacs

India

Posted:4 days ago

Job Description

Role & responsibilities

  • Lead the design, development, and implementation of scalable, reliable, and high-performance data engineering solutions.
  • Architect, build, and maintain robust ETL/ELT pipelines for batch and real-time data processing.
  • Lead the development and optimization of data warehouses, data lakes, and lakehouse architectures.
  • Design and manage data ingestion, transformation, integration, and orchestration workflows.
  • Work with large-scale datasets using Apache Spark, PySpark, Kafka, Hadoop, and other big data technologies.
  • Implement and manage cloud-based data platforms using AWS, Microsoft Azure, or Google Cloud Platform.
  • Develop and optimize data models, database schemas, and data processing frameworks.
  • Ensure data quality, consistency, security, governance, and regulatory compliance across data platforms.
  • Implement workflow orchestration and automation using Apache Airflow, dbt, and related tools.
  • Monitor data pipelines, troubleshoot failures, and optimize system performance and infrastructure costs.
  • Establish data engineering standards, best practices, documentation, and CI/CD processes.
  • Collaborate with data scientists, data analysts, software engineers, and business stakeholders to deliver data-driven solutions.
  • Lead technical discussions, architecture reviews, code reviews, and technology evaluations.
  • Mentor junior and mid-level data engineers and provide technical guidance to the engineering team.
  • Drive continuous improvements in data infrastructure, scalability, reliability, and operational efficiency.

Preferred candidate profile

  • Bachelor's or Master's degree in Computer Science, Information Technology, Data Science, Software Engineering, or a related technical field.
  • 5 to 10 years of professional experience in data engineering, big data development, or data platform engineering.
  • Strong proficiency in Python, SQL, PySpark, and experience with large-scale data processing.
  • Hands-on experience designing and developing complex ETL/ELT pipelines and data integration workflows.
  • Strong knowledge of Apache Spark, Kafka, Airflow, Hadoop, and distributed data processing frameworks.
  • Experience with modern data platforms such as Snowflake, Databricks, Amazon Redshift, Google BigQuery, or Azure Synapse Analytics.
  • Proven expertise in cloud technologies, including AWS, Azure, or Google Cloud Platform.
  • Strong understanding of data warehousing, data lakes, lakehouse architecture, data modeling, and database optimization.
  • Experience with real-time streaming, batch processing, and workflow orchestration.
  • Knowledge of data governance, data quality frameworks, metadata management, and data security practices.
  • Experience with Docker, Kubernetes, Git, CI/CD pipelines, Terraform, and infrastructure automation.
  • Ability to lead technical projects, define data architecture, and establish engineering best practices.
  • Proven experience mentoring engineers, conducting code reviews, and collaborating with cross-functional teams.
  • Strong analytical, problem-solving, communication, and stakeholder-management skills.
  • Ability to translate business requirements into scalable and efficient data engineering solutions.

Skills Required

Data warehousing SQL Modeling Data Pipeline Architecture Advance SQL Integration Data architecture Hadoop

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Remote

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Good co India logo
Good co India

RecommendedJobs for You

Accord Infosystems logo

Lead Data Engineer

Accord Infosystems

Mumbai

Ecolab logo

Bengaluru

Aptean logo

Bengaluru

Hiringhood logo

Hyderabad

Echidna Software logo

Lead Data Engineer

Echidna Software

Bengaluru

Delta System & Software, Inc. logo

Lead Data Engineer

Delta System & Software, Inc.

Chennai

T

India

Neurealm logo

India