Data Engineer

Job Title: Data Engineer

About the Role

Were seeking a Data Engineer with 3–4 years of experience to join our growing tech team. The ideal candidate will have hands-on experience in building and managing scalable data systems on AWS, with strong expertise in data pipelines, real-time streaming, and modern data frameworks.in building and managing scalable data systems on AWS using Spark and modern data frameworks.

Key Responsibilities

• Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of high-volume data.
• Develop Spark-based ETL/ELT workflows for both batch and real-time streaming data.
• Core Skills: Spark, AWS (S3, Glue, Redshift, Kinesis), Kafka, Airflow, Python, SQL
• Integrate data from multiple internal and external systems using Kafka, Kinesis, or other streaming frameworks.
• Build and manage data models, warehouses, and lakehouses using AWS services such as S3, Glue, Redshift, Athena, etc.
• Implement data quality checks, validation rules, and monitoring to ensure reliability and consistency.
• Collaborate with data analysts and scientists to provide clean, structured datasets optimized for analytics and ML.
• Work with orchestration tools (Airflow, MWAA, Step Functions, etc.) for automated workflow scheduling.
• Continuously optimize data pipelines for cost, scalability, and performance.

Required Skills

• Strong programming skills in Python for data manipulation and automation.
• Hands-on expertise in Apache Spark (PySpark or Spark SQL) for large-scale data processing.
• Deep understanding of AWS data ecosystem — S3, Glue, Lambda, Redshift, Athena, EMR, Kinesis, IAM.
• Experience with real-time streaming platforms such as Kafka, Kinesis, or Flink.
• Strong command of SQL and data modeling (star schema, dimensional modeling, partitioning).
• Proficiency with data orchestration and workflow management tools (Airflow, Step Functions, etc.).
• Familiarity with Git, CI/CD, and modern development best practices.
• Experience working in Linux/Unix environments and handling large datasets efficiently.

Good to Have

• Exposure to data lakehouse technologies (Delta Lake, Iceberg, Hudi).
• Understanding of data governance, cataloging, and lineage tools (Glue Data Catalog, Amundsen, DataHub).
• Familiarity with containerization and deployment (Docker, ECS, EKS).
• Basic understanding of AI/ML

Why Join Us

• Work with a modern AWS-based data stack using Spark, Kafka, and scalable storage systems.
• Opportunity to contribute to real-time analytics and AI-driven products.
• Collaborative, high-impact environment with a focus on learning, innovation, and ownership.
• Competitive compensation, flexible working model, and strong career growth path toward Lead Data Engineer roles.
  

Qualifications

• Bachelor’s degree in Computer Science, IT, Engineering, or related technical field.
• 3–4 years of experience in data engineering, big data, or analytics infrastructure.

Location: [Mumbai / Pune / Goa]