Data Engineer

Immediate Joiners preferred! or Max 3 - 4 weeks.

Seeking a skilled Data Engineer with strong experience in SQL coding, Spark development, and Python or Scala programming. The role involves building and optimizing data pipelines, working with large-scale distributed systems, and managing data workflows across Hadoop and HDFS environments. The ideal candidate is hands-on, detail oriented, and comfortable working in a hybrid offshore setup.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL workflows.

• Write efficient and optimized SQL code for data extraction, transformation, and analysis.

• Develop Spark applications using Python or Scala for large-scale data processing.

• Work with Hadoop ecosystem components including HDFS, Hive, and related tools.

• Perform data validation, quality checks, and troubleshooting of pipeline issues.

• Collaborate with onshore and offshore teams to understand data requirements and deliver solutions.

• Optimize data storage, retrieval, and performance across distributed systems.

• Maintain clear documentation and follow best practices for coding, version control, and deployment.

Required Skills & Experience:

• 5 years of experience as a Data Engineer or in a similar role.

• Strong SQL coding and query optimization expertise.

• Hands-on experience with Apache Spark (Python or Scala).

• Solid understanding of Hadoop ecosystem: HDFS, Hive, YARN, MapReduce, etc.

• Experience with distributed computing, data pipelines, and ETL frameworks.

• Familiarity with CI/CD, version control (Git), and agile development.

• Strong analytical and problem-solving skills.

Preferred Qualifications:

• Experience with cloud platforms (AWS, Azure, or GCP) is a plus.

• Knowledge of Kafka, Airflow, or similar orchestration/streaming tools.

• Exposure to data warehousing concepts.