Data Engineer

Job Title :

Location:

Experience:

Role Overview

Data Engineer

Key Responsibilities:

• Design, develop, and maintain scalable big data pipelines and architectures to support business and analytical needs
• Build and optimize data processing systems using technologies such as Hadoop, Hive, Spark, and Python/Java
• Implement data governance principles, ensuring data quality, integrity, and security across all data layers.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Integrate and manage data from diverse sources including SQL, NoSQL databases, APIs, and streaming data.
• Design and maintain real-time data streaming and message queuing systems (e.g., Kafka, Spark Streaming, Storm).
• Conduct data profiling, cleansing, and validation to ensure accuracy and reliability.
• Continuously improve system performance, scalability, and efficiency through automation and optimization.

Qualification:

• Bachelors or Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or a related quantitative field.

• 4+ years

  of hands-on experience as a Data Engineer or in a related data engineering
• role.
• Proven expertise with big data technologies such as

  Hadoop, Hive, Spark, and Kafka.

• Strong programming skills in Python, Java, or Scala
• Proficiency with SQL and NoSQL databases (e.g., MySQL, PostgreSQL, Cassandra).
• Working knowledge of stream-processing frameworks such as Spark Streaming or Storm.
• Strong analytical and problem-solving skills with the ability to design solutions for complex business use cases.
• Excellent project management and organizational skills with experience working in dynamic, cross-functional environments.

Kindly share your resume at kashishmalik@policybazaar.com