Data Engineer

Data Engineer

Roles & Responsibilities:

• Data Pipeline Design & Implementation

  : Design and implement scalable data pipelines for feature extraction, transformation, and loading (

  ETL

  ) using technologies such as

  Pyspark, Scala

  , and relevant big data frameworks.

• Data Governance & Optimization

  : Govern and optimize data pipelines to ensure high reliability, efficiency, and data quality across on-premise and cloud environments.

• Cross-functional Collaboration

  : Collaborate closely with data scientists, ML engineers, and business stakeholders to understand data requirements and translate them into robust technical solutions.

• Best Practices & Compliance

  : Implement best practices for data governance, metadata management, and compliance with regulatory requirements.

• Team Leadership

  : Lead a team of data engineers, providing technical guidance, mentorship, and fostering a culture of innovation and collaboration.

• Continuous Improvement

  : Stay updated with industry trends and advancements in big data technologies, contributing to the continuous improvement of data engineering practices.

Skills Required:

• Strong experience in data engineering with hands-on experience in designing and implementing data pipelines.
• Strong proficiency in programming languages such as

  Pyspark and Scala

  , with experience in big data technologies (

  Cloudera, Hadoop ecosystem

  ).
• Proven leadership experience in managing and mentoring a team of data engineers.
• Excellent communication skills with the ability to collaborate effectively across teams and stakeholders.
• Experience working in a banking or financial services environment is a plus.

QUALIFICATION:

• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field, or equivalent practical experience.