Hadoop Data Engineer / Senior Software Engineer

Hsbc

5 - 10 years

15 - 17 Lacs

pune

Posted:19 hours ago| Platform:

Apply

Skills Required

performance tuning performance testing agile control-m data quality unit testing financial services sql python technical documentation

Work Mode

Work from Office

Job Type

Full Time

Job Description

Expertise in Scala-Spark/Python-Spark development and should be able to Work with Agile application dev team to implement data strategies.
Design and implement scalable data architectures to support the banks data needs.
Develop and maintain ETL (Extract, Transform, Load) processes.
Ensure the data infrastructure is reliable, scalable, and secure.
Oversee the integration of diverse data sources into a cohesive data platform.
Ensure data quality, data governance, and compliance with regulatory requirements.
Monitor and optimize data pipeline performance.
Troubleshoot and resolve data-related issues promptly.
Implement monitoring and alerting systems for data processes.
Troubleshoot and resolve technical issues optimizing system performance ensuring reliability.
Create and maintain technical documentation for new and existing system ensuring that information is accessible to the team.
Implementing and monitoring solutions that identify both system bottlenecks and production issues.

Requirements

Experience in data engineering or related field and hands-on experience of building and maintenance of ETL Data pipelines
Good experience in Designing and Developing Spark Applications using Scala or Python.
Good experience with database technologies (SQL, NoSQL), data warehousing solutions, and big data technologies (Hadoop, Spark)
Proficiency in programming languages such as Python, Java, or Scala.
Optimization and Performance Tuning of Spark Applications
GIT Experience on creating, merging and managing Repos.
Perform unit testing and performance testing.
Good understanding of ETL processes and data pipeline orchestration tools like Airflow, Control-M.
Strong problem-solving skills and ability to work under pressure.
Excellent communication and interpersonal skills.

More Jobs at Hsbc

Consultant Specialist

Pune

4 - 9 yrs

INR 22 - 27 Lacs

Consultant Specialist

Pune

7 - 12 yrs

INR 22 - 27 Lacs

Agile, Scrum/Consultant Specialist

Pune

8 - 10 yrs

INR 13 - 14 Lacs

Appian Development/Senior Consultant Specialist

Pune

5 - 10 yrs

INR 8 - 13 Lacs

Senior Audit Manager - DBS Ops (Mumbai and Gurgaon)

Pune

5 - 10 yrs

INR 8 - 12 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.