Senior Data Engineer

Mastercard

6 - 11 years

10 - 14 Lacs

pune

Posted:14 hours ago| Platform:

Apply

Skills Required

data engineer python data pipelines scala airflow sla data engineering java apache nifi spark apache spark debugging software engineer hadoop ecosystem hadoop big data data integration apache airflow architecture

Work Mode

Work from Office

Job Type

Full Time

Job Description

Job Overview

The Senior Software Engineer will do the design, development, and delivery of advanced big data solutions within the Hadoop ecosystem.
This role involves architecting and implementing scalable data processing frameworks, ensuring high performance and reliability across distributed systems.
The engineer will research and evaluate emerging technologies to optimize data workflows and meet evolving business needs.
Additionally, the position requires hands-on expertise in Cloudera distribution and related tools, ensuring seamless orchestration and integration of data pipelines.
The Senior Software Engineer will also be responsible for maintaining operational excellence, meeting SLA requirements, and mentoring team members to drive technical innovation and best practices.

Role

Contribute individually to the design, development, and implementation of large-scale data engineering solutions within the Hadoop ecosystem, ensuring high performance, scalability, and reliability.
Drive end-to-end technical delivery for projects involving Cloudera distribution, Apache Ozone, Apache Iceberg, Apache Airflow, Apache NiFi, and Apache Spark.
Collaborate with cross-functional teams to define architecture, technical standards, and best practices for big data platforms and workflows.
Perform rigorous performance benchmarking and optimization for newly developed distributed frameworks, ensuring efficient parallel processing and scalable storage solutions across large datasets.
Research and evaluate emerging technologies in distributed data storage, orchestration, and processing frameworks to optimize data pipelines and improve operational efficiency.
Develop strategic technical plans and create architecture proposals addressing solution gaps and business requirements.
Provide technical leadership in planning, design, and implementation phases, ensuring compliance with security, quality, and performance standards.
Ensure adherence to SLAs, manage production incidents, and drive root cause analysis and remediation for complex issues.
Conduct Proof of Concepts (PoCs) for new technologies and contribute to technology roadmaps for the organization.

Education

Bachelor''s degree in Information Systems, Information Technology, Computer Science or Engineering or equivalent experience.

Knowledge / Experience:

6+ years of experience in related field, including 4-5 years of experience in delivering secure solutions in Financial Services Sector is preferred.
Advanced knowledge and understanding of Software Engineering Concepts and Methodologies is required.
Demonstrate MC Core Competencies.

About You

High-energy, detail-oriented, and proactive, with the ability to work independently under pressure while driving results with initiative and self-motivation.
Strong aptitude for quickly learning and implementing new big data technologies, frameworks, and tools, and performing Proof of Concepts (PoCs) to identify optimal solutions.
6+ years of overall experience in software engineering with 5+ years of hands-on experience in Hadoop ecosystem projects, including Cloudera distribution.
Proven expertise in building distributed data pipelines and frameworks using Apache Spark (Scala/Python/Java) for massive parallel processing.
Hands-on experience with Apache Ozone and Apache Iceberg for distributed storage and table formats.
Skilled in designing and managing orchestration workflows using Apache Airflow and Apache NiFi for complex data integration scenarios.
Strong analytical and problem-solving skills for debugging production issues, performing root cause analysis, and implementing mitigation plans.
Excellent communication skillsboth verbal and writtenwith strong collaboration and organizational abilities to work in diverse, geographically distributed teams.
Ability to multitask across multiple projects, provide technical leadership, and mentor junior team members.
Flexible and adaptable to evolving technologies and business needs, with a passion for innovation in big data solutions.

More Jobs at Mastercard

Senior Analyst - Project Management

Pune

5 - 9 yrs

INR 13 - 18 Lacs

Sr. Principal Software Engineer

Pune

9 - 13 yrs

INR 20 - 25 Lacs

Lead Software Engineer-R-245726

Vadodara, Gujarat, India

2 - 4 yrs

Salary: Not disclosed

Consultant-2

Gurugram, Haryana

Experience: Not specified

Salary: Not disclosed

Consultant-2

Gurugram, Haryana

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Data Engineer Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.