Big Data Engineering - Manager

8 - 9 years

14 - 18 Lacs

Pune

Posted:2 months ago| Platform: Naukri logo

Apply

Skills Required

Product management Automation Manager Quality Assurance Database design Information security Billing Performance testing Analytics SQL Python

Work Mode

Work from Office

Job Type

Full Time

Job Description

This position will lead a team of employees and contractors in their career growth Mentorship, problem resolution and team building will be essential to succeed in this role This position will also be responsible for leading software engineering projects in developing high quality, secure, scalable projects & pipeline related applications based on technical/functional requirements specifications and design artifacts within expected time and budget Responsible for assuring the daily activities of the team and the billing meet the requirements of SLA s Team members may be developers, testers, Product Managers, & architects The team is cross-functional, which means that between all its members they possess sufficient skills to do the work Provide ongoing guidance to teams on professionalism, career growth, development standards, automation and tools and processes such as Agile, Lean, etc Work with Product Management and Engineering to translate product use cases and requirements to industry-leading design, architecture and implementations Influence the product owner to balance the business and technical priorities and the value of the architecture work Responsible for end-to-end development of full-stack Spark Scala including programming, unit testing, integration testing, documentation, maintenance and support of systems application software in adherence with Mastercard standards, processes and best practices Execute and own the baseline architecture implementations and design activities collaborating with other Engineers & Engineering Team Collaborate with business and technical partners to identify and scope the opportunities, outline potential value and ROI, identify risks, benefits and constraints Work with project team to meet scheduled due dates, while identifying emerging issues and recommending solutions for problems and independently perform assigned tasks Document software programs as per software development best practices Follow Mastercard quality assurance and quality control processes, managing security best practices in product code, design, infrastructure and architecture Support defect management, code versioning, and code deployments Ensure robust test coverage for full-stack applications including unit testing and support integration and performance testing Collaborate with data engineers to optimize ETL pipelines, data processing workflows, and database design for analytics needs Collaborate with other DWH team to identify the risk in the data availability, mitigate the risk and ensure timely decision on Billing cycle Perform feature/enhancement estimates, demonstrations and related activities Support level 2 & 3 production problem management including root cause identification and resolution Develop overall strategic technical plans, including recommendations for both software engineering and data engineering solutions to address identified gaps Regularly communicate with management about risk analysis and design trade-offs Participate & contribute in Product kick off sessions & Requirement workshops for Direction and course correction w r t domain based solutions, technology feasibility, competence requirements, resources requirements Participate in development and code review of key functionalities All About You Essential Knowledge, Skills, and Experience Proven experience leading full-stack software development teams in a geographically distributed and/or remote work environment Track record demonstrating the ability to build a diverse and strong team - attracting, hiring, retaining, motivating the best talent Proven expertise in leading and driving real-time data pipelines and large-scale distributed systems Hands-on experience with tools such as Hadoop, Apache Spark, Kafka, NiFi, and Big Data ecosystems Strong knowledge of data security, compliance, and data governance standards Proficiency in programming languages such as Java, Python, or Scala Experience providing innovative analytical solutions that have lasting impact in multiple dimensions; strategic, operational, and financial Knowledge of data science concepts, including working with structured and unstructured data and delivering analytics-driven features Creative and nimble with ability to overcome obstacles to solve hard problems Flexible and responsive, with ability to adapt to rapid change in direction or business priority Openness to feedback and strong self-awareness Ability to work within the confines of data governance and privacy standards Strong problem-solving abilities involving integration between application logic, data pipelines, and visualization Desirable Proven expertise in leading and driving real-time data pipelines and large-scale distributed systems Minimum 8 years of experience in Data & Software Engineering with 2-4 years of experience in Managing and leading the team Hands-on experience with tools such as Hadoop, Apache Spark, Kafka, NiFi, and Big Data ecosystems along with SQL & Shell Scripts Strong knowledge of data security, compliance, and data governance standards Proficiency in programming languages such as Java, Python, or Scala Experience with microservices architecture and messaging frameworks (e g , Kafka, RabbitMQ) Extensive familiarity with cloud platforms and container orchestration tools (Kubernetes, Docker) Experience with Agile/Scrum environment preferred Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must: Abide by Mastercard s security policies and practices Ensure the confidentiality and integrity of the information being accessed Report any suspected information security violation or breach, and Complete all periodic mandatory security trainings in accordance with Mastercard s guidelines

Mock Interview

Practice Video Interview with JobPe AI

Start Product Management Interview Now
Dynamic Yield By Mastercard
Dynamic Yield By Mastercard

Technology, Marketing Technology, Personalization

Los Angeles

200+ Employees

424 Jobs

    Key People

  • Liad Agmon

    CEO
  • Adrian B. K. Klok

    CTO

RecommendedJobs for You