390 Apache Nifi Jobs

Role & responsibilities Experience working with NoSQL in at least one of the data stores - HBase, Cassandra, MongoDB Experience in HDFS, Hive, Impala Must have strong technical skills in Neo4j. Experience is schedulers like Airflow, Nifi etc. Experienced in Hadoop clustering and Auto scaling. Develop standardized practices for delivering new products and capabilities using Big Data technologies, including data acquisition, transformation, and analysis. Define and develop client specific best practices around data management within a Hadoop environment on Azure cloud Monitoring health, tuning, and growth of such on-premises and cloud databases Experience in managing security and maintenance o...

As a Senior Big Data Developer with expertise in PySpark and the Apache Big Data ecosystem, your role will involve designing, developing, and optimizing scalable data pipelines. You will collaborate with cross-functional teams to deliver reliable, high-performance data solutions. Your responsibilities will include: - Building and optimizing large-scale data processing pipelines using PySpark - Working with Apache Hadoop, Hive, HDFS, Spark, and other tools for managing big data workloads - Handling ETL development, data ingestion, transformation, and integration from various sources - Collaborating with Data Scientists, Analysts, and business stakeholders to deliver data-driven solutions - En...

Title and Summary Senior Software Engineer - (Full Stack-Java, React.Js) Overview --- Are you a passionate software engineer looking to take on a senior contributor role in a fast-paced environment? Do you thrive on solving complex problems and driving technical excellence? This is a great opportunity to join the team behind Crypto Secure, a powerful risk assessment tool that helps banks gain greater visibility of crypto spend, transaction volumes, and AML risk exposure through a simple, intuitive dashboard. As a Senior Software Engineer, you will play a pivotal role in shaping the technical direction of the product, ensuring high-quality code, and mentoring engineers. Youll be hands-on in d...

Title and Summary Manager Data Scientist About Mastercard: Mastercard is a leading global technology company in the payments industry. Our mission is to connect and empower an inclusive, digital economy that benefits everyone, everywhere, by making transactions safe, simple, smart, and accessible. Leveraging secure data, robust networks, strategic partnerships, and our commitment to innovation, we enable individuals, financial institutions, governments, and businesses to realise their full potential. With a presence in over 210 countries and territories, we are dedicated to building a sustainable world that unlocks priceless possibilities for all, driven by our core value of decency. _______...

Create data pipelines for more efficient and repeatable data science projects Design and implement data architecture solutions that support business requirements and meet organizational needs Collaborate with stakeholders to identify data requirements and develop data models and data flow diagrams Work with cross-functional teams to ensure that data is integrated, transformed, and loaded effectively across different platforms and systems Develop and implement data governance policies and procedures to ensure that data is managed securely and efficiently Develop and maintain a deep understanding of data platforms, technologies, and tools, and evaluate new technologies and solutions to improve...

Overview Design and implement complex ETL/ELT pipelines using PySpark and Airflow for large-scale data processing on GCP.Lead data migration initiatives, including automating the movement of Teradata tables to BigQuery, ensuring data accuracy and consistency. Develop robust frameworks to streamline batch and streaming data ingestion workflows, leveraging Kafka, Dataflow, and NiFi. Collaborate with data scientists to build ML-ready data layers and support analytics solutions. Conduct proof of concepts (POCs) and document performance benchmarking for data throughput and velocity, ensuring optimized data workflows. Enhance CI/CD pipelines using Jenkins and GitLab for efficient deployment and mo...

We are looking for a skilled Business Technology Solutions Consultant with 5-10 years of experience to join our team in India. The ideal candidate will have a strong background in Big Data Development and be able to work on innovative solutions. Roles and Responsibility Design, develop, and implement Big Data solutions using various technologies. Collaborate with cross-functional teams to identify business requirements and develop technical solutions. Develop and maintain large-scale data processing systems and pipelines. Ensure data quality, integrity, and security in all developed systems. Troubleshoot and resolve complex technical issues related to Big Data development. Stay updated with ...

Lead Data Engineer-2 - Apache Nifi, Python, PySpark, Hadoop, Cloudera platforms, and Airflow Overview Mastercard is looking for a Senior Software Engineer to join the Account Level Management(ALM) team in our Pune office thats focused on building ALM Services with Data warehouse skills. The Mastercard Account Level Management platform empowers real-time card level decisioning. As consumers progress along their life stages as card holders, with increasing disposable income and more refined preferences, ALM provides services to issuers so they can effectively offer more relevant benefits and rewards at each stage, to drive loyalty and spend. Role Develop high quality, secure and scalable data ...

Role Description This is a full-time hybrid role for an Apache Nifi Developer based in Gurugram with some work-from-home options. The Apache Nifi Developer will be responsible for designing, developing, and maintaining data workflows and pipelines. The role includes programming, implementing backend web development solutions, using object-oriented programming (OOP) principles, and collaborating with team members to enhance software solutions. Qualifications Knowledge of Apache Nifi and experience in programming Skills in Back-End Web Development and Software Development Data Pipeline Strong understanding of APACHE NIFI Background in Computer Science Excellent problem-solving and analytical s...

As an experienced Data Engineer with 6+ years of experience, your role will involve: - Designing and implementing modern data lakehouse architectures (Delta Lake, or equivalent) on cloud platforms like AWS, Azure, or GCP. - Defining data modeling, schema evolution, partitioning, and governance strategies for high-performance and secure data access. - Owning the technical roadmap for scalable data platform solutions, aligned with enterprise needs and future growth. - Providing architectural guidance and conducting code/design reviews across data engineering teams. - Building and maintaining reliable, high-throughput data pipelines for ingestion, transformation, and integration of structured, ...

Title and Summary Senior Data Scientist, Product Data & AnalyticsSenior Data Scientist, Product Data & Analytics Our Vision:Product Data & Analytics team builds internal analytic partnerships, strengthening focus on the health of the business, portfolio and revenue optimization opportunities, initiative tracking, new product development and Go-To Market strategies. We are a hands-on global team providing scalable end-to-end data solutions by working closely with the business. We influence decisions across Mastercard through data driven insights. We are a team on analytics engineers, data architects, BI developers, data analysts and data scientists, and fully manage our own data assets and so...

Overview --- Are you a passionate software engineer looking to take on a leadership role in a fast-paced environment? Do you thrive on solving complex problems and driving technical excellence? This is a great opportunity to join the team behind Crypto Secure, a powerful risk assessment tool that helps banks gain greater visibility of crypto spend, transaction volumes, and AML risk exposure through a simple, intuitive dashboard. As a Lead Software Engineer, you will play a pivotal role in shaping the technical direction of the product, ensuring high-quality code, and mentoring engineers. Youll be hands-on in development while also leading architecture discussions, removing technical roadbloc...

Summary Overview Enterprise Data Solution (EDS) is focused on enabling insights into Mastercard network and help build data-driven products by curating and preparing data in a secure and reliable manner. Moving to a Unified and Fault-Tolerant Architecture for Data Ingestion and Processing is critical to achieving this mission. As a Software Development Engineer (Data Engineering) in Enterprise Data Solution (EDS), you will have the opportunity to build high performance data pipelines to load into Mastercard Data Warehouse. Our Data Warehouse provides analytical capabilities to number of business users who help different customers provide answer to their business problems through data. You wi...

Role Overview: We are seeking a Data Migration Expert with deep expertise in architecting and executing complex data migrations across hybrid and cloud ecosystems. This is not a DBA role this is a hands-on migration architect who speaks the language of data engineering and thrives on solving high-scale data movement challenges with precision and agility. Key Responsibilities: - Design and implement large-scale data migration strategies across on-prem, hybrid, and cloud environments. - Build and orchestrate real-time data pipelines using CDC and ETL frameworks. - Ensure seamless schema evolution, data lineage tracking, and zero-downtime transfers. - Collaborate with cross-functional teams to ...

Project Role :Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Python (Programming Language) Good to have skills : NAMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, and ensuring that applications function seamle...

Project Role :Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Python (Programming Language) Good to have skills : NAMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, and ensuring that applications function seamless...

As an Integration Specialist at our offshore development center, you will be supporting one of the top banks in the UAE. Your primary expertise should lie in Apache NiFi, Java, and Spring Boot. Here's what we are looking for in a candidate: **Key Responsibilities:** - Minimum 5 years of experience in implementing end-to-end integration solutions using NiFi processors - Minimum 5 years of experience in Java and Spring Boot with Microservices - Minimum 3 years of experience in application security such as SSL Certificates and cryptography - Minimum 2 years of experience in Distributed architecture **Qualifications Required:** - Degree or Post-graduate in Computer Science or related field (or e...

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : O9 Solutions Good to have skills : NAMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing innovative solutions, and ensuring that applications are aligned with business objectives. You will engage in problem-solving activities, parti...

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : O9 Solutions Good to have skills : NAMinimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing innovative solutions, and ensuring that applications are aligned with business objectives. You will engage in problem-solving activities, parti...

Role Overview: As a candidate for this position, you will be responsible for integrating various ERP endpoints and transactional/NoSQL database systems. Your role will involve ensuring data synchronization is maintained within the accepted SLA and TAT. You will closely collaborate with the Product Manager and Engineering Manager to support product and tech enablement. Additionally, you will be creating, executing, monitoring, and maintaining interfaces that support multi-tenancy mode while also performing daily monitoring of applications running on production. Your tasks will include working on new development based on product specifications. Key Responsibilities: - Integrate various ERP end...

Senior DE: Same as above but need to have experience managing data engineers, have experience working with stakeholders, actively engaging for key technical discussion and have a very good technical knowledge. Should have at least 7 years experience working in data engineering. Job Description: As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experie...

Senior DE: Same as above but need to have experience managing data engineers, have experience working with stakeholders, actively engaging for key technical discussion and have a very good technical knowledge. Should have at least 7 years experience working in data engineering. Job Description: As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experie...

Senior DE: Same as above but need to have experience managing data engineers, have experience working with stakeholders, actively engaging for key technical discussion and have a very good technical knowledge. Should have at least 7 years experience working in data engineering. Job Description: As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experie...

Senior DE: Same as above but need to have experience managing data engineers, have experience working with stakeholders, actively engaging for key technical discussion and have a very good technical knowledge. Should have at least 7 years experience working in data engineering. Job Description: As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experie...

Senior DE: Same as above but need to have experience managing data engineers, have experience working with stakeholders, actively engaging for key technical discussion and have a very good technical knowledge. Should have at least 7 years experience working in data engineering. Job Description: As a key member of the technical team alongside Engineers, Data Analysts and Business analysts, you will be expected to define and contribute at a high-level to many aspects of our collaborative Agile development process: Software design, Scala and Python - Spark development experience, automated testing of new and existing components in an Agile, DevOps and dynamic environment. Minimum 1 year experie...