393 Apache Nifi Jobs - Page 4

About The Role Project Role : Cloud Platform Engineer Project Role Description : Designs, builds, tests, and deploys cloud application solutions that integrate cloud and non-cloud infrastructure. Can deploy infrastructure and platform environments, creates a proof of architecture to test architecture viability, security and performance. Must have skills : Splunk Good to have skills : Data Visualization, Data Analytics Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Role:Cloudera SME Development & Integration, Level 10Role Description:We are seeking a skilled Cloudera resource with experience in Cloudera Data Platform (CDP) and hands-on exp...

Role Overview: As a Big Data Engineer at Micron Technology, you will play a crucial role in transforming information into intelligence, inspiring advancements in learning, communication, and innovation. Your responsibilities will include developing ETL/ELT processes, building APIs for application integration, determining transformation requirements, collaborating with Data Scientists, and optimizing the performance of big data and machine learning systems. Key Responsibilities: - Understanding of Big Data Engineering/processing, Business Intelligence, and Advanced analytics - Developing ETL/ELT processes - Knowledge in databases and Data warehouse modeling - Knowledge in Cloud-based data eng...

Overview This position is for Manager Data Engineering in the Commercial Data as a Service group. In this position you will enjoy being responsible for helping define and maintain the data systems key to delivering successful outcomes for our customers. will lead a software development team focused on our core services someone who combines deep technical experience with great people management skills and can help elevate our teams and our approach to engineering. We are building a culture that fosters innovation, built on a foundation of inclusion and diversity. We are looking for a candidate that has proven experience in coding, architecture leadership, and leading engineers in a fast-paced...

Title and Summary Lead Data EngineerOverview We are the global technology company behind the worlds fastest payments processing network. We are a vehicle for commerce, a connection to financial systems for the previously excluded, a technology innovation lab, and the home of Priceless. We ensure every employee has the opportunity to be a part of something bigger and to change lives. We believe as our company grows, so should you. We believe in connecting everyone to endless, priceless possibilities. Our team within Mastercard Services: The Services org is a key differentiator for Mastercard, providing the cutting-edge services that are used by some of the world's largest organizations to mak...

Overview Mastercard is looking for a Senior Software Engineer to join the Account Level Management(ALM) team in our Pune office thats focused on building ALM Services with Data warehouse skills. The Mastercard Account Level Management platform empowers real-time card level decisioning. As consumers progress along their life stages as card holders, with increasing disposable income and more refined preferences, ALM provides services to issuers so they can effectively offer more relevant benefits and rewards at each stage, to drive loyalty and spend. Role o Develop high quality, secure and scalable data pipelines using spark, Scala/Python/Java on Hadoop or object storage like MinIO. o Leverage...

Role Overview: As a Lead Software Test Engineer (Automation Tester) at Mastercard, you will be joining the PVS Identity Solutions team to play a pivotal role in ensuring the quality, performance, and automation of various data-driven pipelines. Your responsibilities will include managing testing infrastructure for Functional test, Automation, and coordinating testing activities across multiple programs and projects. The ideal candidate will have experience in working with large-scale data and automation testing of Java, Cloud Native application/services. Your role will involve leading the development and maintenance of automated testing frameworks, providing technical leadership for new majo...

As a Senior Analyst/Data Engineer in Big Data Analytics & Engineering at Mastercard, your role will involve focusing on data engineering and data unification across various data assets to enable a unified view of data from multiple sources. Your primary responsibility will be to support the development of new innovative data-driven cyber products, services, and actionable insights. - Perform data ingestion, aggregation, and processing to drive relevant insights from available data sets. - Collaborate with different teams such as Product Manager, Data Science, Platform Strategy, and Technology to understand data needs and deliver data solutions that generate business value. - Manipulate and a...

As a Lead Engineer (ML + Data Engineering) in Gurgaon/Jaipur with 6-8 years of experience, your role involves preparing data for model development, testing, and deployment. You will be responsible for designing machine learning systems and AI software to automate predictive models, ensuring accurate user recommendations, and converting unstructured data into useful information. Your tasks will include solving complex problems with multi-layered data sets, optimizing machine learning libraries, and developing ML algorithms for predictive analysis. - Prepare Data followed by Model development, Model Testing, and Deployment. - Design machine learning systems and self-running AI software for aut...

As a NiFi Data Engineer at the company, your role involves designing, developing, and maintaining scalable data pipelines using NiFi. You will ensure the efficient flow and transformation of data to support the platform's analytical needs. Responsibilities: - Bring the guiding principles to life by enhancing experiences for dealerships and customers, collaborating with colleagues, experimenting with new ideas, taking ownership of work, and adapting to growth and changes. - Collaborate with cross-functional teams to understand data requirements and deliver solutions. - Champion data quality and integrity across the organization. - Apply deep technical knowledge of data engineering principles ...

Job Description: As a NiFi Data Engineer at Mindtrilogy, you will be responsible for designing and maintaining scalable data pipelines using Apache NiFi to enable efficient data processing and transformation across systems. You will work closely with cross-functional teams to drive innovative solutions and achieve real business outcomes. Key Responsibilities: - Design and build robust, high-performance data pipelines using Apache NiFi - Write and maintain transformation scripts in Java and Python - Develop and manage ETL/ELT workflows for diverse data sources - Ensure reliable data ingestion, transformation, and distribution processes - Collaborate with Engineers, Developers, and Product Man...

Role & responsibilities Direct Responsibilities Design and develop real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow. Implement and maintain ETL/ELT processes, ensuring data quality and integrity across various data sources. Integrate and manage data flows across diverse systems using NiFi and Kafka topics. Monitor and optimize the performance of data pipelines and troubleshoot issues proactively. Work with the ELK stack (Elasticsearch, Logstash, Kibana) for logging, monitoring, and real-time analytics. Write efficient and optimized SQL queries for data extraction, transformation, and reporting. Collaborate with data analysts and business stake...

Responsibilities What you'll do Engineer, test, document and manage GCP Dataproc, DataFlow and VertexAI services used in high-performance data processing pipelines and Machine Learning. Help developers optimize data processing jobs using Spark, Python, and Java. Collaborate with development teams to integrate data processing pipelines with other cloud services and applications. Utilize Terraform and Tekton for infrastructure as code (IaC) and CI/CD pipelines, ensuring efficient deployment and management. Good to have Experience with Spark for large-scale data processing. Solid understanding and experience with GitHub for version control and collaboration. Experience with Terraform for infras...

As a member of the Natural Language Query (NLQ) platform team, you will be responsible for providing Platform-as-a-Service offerings that are easy to consume, scalable, secure, and reliable using open source-based solutions. You will also play a key role in developing and enhancing the NLQ Copilot platform to meet the evolving needs of multi-tenant environments. Your responsibilities will include implementing Context builder algorithms to generate 100% accurate SQL based on customer needs and collaborating with downstream clients to integrate business requirements. Additionally, you will work closely with data scientists, engineers, and product managers to optimize the performance of the NLQ...

Position Title: System Engineer Senior Location: Chennai Sholinganallur, Elcot Sez, Elcot Sez-600119 Position Type: Full-time Education Required: Bachelors Degree (Computer Science, Engineering, or related field) About the Role We are seeking a Senior System Engineer (Informatica) to join our Analytics Data Management (ADM) team. This is an exciting, fast-paced role for professionals passionate about building analytics platforms and tooling that deliver real business value. As a key member of the team, you will work on engineering strategic data management platforms such as Informatica Enterprise Data Catalog and Apache NiFi , with opportunities to explore other modern tools like IICS/IDMC, ...

Role Overview: You will play a crucial role in the success of the company by designing, developing, and maintaining scalable data pipelines using NiFi. Your main responsibility will be to ensure the smooth flow and transformation of data to support the platform's innovative features and analytical requirements. Key Responsibilities: - Enhance the experiences of dealerships and customers by bringing guiding principles to life. - Collaborate with cross-functional teams to understand data requirements and provide solutions. - Advocate for data quality and integrity across the organization. Qualification Required: - Expertise in building high-performance data pipelines using Apache NiFi, Python,...

Job Title: Data Engineer Role Overview We are seeking a skilled Data Engineer with 510 years of experience to join our dynamic Data Engineering team. The ideal candidate will design, build, and optimize large-scale data pipelines and warehouses that enable real-time analytics and business intelligence across Tanla’s CPaaS ecosystem. You’ll work on diverse datasets, including Call Detail Records (CDR), and collaborate closely with cross-functional teams to ensure data reliability, scalability, and performance. Key Responsibilities: Design and implement ETL/ELT pipelines for large-scale data processing and analytics. Develop and optimize analytical SQL queries for performance and scalability. ...

Overview of the Company: Jio Platforms Ltd. is a revolutionary Indian multinational tech company, often referred to as India's biggest startup, headquartered in Mumbai. Launched in 2019, it's the powerhouse behind Jio, India's largest mobile network with over 400 million users. But Jio Platforms is more than just telecom. It's a comprehensive digital ecosystem, developing cutting-edge solutions across media, entertainment, and enterprise services through popular brands like JioMart, JioFiber, and JioSaavn. Join us at Jio Platforms and be part of a fast-paced, dynamic environment at the forefront of India's digital transformation. Collaborate with brilliant minds to develop next-gen solutions...

Job Description Role Purpose The purpose of the role is to create exceptional architectural solution design and thought leadership and enable delivery teams to provide exceptional client engagement and satisfaction. Job Description: - Data Engineer Roles : Play a large role in the implementation of complex features Lead collaboration with data scientists to understand the existing modeling pipeline and identify optimization opportunities . Oversee the integration and management of data from various sources and storage systems, establishing processes and pipelines to produce cohesive datasets for analysis and modeling. Push the boundaries of analytics and powerful, scalable applications Desig...

You are a highly skilled and experienced Senior Java Developer joining a client team in Vadodara for an exciting opportunity. Your role involves modernizing existing applications by transitioning from a monolithic to a microservices-based architecture, ensuring improved scalability, performance, and resilience. **Key Responsibilities:** - Develop and Maintain Java Applications: Design and deploy scalable Java-based applications using Spring Boot, with production-readiness in focus. - Microservices Transformation: Refactor monolithic systems into a microservices architecture. - Cloud Deployment: Work within private or hybrid cloud infrastructure, ensuring secure and compliant deployments. - K...

As a Lead Data Engineer in our organization, you will play a crucial role in the data engineering team focused on on-premise solutions. Your responsibilities will include: - Designing, developing, and maintaining robust, scalable, and high-performance data pipelines and integration solutions. - Managing and optimizing data storage in Hadoop Distributed File System (HDFS). - Designing and implementing data workflows using Apache NiFi for data ingestion, transformation, and distribution. - Collaborating with cross-functional teams to understand data requirements and deliver efficient solutions. - Ensuring data quality, governance, and security standards are met within the on-premise infrastruc...

Role Overview : Lead Data Architectto define and govern theend-to-end enterprise data architecturefor a modern, scalable Cloudera-basedLakehouse platform. This role demands technical leadership across multiple domains includingCDC pipelines, Kafka streaming, real-time marts, MLOps/LLMOps frameworks, and data virtualization. You will act as theDesign Authority, ensuring architectural consistency, reusability, and compliance across all initiatives. Key Responsibilities : Define and own thetarget state data architecturealigned with business and regulatory needs. Serve asdesign authorityacross CDC ingestion, data lake zones (raw, curated, real-time), consumption layers, and AI enablement. Lead a...

Position - Manager, Data Engineer - Apache Nifi, Python, PySpark, Hadoop, Cloudera platforms, and Airflow Overview : The Mastercard Account Level Management (ALM) platform empowers real-time card level decisioning As consumers progress along their life stages as card holders, with increasing disposable income and more refined preferences, ALM provides services to issuers so they can effectively offer more relevant benefits and rewards at each stage, to drive loyalty and spend Role : Managing multiple scrum teams of Software developers and testers to develop quality software solutions in a timely and cost-effective manner Successfully lead definition, development and delivery of major cross-d...

Position Overview: This is a techno-functional position that combines strong technical skills with a deep understanding of business needs and requirements with 5-7 years or experience. The role focuses on developing and maintaining advanced data engineering solutions for pre-sales value quantification within the Services business unit. As a Sr. Analyst, you will be responsible for creating and optimizing data pipelines, managing large datasets, and ensuring the integrity and accessibility of data to support Mastercards internal teams in quantifying the value of services, enhancing customer engagement, and driving business outcomes. The role requires close collaboration across teams to ensure...

Responsibilities: Implement robust software components for data-driven systems, ensuring efficiency and maintainability. Conduct technical analysis and articulate solutions, designing engineering patterns that can be leveraged across multiple product offerings with a high volume of end-users. Optimize and enhance existing solutions, proactively identifying and resolving complex technical challenges while weighing trade-offs in cost, performance, and scalability. Develop reusable components and services, adhering to corporate engineering standards, best practices, and modern development frameworks. Apply software engineering best practices, leveraging automation in testing, deployment, and mo...

Role & responsibilities Cloud & Infrastructure Management Design, deploy, and manage infrastructure in Azure Cloud, ensuring scalability, security, and cost optimization. Manage and optimize AKS (Azure Kubernetes Service) and containerized workloads using Docker. Configure and administer Linux (Ubuntu) servers across environments. CI/CD & Automation Implement and maintain CI/CD pipelines using Git-based workflows and automation tools. Integrate security scanning and compliance checks into pipelines (DevSecOps best practices). Automate infrastructure provisioning using IaC (Terraform, ARM templates, or Bicep). Data & Middleware Infrastructure Deploy and manage Kafka clusters for real-time dat...