504 Apache Nifi Jobs

As an experienced Backend Developer, you will be responsible for architecting and building complex, scalable, and high-performance backend systems. Your role will involve owning the end-to-end development and delivery of features and enhancements. You will design and optimize large-scale distributed data processing using Apache Spark and Nifi. Collaborating with product managers, architects, and other stakeholders to align on technical vision will be a crucial part of your responsibilities. Additionally, you will lead code reviews and ensure adherence to best practices and high code quality. Mentoring and guiding junior and mid-level engineers will also be part of your role. Key Responsibili...

Role Overview: You will be joining as a Lead Data Engineer for an on-premise environment, where a significant part of your role will involve working closely with your team to design, implement, and optimize data solutions. Your expertise in building and enhancing data pipelines, architectures, and data sets, with a special focus on Python, SQL, Hadoop, HDFS, and Apache NiFi, will be crucial for this position. Key Responsibilities: - Design, develop, and maintain robust, scalable, and high-performance data pipelines and data integration solutions. - Manage and optimize data storage in Hadoop Distributed File System (HDFS). - Design and implement data workflows using Apache NiFi for data inges...

Job Description: You will be responsible for working with AWS/Azure, ETL, SQL, DATABRICKS, Python/Scala, PY-SPARK/Spark, APACHE NIFI or HIVE technologies to support data engineering tasks. Key Responsibilities: - Utilize AWS/Azure services for data processing and storage - Develop and maintain ETL pipelines - Write and optimize SQL queries for data retrieval and manipulation - Use DATABRICKS for data analysis and processing - Program in Python/Scala for data transformation tasks - Work with PY-SPARK/Spark for big data processing - Implement data ingestion and streaming using APACHE NIFI or HIVE Qualifications Required: - Minimum 5 years of experience in data engineering - Proficiency in AWS/...

We are looking for a highly skilled Full Stack Developer with expertise in Java , Reactive Programming , Spring Boot , and Node.js , combined with strong experience in data engineering and cloud-based deployments . This role involves building scalable microservices, designing data pipelines, and working on modern cloud-native architectures. 5 to 8 Years of strong experience in Java , Spring Boot , Reactive programming , and Node.js . Experience with Apache PySpark , Python , and data pipeline development using Apache NiFi . Solid understanding of Postgres , SQL , and JPA . Hands-on experience with Kubernetes , Docker , and Prometheus for cloud deployments. Familiarity with basic network conf...

Role Overview: We are seeking a highly experienced Database and Data Engineer with strong expertise in designing, implementing, and managing large-scale data solutions. The ideal candidate will have deep knowledge of AWS services , database technologies , and data engineering frameworks , ensuring robust, scalable, and secure data pipelines and platforms. Key Responsibilities: Design and implement data pipelines and ETL workflows using Apache NiFi , Spark , and other modern frameworks. Manage and optimize AWS RDS instances for Oracle and PostgreSQL , including performance tuning, backup, and recovery strategies. Architect and maintain data lakes on AWS S3 with Apache Iceberg for efficient da...

Azure Databricks: Hands-on experience working with Azure Databricks for building scalable ETL pipelines and integrating with Azure Data Lake Storage. ETL Development: Strong experience in building ETL pipelines to manage large-scale data flows and data integration. Data Lake Storage: Familiarity with Azure Data Lake Storage and its integration into the broader data architecture. Apache NiFi: Secondary expertise in using Apache NiFi for data integration, transformation, and flow management. Azure Cloud: Knowledge of Azure services related to data engineering, including Azure Data Lake, Azure Synapse, and Azure SQL. Qualifications Bachelor of Engineering

Role Overview: We are looking for a skilled Database & Data Engineer to design, build, and maintain robust data solutions on AWS . The ideal candidate will have hands-on experience with RDS (Oracle & PostgreSQL) , data lakes , and streaming platforms , ensuring efficient data flow and high availability across systems. Key Responsibilities: Develop and maintain data pipelines using Apache NiFi and Spark for batch and streaming data processing. Manage AWS RDS instances for Oracle and PostgreSQL , including configuration, performance tuning, and troubleshooting. Implement and optimize data lakes on AWS S3 using Apache Iceberg for structured and semi-structured data. Build and manage real-time d...

Roles and Responsibilities: Design and manage data pipelines using NiFi and Beam, build scalable ingestion workflows and cloud storage, ensure data quality, monitor performance, troubleshoot issues, and support deployments and optimization. Health insurance Annual bonus Provident fund Office cab/shuttle

Job Title: Data Engineer (NiFi / Kafka / Airflow / ELK / SQL / Power BI) Experience: 5+ Years Location: Bangalore Work Mode: Hybrid Job Description We are looking for an experienced Data Engineer to design, develop, and manage real-time and batch data pipelines within our Digital Platforms team. Responsibilities Build and optimize data pipelines using Apache NiFi, Kafka, and Airflow Develop and maintain ETL/ELT processes ensuring high data quality Manage data integration across systems using NiFi flows and Kafka topics Work with the ELK Stack (Elasticsearch, Logstash, Kibana) for monitoring and analytics Write efficient and optimized SQL queries for data extraction and reporting Develop Powe...

Position Overview: This is a techno-functional position that combines strong technical skills with a deep understanding of business needs and requirements with 5-7 years or experience. The role focuses on developing and maintaining advanced data engineering solutions for pre-sales value quantification within the Services business unit. As a Sr. Analyst, you will be responsible for creating and optimizing data pipelines, managing large datasets, and ensuring the integrity and accessibility of data to support Mastercards internal teams in quantifying the value of services, enhancing customer engagement, and driving business outcomes. The role requires close collaboration across teams to ensure...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : O9 Solutions Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :- As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with various teams to understand their needs, developing innovative solutions, and ensuring that applications are aligned with business objectives. You will engage in problem-solving...

Pro-Active Hire*. Loc: Manyata Tech Park. Exp: 5-8 yrs AI/ML Engineer. - JD - Skills Required: • Exp. with Data Engineering & Data/ML pipelines for Analytics & AI/ML Models • Working with large sets of structured & unstructured data from disparate sources • Foundational acquaintance of Data Science & AI/ML models • SQL, Python, Spark, PySpark • Big Data systems - Hadoop Ecosystem (HDFS, Hive, MapReduce) • Analytics dB like Druid, Data visualization/exploration tools like Superset • Understanding cloud (AWS, Azure, or GCP) • GIT Secondary Skills (preferred): • Apache Airflow, Apache Oozie, Nifi • GCP exp., Big Query, GCP Big Data Ecosystem • Trino/Presto • Familiar with forecasting algorithms...

Must Have: Proficiency in at least one of the following programming languages: Java/Scala/Python Good understanding of SQL Experience of development and deployment of at least one end-to-end data storage/processing pipeline Strong Experience in Spark development with batch and streaming Intermediate level expertise in HDFS and Hive Experience with Pyspark and Data Engineering ETL implementation and migration to spark Experience of working with Hadoop cluster Python, PySpark, Data Bricks developer with knowledge of cloud Experience with Kafka and Spark streaming (Dstream and Structured Streaming) Experience with using Jupyter notebooks or any other developer tool Experience with Airflow or ot...

Company Profile Priority Technology Holdings, Inc. (NASDAQ: PRTH), is headquartered in Alpharetta, Georgia USA. Our India office is located in Chandigarh, where our dynamic team builds state of the art, sophisticated Fintech products & solutions. We are an emerging payments powerhouse that offers a single unified platform for Banking & Payments powering modern commerce. Priority offers a unique family of products which integrate into SMB Payments, B2B Payments and Enterprise Payments to help businesses thrive. We are on a mission to offer an industry agnostic platform that enables businesses to collect, store and send money using various new age payment methods. Priority is an employee-first...

Title and Summary Data Scientist II Data Scientist II Job Family: Product Data & Analytics Job Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. We enable secure, simple, smart, and accessible digital payments. Our technology and innovation help people, businesses, and governments realize their greatest potential. Overview The Product Data & Analytics team builds internal analytic partnerships to strengthen focus on: Business health and portfolio optimization Revenue opportunities and initiative tracking New product development and Go-To-Market strategies We provide scalable end-to-end data solutions and influence decisions across Mastercard...

We are looking for a highly skilled Data Engineer to join our team at ALSTOM India Limited. The ideal candidate will have a strong background in data engineering and excellent problem-solving skills. Roles and Responsibility Design, develop, and implement data pipelines and architectures to support business intelligence and analytics. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems and infrastructure. Ensure data quality, integrity, and security through data validation and testing procedures. Optimize data processing workflows to improve performance and efficiency. Troubleshoot and resolve complex technica...

Overview The Data Warehouse BizOps team is looking for a Site Reliability Engineer who can help us solve problems and enhance our capabilities by supporting applications, services, and platforms. This position is to support the Data Delivery and Analytics program to support the growing need of the product like MyMPA, QMR. Tech Skills: Unix, Shell Scripting, SQL, Python, Apache Nifi, Splunk, Dynatrace, Jenkins, GIT, XLR etc. We regularly review our run state not only from an internal perspective, but also understanding and providing the feedback loop to our development partners on how we can improve the customer experience of our applications. Role: Engage in and improve the whole lifecycle o...

The role of business operations is to be the production readiness steward for the platform. This is accomplished by closely partnering with developers to design, build, implement, and support technology services. A business operations engineer will ensure operational criteria like system availability, capacity, performance, monitoring, self-healing, and deployment automation are implemented throughout the delivery process. Business Operations plays a key role in leading the DevOps transformation at Mastercard through our tooling and by being an advocate for change and standards throughout the development, quality, release, and product organizations. We accomplish this transformation through ...

Lead Data Engineer - (Datawarehouse) - Apache Nifi, Python, PySpark, Hadoop, Cloudera platforms, and Airflow Overview Mastercard is looking for a Lead Software Engineer to join the Account Level Management(ALM) team in our Pune office thats focused on building ALM Services with Data warehouse skills The Mastercard Account Level Management platform empowers real-time card level decisioning As consumers progress along their life stages as card holders, with increasing disposable income and more refined preferences, ALM provides services to issuers so they can effectively offer more relevant benefits and rewards at each stage, to drive loyalty and spend. Skill Requirements: MUST Technical 8-12 ...

Job Overview The Senior Software Engineer will do the design, development, and delivery of advanced big data solutions within the Hadoop ecosystem. This role involves architecting and implementing scalable data processing frameworks, ensuring high performance and reliability across distributed systems. The engineer will research and evaluate emerging technologies to optimize data workflows and meet evolving business needs. Additionally, the position requires hands-on expertise in Cloudera distribution and related tools, ensuring seamless orchestration and integration of data pipelines. The Senior Software Engineer will also be responsible for maintaining operational excellence, meeting SLA r...

The Data Warehouse BizOps team is looking for a Site Reliability Engineer who can help us solve problems and enhance our capabilities by supporting applications, services, and platforms. This position is to support the Data Delivery and Analytics program to support the growing need of the product like MyMPA, QMR. Tech Skills: Unix, Shell Scripting, SQL, Python, Apache Nifi, Splunk, Dynatrace, Jenkins, GIT, XLR etc. We regularly review our run state not only from an internal perspective, but also understanding and providing the feedback loop to our development partners on how we can improve the customer experience of our applications. Responsibilities : Engage in and improve the whole lifecyc...

Title and Summary The Software Engineer II will contribute to the design, development, and maintenance of big data solutions within the Hadoop ecosystem The role involves working with distributed data processing frameworks and supporting the creation of reliable, high-performance data pipelines The engineer will be responsible for hands-on development and troubleshooting using the Cloudera distribution, along with related ecosystem tools such as Apache Ozone, Apache Iceberg, Apache Airflow, and Apache NiFi The position requires hands on experience in Apache Spark to support large-scale, massively parallel data processing tasks The Software Engineer II will collaborate closely with senior tea...

We are looking for a skilled Big Data Administrator & Development Engineer with 8 to 13 years of experience to join our innovative team. The ideal candidate will have a strong passion for data platforms, high-performance systems, and large-scale data processing. Roles and Responsibility Manage and optimize Big Data Clusters for reliable, secure, and high-performance operation. Implement and maintain monitoring solutions using tools like Cloudera Manager and Splunk. Administer access and security controls, including RBAC, encryption, and compliance policies. Develop and automate data workflows using SQL, NiFi, Python, or Spark. Support architectural and technical evolution by collaborating on...

Position: ETL Developer Location : Hyderabad Experience : 3-5 years Mail: meenakshi@beanhr.com Are you passionate about building scalable data pipelines and enabling data-driven decision-making? Were looking for a skilled ETL Developer to join our team in Hyderabad. What youll do: Design, develop, and optimize ETL workflows (SSIS, Azure Data Factory) Integrate data from APIs, ERP, CRM, IoT, and cloud sources Transform, cleanse, and enrich raw data for analytics & BI Load data into modern warehouses (Snowflake, Redshift, BigQuery, Synapse) and data lakes Collaborate with analysts & business users to deliver reliable insights What were looking for: 3-5 years of ETL development/data engineering...

As a NiFi Data Engineer at the company, your role will involve designing, developing, and maintaining scalable data pipelines using NiFi. Your primary responsibility will be to ensure the efficient flow and transformation of data to support the organization's innovative platform and analytical needs. **Responsibilities:** - **Core Competencies:** - Bring guiding principles to life by enhancing experiences for dealerships and customers. - Collaborate with colleagues across A2Z Sync to try new ideas with purpose. - Take responsibility for your work and adapt to change as the company grows. - **Leadership Competencies:** - Collaborate with cross-functional teams to understand data requirements ...