NIFI Developer

Role: NIFI Developer

Notice period: Notice Serving Candidates or Immediate Joiners Preferred

Client: Marriott

Payroll: Dminds

Work Mode: Remote

nterview Mode: Virtual

We’re looking for someone who has built deployed and maintained NIFI clusters.

Roles & Responsibilities:

·Implemented solutions utilizing Advanced AWS Components: EMR, EC2, etc integrated with Big Data/Hadoop Distribution Frameworks: Zookeeper, Yarn, Spark, Scala, NiFi etc.

·Designed and Implemented Spark Jobs to be deployed and run on existing Active clusters.

·Configured Postgres Database on EC2 instances and made sure application that was created is up and running, Trouble Shooted issues to meet the desired application state.

·Experience in creating and configuring secure VPC, Subnets, and Security Groups through private and public networks.

·Created alarms, alerts, notifications for Spark Jobs to email and slack group message job status and log in CloudWatch.

·NiFi data Pipeline to process large set of data and configured Lookup’s for Data Validation and Integrity.

·generation large set of test data with data integrity using java which used in Development and QA Phase.

·Spark Scala, improving the performance and optimized of the existing applications running on EMR cluster.

·Spark Job to Convert CSV data to Custom HL7/FHIR objects using FHIR API’s.

·Deployed SNS, SQS, Lambda function, IAM Roles, Custom Policies, EMR with Spark and Hadoop setup and bootstrap scripts to setup additional software’s needed to perform the job in QA and Production Environment using Terraform Scripts.

·Spark Job to perform Change Data Capture (CDC) on Postgres Tables and updated target tables using JDBC properties.

·Kafka Publisher integrated in spark job to capture errors from Spark Application and push into Postgres table.

·extensively on building Nifi data pipelines in docker container environment in development phase.

·Devops team to Clusterize NIFI Pipeline on EC2 nodes integrated with Spark, Kafka, Postgres running on other instances using SSL handshakes in QA and Production Environments.