TECHNICAL ANALYST

Design and develop data ingestion pipelines using Apache NiFi for structured, semi-structured, and unstructured data sources.

Build and maintain real-time and batch processing workflows to ingest data into AWS-based data lakes or warehouses.

Configure and optimize NiFi processors, templates, and flows for scalability and reliability.

Implement monitoring, alerting, and recovery strategies for NiFi flows and downstream processes.

Integrate NiFi with AWS services such as S3, Lambda, Redshift, Glue, Kinesis, and RDS.

Ensure data quality, integrity, and compliance across pipelines. Work closely with data analysts, architects, and business teams to define data requirements and delivery models.

Document workflows, metadata, and lineage using tools like Apache Atlas or custom catalogs.

Required Skills & Qualifications

6-7 years of experience in data engineering or data integration roles.

Strong hands-on experience with Apache NiFi including custom processor development and flow orchestration.

Solid experience with AWS services, especially: o S3, Lambda, Glue, Kinesis, Redshift, RDS

Strong understanding of data ingestion patterns (CDC, batch, streaming).

Proficiency in SQL, Python, or Java for data transformations or scripting.

Experience in handling large datasets and performance optimization in data workflows.

Good to Have

Exposure to Apache Kafka, Airflow, or Step Functions.

Experience with data lake architectures and tools like Delta Lake, Lake Formation, or Athena.

Knowledge of data governance and cataloging tools (AWS Glue Data Catalog, Apache Atlas).

Experience working in Agile teams with tools like Jira, Confluence.