Big Data Developer

Big Data Developer

Experience:

Salary :

Preferred Notice Period

Shift

Opportunity Type:

Placement Type:

(*Note: This is a requirement for one of Uplers' Clients)

Must have skills required :

Airflow, Avro, Core Frameworks, Data Formats & Storage, dbt, HDFS, Hive, Iceberg, NIFI, Oozie, ORC, Orchestration & Workflow, Programming language, PySpark, S3, Scala, Java, Kafka

Nexus Cognitive (One of Uplers' Clients) is Looking for:

Big Data Developer who is passionate about their work, eager to learn and grow, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you.

Role Overview Description

Role Summary / Purpose

The Big Data Developer plays a key role in the modernization of data ecosystem supporting the migration of legacy MAPR/Cloudera/Hortonworks applications to open-source frameworks compatible with NexusOne.

This individual will focus on refactoring, optimizing, and validating data processing pipelines, ensuring performance, scalability, and alignment with enterprise data standards.

The role requires strong technical expertise across distributed data systems, open-source frameworks, and hybrid data environments.

Core Responsibilities

• Analyze, refactor, and modernize Spark/MapReduce/Hive/Tez jobs for execution within NexusOne’s managed Spark and Trino environments.
• Design, build, and optimize batch and streaming pipelines using Spark, NiFi, and Kafka.
• Convert existing ETL jobs and DAGs from Cloudera/MAPR ecosystems to open-source equivalents.
• Collaborate with Data Engineers and Architects to define new data ingestion and transformation patterns.
• Tune performance across large-scale data processing workloads (partitioning, caching, resource allocation).
• Implement data quality and validation frameworks to ensure consistency during migration.
• Support code reviews, performance tests, and production readiness validation for migrated workloads.
• Document conversion approaches, dependencies, and operational runbooks.
• Partner with Wells Fargo application SMEs to ensure domain alignment and business continuity.

Key Skills & Tools

• Core Frameworks: Apache Spark, PySpark, Airflow, NiFi, Kafka, Hive, Iceberg, Oozie
• Programming Languages: Python, Scala, Java
• Data Formats & Storage: Parquet, ORC, Avro, S3, HDFS
• Orchestration & Workflow: Airflow, DBT
• Performance Optimization: Spark tuning, partitioning strategies, caching, YARN/K8s resource tuning
• Testing & Validation: Great Expectations, Deequ, SQL-based QA frameworks
• Observability & Monitoring: Datadog, Grafana, Prometheus

Ideal Background

• 4–8 years of experience in big data engineering or application modernization in enterprise settings.
• Prior experience with Cloudera, MAPR, or Hadoop ecosystems, transitioning to open-source frameworks.
• Strong understanding of distributed data architectures and data lake design principles.
• Exposure to hybrid or cloud-native environments (AWS, GCP, or Azure).
• Familiarity with regulated environments (financial services, telecom, healthcare) is a plus.

Success Criteria

• Successful refactoring and execution of legacy data pipelines within NexusOne environments.
• Measurable performance improvements (execution time, cost optimization, data quality metrics).
• Delivered migration artifacts — including conversion patterns, reusable scripts, and playbooks.
• Positive feedback from Wells Fargo application owners on migration support and knowledge transfer.
• Consistent adherence to coding standards, documentation, and change management practices.

How to apply for this opportunity:

Easy 3-Step Process:

2. Upload updated Resume & Complete the Screening Form

3. Increase your chances to get shortlisted & meet the client for the Interview!

About Our Client:

Nexus Cognitive takes enterprises from data to outcomes at unprecedented speed and scale. We’ve revolutionized the way enterprises get value from their data with a composable, agnostic framework that enables you to rapidly build new solutions with modular, pre-integrated data components. Built with open standards, we give you the freedom to work with the data, systems, and toolsets of your choice, while our universal data catalog provides robust governance to ensure compliance and cut risks.

Through close customer collaboration, we design solutions that connect data pipelines and increase data access from on-prem to multi-cloud, with complete visibility across the data ecosystem. Cut data complexity, get AI-ready, and prove ROI in weeks, not months with the fully managed data framework and outcomes from Nexus Cognitive.

About Uplers:

Uplers is the #1 hiring platform for SaaS companies, designed to help you hire top product and engineering talent quickly and efficiently. Our end-to-end AI-powered platform combines artificial intelligence with human expertise to connect you with the best engineering talent from India.

With over 1M deeply vetted professionals, Uplers streamlines the hiring process, reducing lengthy screening times and ensuring you find the perfect fit. Companies like GitLab, Twilio, TripAdvisor, and AirBnB trust Uplers to scale their tech and digital teams effectively and cost-efficiently.

Role & responsibilities

Preferred candidate profile