Big Data Developer

• Database Expertise:

Strong hands-on experience with Greenplum, Teradata, MongoDB, and Oracle for data modelling, querying, and performance optimization.

• Streaming & Real-Time Data:

Proficient in Apache Kafka for building and managing real-time data streaming pipelines.

• Data Integration & ETL Tools:

1. Informatica: Experience with HTTP Transformation for REST APIs, XML Transformation, and OAuth-based authentication.

2. StreamSets: Skilled in using components like HTTP Client Processor with OAuth2 authentication for API integration.

• Big Data & Lakehouse Technologies:

i. Strong development experience with PySpark for distributed data processing.

ii. Familiarity with Apache Iceberg for managing large-scale, transactional datasets in Lakehouse architectures.

• Scripting & Automation:

1. Advanced scripting skills in Python and Unix/Linux shell for automation, data manipulation, and workflow management.

2. Proven ability to debug and promote fixes under high-priority scenarios.

• Data Replication:

1. Basic working knowledge/Understanding of Oracle Goldengate and CDC frameworks for data replication across heterogeneous systems.

• Workflow Orchestration:

1. Experience with Control-M and Apache Airflow for scheduling, monitoring, and managing complex/custom data workflows.

• Cloud & DevOps:

1. Familiarity with CI/CD tools and AWS ECS for containerized deployments and continuous integration.

• Soft Skills:

1. Strong problem-solving and debugging capabilities.

2. Flexible and dependable in supporting teams during critical issue resolution and extended hours when needed.