Data Engineer

Sonata Software

5 - 10 years

12 - 22 Lacs

hyderabad chennai bengaluru

Posted:-1 days ago| Platform:

Apply

Skills Required

airflow pyspark etl development aws python db2

Work Mode

Hybrid

Job Type

Full Time

Job Description

Builds and runs robust data pipelines for the CIF platform across batch, CDC and streaming modes, ensuring data reliability, lineage and SLAs for downstream CRM and analytics.

Key Responsibilities

Design, implement and optimise ELT/ETL pipelines across Landing Staging Curated ODS Data Marts.
• Ingest data via CDC (Debezium) and streams (Kafka/MSK); orchestrate jobs with Airflow.• Develop Glue/Datastage jobs, unit tests and observability (logs/metrics/traces).• Implement data partitioning, schema evolution, and performance tuning.• Document lineage and contracts; collaborate with Governance on catalogue & glossary.

Required Technical Proficiency / Stack

AWS (Glue, S3, Lambda, MSK/Kafka, Redshift, API Gateway, IAM, CloudWatch, Airflow)
• IBM DataStage 11.7, AWS Glue Jobs, Debezium CDC, REST/JSON, gRPC, Kafka• SQL Server, DB2, PostgreSQL• Python, Jupyter Notebooks (SageMaker or Data Fabric)• Orchestration & CI/CD: Airflow, Jenkins, Git, Terraform, CloudFormation, GitOps• Governance & Lineage: Collibra, Alation, DataHub, AWS Glue Data Catalog

Preferred Background

Banking/financial services experience; MDM/ODS exposure.
• Working knowledge of BIAN-aligned domains and canonical models.• Strong SQL engineering and performance optimisation.

More Jobs at Sonata Software

L2 Support Office 365

Bengaluru

4 - 6 yrs

INR 1 - 6 Lacs

Selenium AT - Pennymac

Bengaluru, Karnataka

7 - 7 yrs

Salary: Not disclosed

D365 F&O Finance Functional Consultant

Hyderabad, Chennai, Bengaluru

8 - 12 yrs

INR 20 - 35 Lacs

Salesforce Architect

Bengaluru

10 - 20 yrs

INR 22 - 37 Lacs

SAP Business Analyst with SAP SD & EDI

Hyderabad, Chennai, Bengaluru

6 - 11 yrs

INR 13 - 23 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.