Data Engineer

Job Title:

Location:

Experience Level:

About the Role:

Data Engineer

data integrity and scalability

Key Responsibilities:

• Design, develop, and maintain

  real-time data ingestion pipelines

  using CDC from relational and NoSQL databases.
• Implement and manage

  data streaming solutions

  using Apache Kafka, Kafka Connect, and Debezium.
• Configure and monitor Debezium connectors for accurate data capture.
• Develop

  real-time data transformation

  logic using Spark Streaming, Flink, or Kafka Streams.
• Ensure

  data quality, consistency, and governance

  across real-time pipelines.
• Collaborate with application teams, analysts, and data scientists to meet real-time data needs.
• Troubleshoot and resolve pipeline, Kafka cluster, and connector issues.
• Implement monitoring, logging, and alerting to ensure availability and performance.
• Document real-time architectures, flows, and procedures.
• Adhere to

  data governance and compliance

  standards.

Technical Skills:

• Real-time Data Integration:

  Strong experience with CDC mechanisms.

• Streaming Tech:

  Apache Kafka, Kafka Connect, Kafka Streams.

• CDC Tools:

  Debezium (hands-on expertise).

• Programming:

  Python or Java (Spring Boot, REST APIs), Shell scripting.

• Big Data:

  Apache Spark Streaming, Apache Flink (preferred).

• Cloud:

  GCP (BigQuery, Data Lakes) or equivalent platforms.

• Databases:

  Relational + NoSQL for CDC integration.

• SQL:

  Advanced querying & validation skills.

• DevOps:

  Docker, Kubernetes, CI/CD deployment.

• Monitoring:

  Prometheus, Grafana for real-time data health.

Collaboration & Soft Skills:

• Strong collaboration with cross-functional engineering and business teams.
• Ability to clearly document and present

  real-time data architectures

  .
• Skilled in

  stakeholder communication

  (technical & non-technical).
• Proactive problem solver with attention to

  data integrity & reliability

  .
• Adaptable, self-driven, and detail-oriented.

Qualifications:

• BE/B.Tech and/or M.Tech in a relevant discipline.
• 46 years of

  Data Engineering experience

  with a focus on

  CDC & real-time data integration

  .
• Exposure to

  BFSI/NBFC domain

  is a strong advantage.