Senior Data Engineer

We are seeking a Senior Data Engineer to design, build, and optimize our data infrastructure. The ideal candidate will have hands-on experience in data migration, report generation, and real-time data streaming using Apache Kafka. You ll play a key role in ensuring seamless data movement across systems, creating reliable reporting pipelines, and enabling real-time analytics for business-critical applications. This is a highly technical role requiring a deep understanding of databases, distributed systems, and modern data engineering practices.

This role requires deep technical expertise, attention to detail, and the ability to collaborate effectively with data architects, DBAs, and business stakeholders .

Key Responsibilities:

• Design, develop, and maintain data pipelines for ingestion, transformation, and reporting.


• Plan and execute data migration strategies from legacy systems to modern platforms.


• Designing and maintaining ETL (Extract, Transform, Load) pipelines. Monitor ETL jobs and troubleshoot issues proactively.


• Develop and manage Kafka producers and consumers for real-time data streaming and integration.


• Collaborate with backend and analytics teams to deliver accurate and timely business reports.


• Optimize SQL queries and database structures for performance and scalability.


• Build and optimize SQL/NoSQL database queries and stored procedures for generating complex reports.


• Ensure data quality, consistency, and compliance across environments.


• Work with stakeholders to define data models and reporting requirements.


• Troubleshoot production issues and provide long-term, scalable solutions.


• Participate in code reviews, provide technical mentorship, and contribute to architectural decisions.


• Mentor junior engineers and contribute to data architecture best practices.


• Create Design Documents for Reporting and Data Migrations.

Required Skills & Qualifications:

• Bachelor s or Master s degree in Computer Science, Information Systems, or a related field.


• 7+ years of experience in data engineering or related backend/data roles.


• Hands-on experience with at least Pentaho.


• Strong programming skills in Java, Python, or Scala.


• Proven experience in data migration projects, including data mapping, validation, and reconciliation.


• Expertise in Apache Kafka (producer/consumer setup, schema registry, stream processing).


• Strong knowledge of SQL and experience with relational databases (PostgreSQL, MySQL, Oracle, etc.).


• Experience with ETL design, data modeling, and data warehousing concepts.


• Familiarity with big data technologies (e.g., Spark, Hadoop) and cloud platforms (AWS/GCP/Azure).


• Understanding of microservices, API integrations, and data security principles.


• Excellent analytical, problem-solving, and communication skills.


• Good communication and documentation skills.


• Excellent analytical and problem-solving skills.


• Experience with data profiling, cleansing, and transformation.


• Familiarity with data warehousing concepts.

Preferred Qualifications:

• Experience with Python or Shell scripting for custom ETL logic.


• Familiarity with agile methodologies and Jira.


• Experience in migrating data across systems with different data models and formats.


• Knowledge of data governance and compliance standards.

• Experience in working on data archival, purging strategies, and incremental loads.