Job
Description
Role Summary Red Orange Infotech is seeking experienced Data Engineers to join our team in Gurugram, India. In this role, you will design and implement robust, scalable data architectures that handle real-time and batch data processing using advanced open-source and cloud-native technologies. You will work closely with cross-functional teams to build data pipelines, manage distributed systems, and optimize data access patterns to support analytics and business intelligence initiatives. This is a full-time position requiring 5-10 years of experience, with a 5-day work-from-office commitment in Gurugram. Key Responsibilities· · Design, build, and maintain scalable ETL/ELT pipelines for both structured and semi-structured data. · Develop data ingestion and streaming pipelines using Kafka, Kafka Connect, and Debezium for real-time CDC (Change Data Capture). · Implement data lake solutions and manage Apache Iceberg tables for large-scale, immutable data storage. · Integrate and optimize Trino or PostgreSQL queries for high-performance analytics and federated querying. · Use Flink for real-time data processing and streaming transformations. · Manage large-scale search and analytics using Elasticsearch or OpenSearch. · Collaborate with DevOps to deploy scalable solutions on containerized infrastructure, supporting Prometheus for monitoring data pipeline health. · Optimize and monitor cloud storage integrations such as Ceph for distributed data lake architectures. · Implement robust data quality, validation, and governance mechanisms to ensure reliable data pipelines. Right Candidate Profile The ideal candidate is a technically adept Data Engineer with deep experience in real-time and batch data architectures. You are comfortable managing high-throughput streaming systems, optimizing distributed queries, and working across complex data ecosystems. You take ownership of the reliability and performance of your data flows, ensuring that downstream users receive accurate, timely data. With 3–7 years of experience, you thrive in collaborative environments and contribute meaningfully to Red Orange Infotech’s mission of delivering innovation through process-driven, intelligent data systems. Must Have Skills · Expertise in SQL, especially PostgreSQL and Trino query optimization. · Strong experience with Apache Kafka, Kafka Connect, and Debezium for real-time data pipelines. · Proficiency in using Apache Iceberg for large-scale data lakehouse architectures. · Hands-on experience with OpenSearch or Elasticsearch for querying and indexing large datasets. · Experience with Apache Flink for real-time data processing and event-driven applications. · Strong command of Python for data pipeline scripting and transformation logic. · Familiarity with Prometheus for pipeline monitoring and system health metrics. · Knowledge of cloud object storage systems like Ceph and their integration into data platforms. · Deep understanding of ETL/ELT design patterns, data validation, and quality assurance. Nice to Have Skills · Experience integrating data workflows into CI/CD environments. · Familiarity with data Lakehouse architecture and versioned datasets. · Exposure to stream processing optimizations, watermarking, and late event handling. · Understanding of data security, access controls, and compliance frameworks. Qualifications and Work Mode · Bachelor’s degree in Computer Science, Engineering, or a related field. · 3–7 years of relevant experience in data engineering or related big data roles. · 5-day work-from-office role based in Gurugram, India.
