Data Engineer

This role is for one of the Weekday's clients

Salary range: Rs 2000000 - Rs 3000000 (ie INR 20-30 LPA)

Data Engineer

Requirements

Key Responsibilities

• Data Engineering & Pipeline Development
• Design, develop, and maintain scalable data pipelines capable of handling 100M+ records and multi-tenant systems.
• Integrate and streamline data ingestion from third-party sources such as GitHub, Jira, Jenkins, Bitbucket, and other developer or analytics platforms.
• Implement transformation logic, incremental updates, and automated workflows for continuous data refresh.
• Backend & System Architecture
• Build and optimize services using Java, AWS Lambda, ECS, CloudWatch, S3, RDS, and other cloud-native components.
• Architect and maintain high-throughput systems with strong performance, fault tolerance, and security.
• Collaborate with backend teams to standardize data contracts, APIs, and domain models.
• Database & Query Optimization
• Own and optimize PostgreSQL environments — partitioning, indexing, schema evolution, query tuning, and performance debugging.
• Implement data modeling best practices to support complex analytical workloads and reporting needs.
• Ensure data accuracy, consistency, and version control across schema changes and upgrades.
• Data Quality, Monitoring & Automation
• Build monitoring and alerting frameworks to track job reliability, latency, data inconsistencies, and infrastructure performance.
• Create automated systems for validation, deduplication, and enforcement of data standards.
• Work with DevOps or Platform Teams to establish CI/CD pipelines for data workflows.
  

Required Skills & Experience

• 6-10 years of hands-on experience as a Backend Engineer or Data Engineer in high-scale, data-centric environments.
• Expert-level proficiency in Java for building microservices, data processing systems, or backend automation.
• Strong experience with AWS cloud stack — S3, ECS, Lambda, RDS, and CloudWatch.
• Proven ability to ingest, parse, and transform data from multiple external APIs and complex data sources.
• Deep practical knowledge of PostgreSQL optimization techniques including indexing, partitioning, performance tuning, and query planning.
• Hands-on experience building scalable data workflows, event-driven systems, or analytics platforms.
  

Bonus / Good-to-Have Skills

• Exposure to dbt, ClickHouse, Kafka, Temporal, Superset, or developer-analytics ecosystems.
• Experience in distributed systems or streaming-based architectures.
• Prior work in developer tooling, DevOps analytics, CI/CD intelligence, or productivity insights