Data Engineer - Lead

Job Description

Location: Hyderabad Role: Permanent Mode: WFO JOB RESPONSIBILITIES: Tracks the various Machine learning projects and their data needs. Tracks and improves Kanban process of product maintenance Drives complex technical discussions both within company and outside data partners Actively Contributes to the design of machine learning solutions by having a deep understanding of how the data is used and how new sources of data can be introduced Advocates for investments in tools and technologies to streamline data workflows and reduce technical debt Continuously explores and adopts emerging technologies and methodologies in data engineering and machine learning Develops and maintains scalable data pipelines to support machine learning models and analytics Collaborates with data scientists to ensure efficient data processing and model deployment Ensures data quality, integrity, and security across all stages of the data pipeline Implements monitoring and alerting systems to detect anomalies in data processing and model performance Enhances data versioning, data lineage, and reproducibility practices to improve model transparency and auditing . QUALIFICATION 5+ years of experience in data engineering or related fields, with a strong focus on building scalable data pipelines to support machine learning workflows. Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or other relevant fields. Specific experience in Kafka needed . Snowflake and data bricks would be huge plus. Proven expertise in designing, implementing, and maintaining large-scale, high-performance data architectures and ETL processes managing 1TB a day. Strong knowledge of database management systems (SQL and NoSQL), distributed data processing (e.g., Hadoop, Spark), and cloud platforms (AWS, GCP, Azure). Experience working closely with data scientists and machine learning engineers to optimize data flows for model training and real-time inference with latency requirements. Hands-on experience with data wrangling, data preprocessing, and feature engineering to ensure clean, high-quality data for machine learning models. Solid understanding of data governance, security protocols, and compliance requirements (e.g., GDPR, HIPAA) to ensure data privacy and integrity. Preferred Experience in data pipelines and analytics for video-game development Experience in Advertising industry Experience in online businesses where transactions happen without human intervention. Show more Show less