Senior Data Engineer Ifood Web Media Technology

5.0 - 8.0 years

4 - 9 Lacs

chennai, bengaluru, delhi / ncr

Work from Office

Job Title: Senior Data Engineer OTT Streaming Platform Location: [ Remote] Key Responsibilities 1. Data Pipeline Development & Maintenance Design, implement, and maintain scalable ETL/ELT pipelines for ingesting, processing, and transforming structured, semi-structured, and unstructured data from multiple sources (playback events, ad servers, content catalogs, third-party APIs). Ensure real-time streaming data ingestion for analytics, personalization, and A/B testing using tools like Kafka, Kinesis, or Pub/Sub. Build batch and hourly refresh pipelines for operational reporting, audience insights, and decision support. Implement AI-driven data processing workflows to optimize transformation logic, resource usage, and anomaly detection. 2. Data Modeling & Warehousing Develop and maintain optimized data models for content performance, audience segmentation, churn prediction, and monetization analysis. Manage cloud-based data warehouses (Snowflake, BigQuery, Redshift, or Databricks) for high-volume analytics workloads. Utilize AI-based optimization for schema design, query acceleration, and automated indexing to enhance performance and reduce costs. Ensure schema design aligns with analytical and operational needs. 3. Data Quality, Governance & Security Implement data validation, monitoring, and alerting frameworks to ensure accuracy, consistency, and availability of datasets. Integrate AI-enabled data quality monitoring for proactive detection of inconsistencies, delays, or corruptions. Work with the Data Governance team to maintain compliance with GDPR, CCPA, and other privacy regulations . Enforce secure access controls for sensitive user and business data. 4. Collaboration & Business Impact Partner with Data Science and ML teams to prepare training datasets for recommendation algorithms, ad targeting, and personalization models. Collaborate with Product and Marketing teams to deliver audience insights and campaign performance metrics. Drive AI adoption across data engineering workflows to improve agility, scalability, and decision-making speed. Provide technical mentorship to junior engineers. Qualifications Required: Bachelors/Masters degree in Computer Science, Data Engineering, or related field. 6+ years of experience in data engineering roles, preferably in high-scale consumer tech, media, or OTT streaming environments. Strong programming skills in Python, Scala, or Java for data processing. Hands-on experience with distributed data frameworks (Apache Spark, Flink, Beam). Proven expertise with streaming data technologies (Kafka, Kinesis, Pub/Sub). Proficiency in SQL and performance optimization for large datasets. Experience with cloud platforms (AWS, GCP, Azure) and services like S3, EMR, Dataflow, Glue, etc. Strong understanding of data modeling (star/snowflake schemas) and warehousing best practices. Practical experience in applying AI/ML techniques for data pipeline optimization, automated schema evolution, query performance tuning, and anomaly detection. Preferred: Experience with OTT/Media streaming analytics (e.g., playback logs, CDN metrics, ad impression data). Familiarity with recommendation systems, A/B testing frameworks, and audience segmentation . Knowledge of ad tech pipelines for AVOD/FAST (VAST, SSAI, client-side tracking). Exposure to machine learning data pipelines and MLOps practices. Key Skills & Attributes Ability to work with petabyte-scale datasets with performance and cost-efficiency in mind. Strong analytical thinking with a business impact mindset. Excellent communication skills for cross-functional collaboration. Self-driven, proactive, and able to work in fast-paced environments. Demonstrated ability to blend traditional data engineering with AI-powered automation . Performance Metrics Data pipeline uptime and latency. Data accuracy and completeness. Query performance improvements. AI-enabled efficiency gains in data processing and warehouse management. Contribution to business KPIs (e.g., CLV, data warehouse, feature impact, A&B testing etc ).

Posted Date not available

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.