Senior Data Engineer

Job Title: Senior Data Engineer OTT Streaming Platform

Location:

Key Responsibilities

1. Data Pipeline Development & Maintenance

• Design, implement, and maintain

  scalable ETL/ELT pipelines

   for ingesting, processing, and transforming structured, semi-structured, and unstructured data from multiple sources (playback events, ad servers, content catalogs, third-party APIs).
• Ensure

  real-time streaming data

   ingestion for analytics, personalization, and A/B testing using tools like Kafka, Kinesis, or Pub/Sub.
• Build batch and

  hourly refresh pipelines

   for operational reporting, audience insights, and decision support.
• Implement

  AI-driven data processing workflows

   to optimize transformation logic, resource usage, and anomaly detection.

2. Data Modeling & Warehousing

• Develop and maintain

  optimized data models

   for content performance, audience segmentation, churn prediction, and monetization analysis.
• Manage

  cloud-based data warehouses

   (Snowflake, BigQuery, Redshift, or Databricks) for high-volume analytics workloads.
• Utilize

  AI-based optimization

   for schema design, query acceleration, and automated indexing to enhance performance and reduce costs.
• Ensure schema design aligns with analytical and operational needs.

3. Data Quality, Governance & Security

• Implement

  data validation, monitoring, and alerting

   frameworks to ensure accuracy, consistency, and availability of datasets.
• Integrate

  AI-enabled data quality monitoring

   for proactive detection of inconsistencies, delays, or corruptions.
• Work with the Data Governance team to maintain compliance with

  GDPR, CCPA, and other privacy regulations

  .
• Enforce secure access controls for sensitive user and business data.

4. Collaboration & Business Impact

• Partner with Data Science and ML teams to prepare training datasets for recommendation algorithms, ad targeting, and personalization models.
• Collaborate with Product and Marketing teams to deliver audience insights and campaign performance metrics.
• Drive

  AI adoption

   across data engineering workflows to improve agility, scalability, and decision-making speed.
• Provide technical mentorship to junior engineers.

Qualifications Required:

• Bachelors/Masters degree in Computer Science, Data Engineering, or related field.

• 6+ years

   of experience in data engineering roles, preferably in high-scale consumer tech, media, or OTT streaming environments.
• Strong programming skills in

  Python, Scala, or Java

   for data processing.
• Hands-on experience with

  distributed data frameworks

   (Apache Spark, Flink, Beam).
• Proven expertise with

  streaming data technologies

   (Kafka, Kinesis, Pub/Sub).
• Proficiency in

  SQL

   and performance optimization for large datasets.
• Experience with

  cloud platforms

   (AWS, GCP, Azure) and services like S3, EMR, Dataflow, Glue, etc.
• Strong understanding of

  data modeling

   (star/snowflake schemas) and warehousing best practices.
• Practical experience in

  applying AI/ML techniques

   for data pipeline optimization, automated schema evolution, query performance tuning, and anomaly detection.

Preferred:

• Experience with

  OTT/Media streaming analytics

   (e.g., playback logs, CDN metrics, ad impression data).
• Familiarity with

  recommendation systems, A/B testing frameworks, and audience segmentation

  .
• Knowledge of

  ad tech pipelines

   for AVOD/FAST (VAST, SSAI, client-side tracking).
• Exposure to

  machine learning data pipelines

   and MLOps practices.

Key Skills & Attributes

• Ability to work with

  petabyte-scale datasets

   with performance and cost-efficiency in mind.
• Strong analytical thinking with a business impact mindset.
• Excellent communication skills for cross-functional collaboration.
• Self-driven, proactive, and able to work in fast-paced environments.
• Demonstrated ability to

  blend traditional data engineering with AI-powered automation

  .

Performance Metrics

• Data pipeline uptime and latency.
• Data accuracy and completeness.
• Query performance improvements.
• AI-enabled efficiency gains in data processing and warehouse management.
• Contribution to business KPIs (e.g., CLV, data warehouse, feature impact, A&B testing etc ).