Senior Data Engineer

Who We Are

Verve has created a more efficient and privacy-focused way to buy and monetize advertising. Verve is an ecosystem of demand and supply technologies fusing data, media, and technology together to deliver results and growth to both advertisers and publishersno matter the screen or location, no matter who, what, or where a customer is. With 30 offices across the globe and with an eye on servicing forward-thinking advertising customers, Verves solutions are trusted by more than 90 of the United States top 100 advertisers, 4,000 publishers globally, and the worlds top demand-side platforms. Learn more at www.verve.com.

Who You Are

We're looking for a Senior Data Engineer to join our Data Processing team at Verve. This team is responsible for designing and maintaining scalable, real-time, and batch data processing pipelines that enable high-quality insights and analytics. The Data Processing team works closely with the Data Analytics and Data Science teams to ensure efficient data flows across the organization.

In this role, you'll focus on developing and optimizing data processing workflows, ETL pipelines, and streaming solutions. Precision, efficiency, and high-quality deliverables are essential, and you'll report to the Engineering Manager, Data Processing.

If you are passionate about working with big data technologies, real-time data processing, and optimizing large-scale ETL workflows, wed love to hear from you!

This is a hybrid position with three days per week in our Bengaluru

office.

What You Will Do

Domain Expertise:

• Design, develop, and maintain scalable, efficient, and cost-effective data pipelines for real-time and batch data processing.
• Build high-performance ETL workflows leveraging Kafka, Apache Spark, and Scala.
• Implement best practices for data modeling, storage, and retrieval, ensuring high availability and reliability.
• Work with GCP-based data services, including BigQuery, Dataflow, and Cloud Storage.
• Optimize the performance and cost efficiency of data processing jobs.

Data Processing & Engineering

• Architect and implement scalable streaming and batch processing solutions.
• Develop real-time data processing pipelines using Kafka and Apache Spark Structured Streaming.
• Ensure the stability and reliability of high-volume data ingestion pipelines.
• Write clean, testable, and efficient code using Scala.
• Ensure compliance with data privacy and security best practices.

Collaborative Work

• Work closely with Data Analytics and Data Science teams to ensure data is processed, stored, and accessed efficiently.
• Collaborate with the Data Analytics team to provide structured data for data warehouses, reporting APIs, and UIs.
• Support the Data Science team by ensuring high-quality data availability for machine learning models.
• Work cross-functionally with engineering and product teams to design end-to-end data solutions.
• Coordinate with global teams in different time zones to deliver projects successfully.
• Extend optimized solutions to sister companies within Verve Group.

What Will You Bring

Must-Have Qualifications:

• 5+ years of experience as a Data Engineer working on large-scale ETL/data processing pipelines.
• Experience handling daily terabytes of data efficiently.
• Strong knowledge of Apache Kafka for streaming data processing.
• Strong expertise in Apache Spark (Scala) for batch and real-time processing.
• Proficiency in orchestration tools like Apache Airflow.
• Proficiency in GCP, including services like BigQuery, Dataflow, and Dataproc.
• Solid understanding of data modeling techniques, including Star/Snowflake schema and state of the art Lakehouses.
• Strong SQL skills, including query optimization and performance tuning.
• Familiarity with Apache Druid or other OLAP engines.
• Strong knowledge of infrastructure as code (Terraform).

Nice-to-Have Qualifications:

• Experience with AdTech or high-velocity data processing environments.
• Hands-on experience with Hadoop ecosystem tools.
• Knowledge of containerization (Docker, Kubernetes).
• Familiarity with BI tools like Looker
• Basic DevOps knowledge

What We Offer

Just a few of the benefits waiting for you at Verve:

• Keep healthy through our

  Apollo Health Check-ups

  , and stay covered with our

  Medical Insurance

  for you and your family
• Pick what matters most to you in our INR 4100/month

  Personalized Benefits Platform:

  Travel, entertainment, food, fitness, and healthcare
• Have us cover your

  office lunches 3 times a week through Zomato

  , and enjoy office sweets, snacks, and beverages
• Boost your professional knowledge with our annual

  training budget & internal webinars

  , and level up your language skills through our

  German/English classes

• Recharge with

  19 paid vacation days + 3 Wellness days

  throughout the year, in addition to the public holidays
• Strengthen team connections while exploring new cultures through our monthly

  Work&Travel raffle,

  offering a chance to work from one of our global offices (after 2 years of employment)
• and even more reasons to join us!

Verve provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.