Data Engineer

Role: Data Engineer II

Data Engineer II

Responsibilities:

• Design and maintain high-throughput data platform architecture handling hundreds of billions of daily events.
  
• Explore, refine, and assemble large, complex data sets aligned with business and product needs.
  
• Identify and implement internal process improvements, such as automation, data delivery optimization, and infrastructure re-design for scalability.
  
• Build infrastructure for optimal extraction, transformation, and loading (ETL) of data from diverse sources using Spark, EMR, Snowpark, Kafka, and related technologies.
  
• Collaborate with stakeholders across distributed teams to resolve data-related issues and support infrastructure requirements.
  
• Translate business requirements into clear technical solutions for both technical and non-technical audiences.
  

Qualifications:

• 2+ years of experience in a Data Engineer role.
  
• Bachelors degree (or higher) in Computer Science or related Engineering field.
  
• Proven experience building and optimizing big data pipelines, architectures, and datasets.
  
• Strong working knowledge of

  Databricks/Spark

  and associated APIs.
• Proficiency in programming/scripting with

  Python, Java, or Scala

  .
• Experience with relational databases, SQL authoring/optimization, and working across multiple database technologies.
  
• Hands-on experience with

  AWS cloud services

  such as EC2, EMR, and RDS.
• Familiarity with NoSQL data stores (e.g., Elasticsearch, Apache Druid).
  
• Experience with data pipeline and workflow management tools such as

  Airflow

  .
• Ability to perform root cause analysis on complex data and processes to solve business problems and identify improvements.
  
• Strong experience with unstructured and semi-structured data formats (JSON, Parquet, Iceberg, Avro, Protobuf).
  
• Deep knowledge of data transformation, data structures, metadata, dependency, and workload management.
  
• Demonstrated ability to process, manipulate, and extract insights from large datasets.
  
• Working knowledge of stream processing, message queuing, and highly scalable big data storage systems.
  
• Experience collaborating with cross-functional teams in fast-paced environments.
  

Preferred Skills:

• Experience with streaming systems such as

  Kafka, Spark Streaming, or Kafka Streams

  .
• Knowledge of

  Snowflake/Snowpark

  .
• Familiarity with

  DBT

  .
• Exposure to AdTech industry data and systems.

Thanks & Regards,

Gloria Dias

Research Associate | LH

Gloria.Dias@persolapac.com

persolindia.com

Pune, India

CONFIDENTIAL NOTE:

By submitting your curriculum vitae or other personal data to us in connection with your job application or in your capacity as our employee, contractor, associate, partner or vendor, you acknowledge that you have carefully read and agreed to the terms of our Privacy Policy and the consent notice thereunder. You hereby provide voluntary consent to the collection, use, processing and disclosure of your personal data by us and our affiliates, in accordance with and for the purposes set out in our Privacy Policy and for other legitimate purposes as specified under applicable law. Your submission of personal data via email implies that you have not expressly dissented to the processing of personal data for the stated purpose. For a detailed understanding of our data collection practices, please refer to our Privacy Policy accessible here. If at any time, you wish to expressly withdraw your consent or have any grievance, you can do so by submitting a request to our designated consent manager, as provided in our Privacy Policy. Your privacy is of utmost importance, and we are committed to address the queries you have in this regard.

SECURITY NOTE: