Data Engineer – Databricks & Spark

Senior Data Engineer–Big Data & Analytics

Location:

Tech Stack:

Desired Experience:

About the Project

:Join an innovative project focused on revolutionizing how companies manage and optimize their promotional activities. This data-driven solution enables businesses to analyze the effectiveness of promotions, plan more impactful campaigns, and integrate seamlessly with existing ecosystems. Leveraging machine learning and advanced analytics, the platform enhances vendor collaboration, improves deal negotiation, and maximizes return on investment (ROI)

.
Rol

d Senior Data Engine

T.
Responsibiliti

• es:Design, build, and optimize scalable data pipelines usi

  ng Databricks, SQL, and Apache Sp

• arkDevelop and maintain robust ETL processes for large-scale data syst
• emsCollaborate with cross-functional teams to understand business needs and translate them into data soluti
• onsEnsure high performance and quality of data workfl
• owsSupport machine learning and analytics teams with curated, reliable datas
• etsImplement best practices in data engineering, testing, and documentat

ion
Qualificati

• ons:Deep expertise

  in Databricks, Apache Spark, and

• SQL3–8 years of hands-on experience in designing and maintaining scalable ETL pipelines, with a strong foundation in data engineering best practices and cloud platf
• ormsStrong experience w

  ith data mode

  ling

  and ETL pipeline develop

• mentProficient

  in Py

  thon for data manipulation and scrip
• tingHands-on experience w

  ith

  AWS (especia

  ll

  y S3

  and

• ECS)Comfortable working i

  n a Linux/Ub

  untu environ
• mentFamiliarity with dbt (data build tool) for managing data transformat
• ionsPrior experience working in promotional optimization or the retail analytics do
• mainStrong problem-solving skills, preferring logical and innovative solutions over legacy experi
• enceExcellent communication skills with the ability to clarify requirements directly with stakehol
• dersWillingness to engage in pair programming and collaborative problem-sol
• vingProactive, self-driven, and able to work independently in a distributed team environ