Data Engineer, ML & Analytics

We are looking for a versatile Data Engineer to join the Personalization Recommendations (PnR) team. This role is central to our data-driven efforts, bridging the gap between core data infrastructure, machine learning, and analytics. You will be a hands-on contributor responsible for the end-to-end data lifecycle: from building robust ingestion and data pipelines to performing critical feature engineering and developing analytics dashboards to visualize model and system performance.

You will be a key partner to our Data Science and ML teams, ensuring they have the high-quality features and performance insights needed to innovate. This role is ideal for an engineer who excels at building robust data solutions and is also passionate about applying their skills to create ML features and measure their impact.

A SNAPSHOT OF YOUR RESPONSIBILITIES

• Data Pipelines Ingestion: Design, build, and maintain scalable and reliable data pipelines on the Databricks platform to ingest, process, and transform large-scale data.

• Feature Engineering: Collaborate with Data Scientists to design, build, and maintain scalable feature engineering pipelines, transforming raw data into high-quality signals for ML models.

• Analytics Visualization: Develop and maintain analytics dashboards and reports to monitor data pipeline health, feature drift, and model performance, providing key insights to the PnR team.

• Data Modeling: Develop robust data models, schemas, and ETL/ELT workflows within our data lakehouse architecture (Delta Lake).

• Optimization Collaboration: Optimize Spark jobs for performance and cost, and collaborate closely with Data Scientists and ML Engineers to understand their data requirements.

• Data Governance: Implement data quality checks and monitoring to ensure the integrity of our core datasets and features.

WHAT YOU WILL NEED

• At least 3-5 years of experience in data engineering, with a focus on building large-scale data pipelines and data warehousing solutions.

• Deep, hands-on expertise with the Databricks platform, including Spark, Delta Lake, and job scheduling/orchestration.

• Proficient in Python for data processing (PySpark, Pandas) and advanced SQL for complex data analysis.

• Proven experience with feature engineering techniques for machine learning.

• Experience with data visualization tools (eg, Tableau, Looker, or Python libraries like Matplotlib/Seaborn) and building analytical dashboards.

• Solid understanding of data warehousing concepts, data modeling, and schema design.

• Experience working in cloud-native environments such as AWS or GCP.

• Proven ability to collaborate effectively with data scientists, analysts, and other cross-functional teams.

NICE TO HAVE, BUT NOT REQUIRED

• Experience with MLOps concepts and tools, such as feature stores or MLflow.

• Exposure to real-time data processing and streaming architectures (eg, Kafka, Flink, Spark Structured Streaming).