Principal Data Engineer

Role & responsibilities

As a Principal Data Engineer in the Data Warehouse team you will be relied on to lead a team of Sr. Data Engineers, propose Architecture, solve complex situations and data model, interact directly with client for requirement gathering, build a holistic roadmap, identify risks and have mitigation strategy, develop and deliver high-quality features and finish tasks to a high standard. Youll be part of an emerging team, focussing on building and improving our innovative BNL Data Warehouse offerings applying marketing suites.

Must Have:

• Overall, 10 + years of experience in data engineering with 4+ years leading team
• 8+ years of hands-on experience in using Python, JVM and Shell Scripting with production experience.
• 5+ years of proven experience on building cloud native data intensive applications, Amazon Web Service experience is a must.
• Experience on Data Warehouse design and development per the business nature using any of the modern data warehouses.
• Strong experience in Databricks and DBT experience is required.
• Highly proficient in architecting and implementing workflow management systems, Airflow is a must.
• Hands-on experience of building and maintaining Spark applications.
• Fundamental understanding of Parquet, Delta Lake and other OTFs file formats.
• Strong experience on building CI/CD processes, experience with GitHub Actions, ArgoCD or Jenkins.
• Experience with unit testing and test automation frameworks.
• Propose and review Architecture.
• Solve complex situations and data model.

Good / Nice to have:

• Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).
• Familiarity on an IaC tool such as Terraform, CDK or CloudFormation is a must.
• Experience with observability systems (such as Datadog or Prometheus/Grafana)

Soft skills:

• Lead a team of 4-5 Sr.Engg
• client communication and Requirement gathering
• Build a holistic Tech roadmap for DWH (collaborate with business + teach leads)
• identify risks and have mitigation strategy.
• Mentoring Junior team members.
• Guiding and revieing PR. Guide on best practices.
• Learning aptitude: Ready to upskill on new tech stacks / skills.
• Comfortable providing, receiving constructive feedback, particularly while participating in code reviews and ability to not only code, but also provide good quality readable business documentation.
• Focus on business value vs engineering interest and focus on business value vs engineering interest.
• Pro-active / service-oriented towards our internal customers