Data Platform Engineer

Data Platform Engineer

Roles & Responsibilities

• Platform Development & Integration:

  Design, develop, implement, and maintain solutions using Cloud and Big Data technologies. You will work closely with the Enterprise Data Lake teams to ensure applications align with architectural and development guidelines.

• Technical Evaluation:

  Research and evaluate technical solutions, including

  Databricks

  and

  AWS

  services,

  NoSQL

  databases, and data science packages. The focus of this evaluation will be on enterprise deployment capabilities like security, scalability, and cost management.

• Data Management & Analytics:

  Build

  ETL

  or

  ELT

  pipelines and work with data lake, data fabric, and data mesh concepts. You will also develop a basic understanding of core business problems to identify opportunities for using advanced analytics.

• Collaboration & Agile Practices:

  Work as part of an

  Agile

  development team, participating in ceremonies and embracing new responsibilities. You will also assist in building relationships with internal and external business stakeholders and work with platform stakeholders to ensure effective cost observability and control mechanisms.

• Best Practices:

  Adhere to software engineering best practices, including version control (

  Git

  ),

  CI/CD

  , and automated unit testing. You will also assist in reviewing third-party providers for new features that could be incorporated into the data platform.

Qualifications

• A Master's degree with 1-3 years of experience, a Bachelor's degree with 3-5 years of experience, or a Diploma with 8+ years of experience in computer science or engineering.
• Experience with

  Databricks

  (or

  Snowflake

  ), including cluster setup and tuning.
• Experience with common data processing libraries (

  Pandas, PySpark, SQLAlchemy

  ) and relational databases.
• Proficiency in programming languages such as

  SQL, Python

  , or

  Java

  .
• Experience with

  UI

  frameworks (

  Angular.js

  or

  React.js

  ) is also a must-have.
• Experience with

  Cloud

  technologies, preferably

  AWS

  , is a plus.

• Cloud Certifications

  from

  AWS, Databricks

  , or Microsoft are highly valued.

Soft Skills

• Problem-Solving:

  Excellent analytical and troubleshooting skills, with a focus on problem-solving and an ability to bring consistency and quality assurance to projects.

• Communication:

  Excellent written and verbal communication skills, with the ability to translate technical content into business language.

• Initiative:

  A high degree of initiative and self-motivation, with the ability to embrace new responsibilities and technologies.

• Teamwork:

  A team-oriented mindset and the ability to work effectively with global, virtual teams.