Senior Data Engineer - Real-time Resource Allocation

In this vital role you will be responsible for the development and implementation of our data strategy. The ideal candidate possesses a strong blend of technical expertise and data-driven problem-solving skills. As a Data Engineer, you will play a crucial role in building and optimizing our data pipelines and platforms in a SAFE Agile product team.

Roles & Responsibilities:

• Chip into the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions.
• Deliver for data pipeline projects from development to deployment, managing, timelines, and risks.
• Ensure data quality and integrity through meticulous testing and monitoring.
• Leverage cloud platforms (AWS, Databricks) to build scalable and efficient data solutions.
• Work closely with product team, and key collaborators to understand data requirements.
• Enforce to data engineering industry standards and standards.
• Experience developing in an Agile development environment, and comfortable with Agile terminology and ceremonies.
• Familiarity with code versioning using GIT and code migration tools.
• Familiarity with JIRA.
• Stay up to date with the latest data technologies and trends

Basic Qualifications:

• Masters degree and 4 to 6 years of Information Systems experience OR
• Bachelors degree and 6 to 8 years of Information Systems experience OR
• Diploma and 10 to 12 years of Information Systems experience.
• Demonstrated hands-on experience with cloud platforms (AWS, Azure, GCP)
• Proficiency in Python, PySpark, SQL.
• Development knowledge in Databricks.
• Good analytical and problem-solving skills to address sophisticated data challenges.

Must-Have Skills:

• Experienced with data modeling
• Experienced working with ETL orchestration technologies
• Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and DevOps
• Familiarity with SQL/NOSQL database

Good-to-Have Skills:

• Experience with DevOps CI/CD build and deployment pipeline
• Experience with design patterns, data structures, test-driven development
• Experience with Python-based visualization frameworks like Plotly.

Soft Skills:

• Skilled in breaking down problems, documenting problem statements, and estimating efforts.
• Effective communication and interpersonal skills to collaborate with multi-functional teams.
• Excellent analytical and problem solving skills.
• Strong verbal and written communication skills
• Ability to work successfully with global teams
• High degree of initiative and self-motivation.
• Team-oriented, with a focus on achieving team goals