Senior Data Engineer - Real-time Resource Allocation

Senior Data Engineer

Roles & Responsibilities

• Data Pipeline Development:

  Design, develop, and implement data pipelines,

  ETL/ELT

  processes, and data integration solutions. You will be responsible for delivering data pipeline projects from development to deployment, managing timelines and risks.

• Quality and Optimization:

  Ensure data quality and integrity through meticulous testing and monitoring. You will leverage cloud platforms like

  AWS and Databricks

  to build scalable and efficient solutions that adhere to industry standards.

• Collaboration and Agile Practices:

  Work closely with the product team and stakeholders to understand data requirements. You will work in an

  Agile

  development environment, and familiarity with

  JIRA

  and

  GIT

  is essential for code versioning and migration.

• Continuous Learning:

  Stay up-to-date with the latest data technologies and trends to contribute to our evolving data strategy.

Qualifications

• A Master's degree with experience, a Bachelor's degree with extensive experience, or a Diploma with a substantial background in Information Systems.
• Demonstrated hands-on experience with cloud platforms (

  AWS, Azure, GCP

  ).
• Proficiency in

  Python, PySpark, and SQL

  .
• Development knowledge in

  Databricks

  .
• Experience with

  data modeling

  and

  ETL orchestration technologies

  .
• Experience with software engineering best practices, including version control (

  Git

  ),

  CI/CD

  (Jenkins, Maven), automated unit testing, and

  DevOps

  .
• Good analytical and problem-solving skills.
• Familiarity with

  SQL/NOSQL

  databases.

Soft Skills

• Problem-Solving:

  Skilled in breaking down problems, documenting problem statements, and estimating efforts.

• Communication:

  Effective communication and interpersonal skills to collaborate with multi-functional and global teams.

• Proactiveness:

  A high degree of initiative, self-motivation, and a team-oriented mindset focused on achieving team goals.