Data Engineer

• Design, build, and maintain scalable data pipelines and workflows.
  
• Manage and optimize cloud-native data platforms on Azure with Databricks and Apache Spark (12 years).
  
• Implement CI/CD workflows and monitor data pipelines for performance, reliability, and accuracy.
  
• Work with relational databases (Sybase, DB2, Snowflake, PostgreSQL, SQL Server) and ensure efficient SQL query performance.
  
• Apply data warehousing concepts including dimensional modelling, star schema, data vault modelling, Kimball and Inmon methodologies, and data lake design.
  
• Develop and maintain ETL/ELT pipelines using open-source frameworks such as Apache Spark and Apache Airflow.
  
• Integrate and process data streams from message queues and streaming platforms (Kafka, RabbitMQ).
  
• Collaborate with cross-functional teams in a geographically distributed setup.
  
• Leverage Jupyter notebooks for data exploration, analysis, and visualization.

Requirements:

• 4+ years of experience in data engineering or a related field.
  
• Strong programming skills in Python with experience in Pandas, NumPy, Flask.
  
• Hands-on experience with pipeline monitoring and CI/CD workflows.
  
• Proficiency in SQL and relational databases.
  
• Familiarity with Git for version control.
  
• Strong communication and collaboration skills with ability to work independently.

Good to have skills: