Sr. Data Engineer

Title: Data Engineer

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETRM systems.
• Work on data integration projects within the Energy Trading and Risk Management (ETRM) domain.
• Collaborate with cross-functional teams to integrate data from ETRM trading systems like Allegro, RightAngle, and Endur.
• Optimize and manage data storage solutions in Data Lake and Snowflake.
• Develop and maintain ETL processes using Azure Data Factory and Databricks.
• Write efficient and maintainable code in Python for data processing and analysis.
• Ensure data quality and integrity across various data sources and platforms.
• Ensure data accuracy, integrity, and availability across various trading systems.
• Collaborate with traders, analysts, and IT teams to understand data requirements and deliver robust solutions.
• Optimize and enhance data architecture for performance and scalability

Mandatory Skills:

• Python/ pyspark - 5+ years
• Fast API - 3+ years
• Pydantic - 3+ years
• SQL Alchemy - 3+ years
• Snowflake or SQL - 3+ years
• Data Lake - 1+ years
• Azure Data Factory (ADF) - 1+ years
• CI\CD,

  Azure fundamentals

  , GIT - 1+ Years
• Integration of data solutions with CETRM trading Systems(Allegro, RightAngle, Endur) 2+ years (Good to have)