Senior Data Engineer

Your Role:

You will be a part of the data engineering team and working on in-house products in for Data Foundation Collaborating closely with Data Engineering Leads and managers and other DataOps teams. This role involves hands-on development work and will directly report to Data Engineering Lead.

Must have skills:

• Azure Databricks, Unity Catalog, Delta live tables
• Data Modelling & Architecture
• ETL pipeline design
• Expert in Python, Pyspark and SQL
• Azure Data Factory
• Azure DevOps
• Logging and Monitoring using Azure / Databricks services
• Apache Kafka
• Strong experience in Azure cloud data services & technologies

Nice to have:

• Synapse, Fabric, PowerBI, Azure Functions, Azure Logic apps
• Azure API service
• Microsoft EntraID
• GCP & AWS cloud services knowledge
• Azure Networking
• Databricks online table
• Jira
• Shell scripting / Azure CLI knowledge

Your responsibilities will include:

• Experience in designing, developing, and delivering scalable, reliable, and high-performing software solutions.
• Design, development, and maintenance of scalable data pipelines and ETL processes.
• Monitor and optimize data infrastructure performance.
• Participate in code reviews, design reviews, testing, and deployment processes.
• Ensure adherence to coding standards, best practices, and architectural guidelines.
• Inspire, advise, and drive the selection of development approaches.
• Participate in technical discussions with other teams/departments.
• Communicate with the Engineering Manager, Product Owner, Business Analyst, and Scrum Master to align on project/sprint goals, timelines, and resource allocation.

You are a good match if you have:

• Min 5-7+ years of working experience in the field of Data Engineering or Software Engineering.
• Hands-on experience and in-depth knowledge of the technologies listed as mandatory in the Technology Stack section.
• Strong understanding and implementation of software development principles, coding standards, and modern architecture.
• Experience working on diverse projects with varying technologies, products, and systems.
• Strong problem-solving skills and the ability to make critical technical decisions.
• Ability to guide and mentor other team members.
• Strong stakeholder engagement and influencing skills.
• Pragmatic and collaborative team player.
• Experience with Databricks, data pipelines, and CI/CD.
• Proficiency in Python, SQL, and big data technologies in cloud environments.
• Pragmatic and collaborative team player.