Delta Lakes & Relational Database Engineer

Role Overview: As a Data Engineer Grade 5 at our company in Hyderabad, TL, you will be responsible for building solutions against delta lakes and relational databases. Your main focus will be working on Azure Databricks, writing code in Python and SQL with PySpark. Your tasks will include analysis, design, implementation, and testing of solutions. We highly value automated testing to ensure quality deliverables. One of your key responsibilities will be extending the functionality of the consistency check engine for sub-ledger data and other tenants. Given the significant data volume you will be dealing with, you will also be required to optimize Spark performance deeply. Key Responsibilities: - Build solutions against delta lakes and relational databases - Work on Azure Databricks using Python, SQL, and PySpark - Perform analysis, design, implementation, and testing - Extend the functionality of the consistency check engine for sub-ledger data and other tenants - Optimize Spark performance for large datasets with billions of rows and hundreds of columns Qualifications Required: - Bachelor's or master's degree in computer science or electrical engineering - Expert software engineering skills with proven experience in developing complex software systems - Strong core Python software engineering experience - Significant experience with projects using relational data models and SQL - Spark experience is a must - Good analytical skills and the ability to break down complexity into achievable steps - Highly proficient in English (written and verbal) (Note: No additional details about the company were provided in the job description),