Data Engineer

About Us:

Job Summary:
We are seeking skilled Data Engineers with strong expertise in Databricks, PySpark, Spark SQL, Python, and Delta Lake to design and develop robust data pipelines and cloud integrations. You will work primarily on Databricks deployed on either AWS or Azure cloud-specific expertise is not mandatory but cross-cloud exposure is a plus. Hands-on experience with Databricks Unity Catalog, Federated Catalog, and Infrastructure as Code (Terraform) is required. You should be proficient in writing unit and integration tests using pytest to ensure high-quality, reliable data solutions.

Key Responsibilities:
* Design, build, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, Python, and Delta Lake. * Implement and manage Databricks Unity Catalog and Federated Catalog for secure and organized data governance. * Develop and maintain Infrastructure as Code (IaC) using Terraform for cloud resource provisioning. * Integrate Databricks solutions with AWS and/or Azure cloud services as needed. * Write robust unit and integration tests using pytest to ensure code quality and reliability. * Optimize data workflows for performance, scalability, and cost efficiency. * Collaborate with data architects, cloud engineers, and stakeholders to deliver end-to-end data solutions. * Follow best practices for version control, CI/CD, and documentation of data engineering processes.

Role Requirements and Qualifications:
* Proven experience working with Databricks, PySpark, Spark SQL, Python, and Delta Lake in production environments. * Hands-on expertise in implementing and managing Databricks Unity Catalog and Federated Catalog. * Strong knowledge of data pipelines, cloud data integration, and distributed data processing frameworks. * Experience with Infrastructure as Code (IaC) using Terraform for deploying and managing cloud resources. * Familiarity with AWS and/or Azure cloud platforms; exposure to both is a plus but not mandatory. * Proficiency in writing unit tests and integration tests using pytest or similar frameworks. * Solid understanding of best practices for data security, data governance, and compliance. * Good grasp of CI/CD pipelines and version control systems (e.g., Git). * Bachelor s or Master s degree in Computer Science, Data Engineering, or a related field (or equivalent work experience). * Strong analytical, problem-solving, and communication skills.

Why Join Us:
* Opportunities to work on transformative projects, cutting-edge technology and innovative solutions with leading global firms across industry sectors. * Continuous investment in employee growth and professional development with a strong focus on up & re-skilling. *Competitive compensation & benefits, ESOPs and international assignments. * Supportive environment with healthy work-life balance and a focus on employee well-being. * Open culture that values diverse perspectives, encourages transparent communication and rewards contributions.

How to Apply:
If you are interested in joining our team and meet the qualifications listed above, please apply and submit your resume highlighting why you are the ideal candidate for this position.