You will be responsible for testing end-to-end data pipelines built in Azure Data Factory and Databricks, ensuring data flows accurately from source to sink.
Using PySpark and Python, you will validate complex business logic and data transformations created by the engineering team.
You will engage in pipeline reviews, treating test as code and ensuring that the engineering concepts applied are robust and scalable.
You will manage and execute tests within Azure DevOps environments, ensuring that data quality is maintained throughout the continuous integration and delivery lifecycle.
While the organizational title is Tester, your role is that of a hands-on technical contributor. You will take ownership of the validation strategy for your specific domain.

Why you will love this role

This isn t traditional manual or API testing. You will be writing code and working with the same heavy-weight tech stack as the developers.
You are the final gatekeeper for data integrity, ensuring that the insights derived from our platforms are accurate and reliable.
While we are looking for individual contributors, we are positioning this role for future scaling. You will have the opportunity to influence how our data QA processes evolve.
We have stripped away the overhead you won t be bogged down by data generation, lineage tracing, or monitoring. You get to focus entirely on the logic and the code.

We would like you to bring along

6 to 10 years of total experience, with at least 5-7 years of focused experience in data-centric technical roles.
Strong Python and PySpark skills are non-negotiable. You must be able to write scripts to validate large datasets.
Hands-on experience with Azure Data Factory and Databricks.
Advanced SQL knowledge for data manipulation, comparison, and validation.
Practical experience with Azure DevOps for managing pipelines and version control.
A solid understanding of data engineering principles you must be able to speak the same language as the developers you are testing for.

Good-to-have skills

Slight experience as a Team Lead or a desire to mentor others as the team scales.
While we can train you on specific QA methodologies, a foundational understanding of the software testing life cycle (STLC) is a plus.
Experience identifying bottlenecks within PySpark jobs or SQL queries.

Mock Interview

Practice Video Interview with JobPe AI

Start DevOps Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.