Software Development Engineer III

Must-Have Skills:

• Hands-on experience with Databricks and Apache Spark for building and validating scalable data pipelines
• Strong expertise in AWS services including S3, Glue, Athena, Redshift, and Lake Formation
• Proficient in Python, PySpark, and SQL for developing test automation and validation logic
• Experience validating data from various file formats such as JSON, CSV, Parquet, and Avro
• In-depth understanding of

  data integration workflows

  including

  batch

  and

  real-time (streaming)

  pipelines
• Strong ability to define and automate

  data quality checks

  : schema validation, null checks, duplicates, thresholds, and transformation validation
• Experience designing

  modular, reusable automation frameworks

  for large-scale data validation
• Skilled in integrating tests with

  CI/CD tools

  like

  GitHub Actions

  ,

  Jenkins

  , or

  Azure DevOps

• Familiarity with

  orchestration tools

  such as

  Apache Airflow

  ,

  Databricks Jobs

  , or

  AWS Step Functions

• Hands-on experience with

  API testing

  using

  Postman

  ,

  pytest

  , or custom automation scripts
• Proven track record of

  leading and mentoring QA/test engineering teams

• Ability to define and own

  test automation strategy and roadmap

  for data platforms
• Strong

  collaboration skills

  to work with engineering, product, and data teams
• Excellent communication skills for presenting

  test results, quality metrics

  , and

  project health

  to leadership
• Contributions to

  internal quality dashboards

  or

  data observability systems

• Awareness of

  metadata-driven testing approaches

  and

  lineage-based validations

• Experience working with agile Testing methodologies such as Scaled Agile.
• Familiarity with automated testing frameworks like Selenium, JUnit, TestNG, or PyTest.