2 Airflow Jobs Jobs

Experience Range: 4-8 years in Data Quality Engineering Job Summary: As a Senior Data Quality Engineer, you will play a key role in ensuring the reliability and accuracy of our data platform and projects. Your primary responsibility will be developing and leading the product testing strategy while leveraging your technical expertise in AWS and big data technologies. You will also guide the team in implementing shift-left testing using Behavior-Driven Development (BDD) methodologies integrated with AWS CodeBuild CI/CD. Your contributions will ensure the successful execution of testing across multiple data platforms and projects. Key Responsibilities: Develop Product Testing Strategy: Collaborate with stakeholders to define and implement the product testing strategy. Identify key platform and project responsibilities, ensuring a comprehensive and effective testing approach. Lead Testing Strategy Implementation: Take charge of implementing the testing strategy across data platforms and projects, ensuring thorough coverage and timely completion of tasks. BDD & AWS Integration: Utilize Behavior-Driven Development (BDD) methodologies to drive shift-left testing and integrate AWS services such as AWS Glue, Lambda, Airflow jobs, Athena, Quicksight, Amazon Redshift, DynamoDB, Parquet, and Spark to improve test effectiveness. Test Execution & Reporting: Design, execute, and document test cases while providing comprehensive reporting on testing results. Collaborate with the team to identify the appropriate data for testing and manage test environments. Collaboration with Developers: Work closely with application developers and technical support to analyze and resolve identified issues in a timely manner. Automation Solutions: Create and maintain automated test cases, enhancing the test automation process to improve testing efficiency. Must-Have Skills: Big Data Platform Expertise: At least 2 years of experience as a technical test lead working on a big data platform, preferably with direct experience in AWS. Strong Programming Skills: Proficiency in object-oriented programming, particularly with Python. Ability to use programming skills to enhance test automation and tooling. BDD & AWS Integration: Experience with Behavior-Driven Development (BDD) practices and AWS technologies, including AWS Glue, Lambda, Airflow, Athena, Quicksight, Amazon Redshift, DynamoDB, Parquet, and Spark. Testing Frameworks & Tools: Familiarity with testing frameworks such as PyTest, PyTest-BDD, and CI/CD tools like AWS CodeBuild and Harness. Communication Skills: Exceptional communication skills with the ability to convey complex technical concepts to both technical and non-technical stakeholders. Good-to-Have Skills: Automation Engineering: Expertise in creating automation testing solutions to improve testing efficiency. Experience with Test Management: Knowledge of test management processes, including test case design, execution, and defect tracking. Agile Methodologies: Experience working in Agile environments, with familiarity in using Agile tools such as Jira to track stories, bugs, and progress. Experience Range: Minimum Requirements: Bachelors degree in Computer Science or related field, or HS/GED with 8 years of experience in Data Quality Engineering. At least 4 years of experience in big data platforms and test engineering, with a strong focus on AWS and Python. Skills Test Automation,Python,Data Engineering

Job Summary: As a Senior Data Quality Engineer, you will play a crucial role in ensuring the reliability and accuracy of our data platform and projects. Your primary responsibilities will involve developing and leading the product testing strategy, leveraging your technical expertise in AWS and big data technologies. You will also work closely with the team to implement shift-left testing using Behavior-Driven Development (BDD) methodologies integrated with AWS CodeBuild CI/CD. Key Responsibilities: Develop Product Testing Strategy: Collaborate with stakeholders to define and design the product testing strategy, identifying key platform and project responsibilities. Lead Testing Strategy Implementation: Take charge of implementing the testing strategy, ensuring its successful execution across the data platform and projects. Oversee and coordinate testing tasks to ensure thorough coverage and timely completion. BDD and AWS Integration: Guide the team in utilizing Behavior-Driven Development (BDD) practices for shift-left testing. Leverage AWS services (e.g., AWS Glue, Lambda, Airflow, Athena, Quicksight, Redshift, DynamoDB, Parquet, Spark) to enhance testing effectiveness. Test Case Management: Work with the team to identify and prepare data for testing, create/maintain automated test cases, execute test cases, and document results. Problem Resolution: Assist developers and technical support staff in resolving identified issues in a timely manner. Automation Engineering Solutions: Create test automation solutions that improve the efficiency and coverage of testing efforts. Must-Have Skills: Big Data Platform Expertise: At least 2 years of experience as a technical test lead working on a big data platform, preferably with direct experience in AWS. AI/ML Familiarity: Experience with AI/ML concepts and practical experience working on AI/ML-driven initiatives. Synthetic Test Data Creation: Knowledge of synthetic data tooling, test data generation, and best practices. Offshore Team Leadership: Proven ability to lead and collaborate with offshore teams, managing projects with limited real data access. Programming Expertise: Strong proficiency in object-oriented programming, particularly with Python. Testing Tools/Frameworks: Familiarity with tools like PyTest, PyTest-BDD, AWS CodeBuild, and Harness. Excellent Communication: Ability to communicate effectively with both technical and non-technical stakeholders, explaining complex technical concepts in simple terms. Good-to-Have Skills: Experience with AWS Services: Familiarity with AWS DL/DW components like AWS Glue, Lambda, Airflow jobs, Athena, Quicksight, Amazon Redshift, DynamoDB, Parquet, and Spark. Test Automation Experience: Practical experience in implementing test automation frameworks for complex data platforms and systems. Shift-Left Testing Knowledge: Experience in implementing shift-left testing strategies, particularly using Behavior-Driven Development (BDD) methodologies. Project Management: Ability to manage multiple testing projects simultaneously while ensuring the accuracy and quality of deliverables. Minimum Requirements: Bachelors in Computer Science and 4 years of relevant experience, or High School/GED with 8 years of relevant experience. Relevant Experience: Big Data platform testing, test strategy leadership, automation, and working with AWS services and AI/ML concepts. Skills Test Automation,Python,Data Engineering