ETL Automation Tester

You are an experienced ETL Automation Tester with a solid background in testing data pipelines, data sources, and ETL processes. Your primary responsibility will be to validate and automate the testing of ETL jobs that extract, transform, and load data into target databases. Proficiency in Python scripting is essential for building and maintaining automation frameworks for data validation, ensuring data integrity across systems, and handling large datasets efficiently. Your key responsibilities include performing detailed testing of ETL processes to guarantee accurate extraction, transformation based on business rules, and loading into target databases. You will develop and execute Python scripts for automating ETL validation and data quality checks, collaborating closely with Data Engineers and Business Analysts to understand data flows, transformations, and reporting requirements. Creating comprehensive test cases and scenarios for different stages of the ETL pipeline to cover all data integration points is crucial. You will also validate transformation logic, data aggregation, filtering, and enrichment rules, monitor data loading processes into data warehouses for correctness, completeness, and timeliness, and write and execute complex SQL queries to verify data integrity by comparing source and target datasets. Identifying and troubleshooting data anomalies, mismatches, and discrepancies in both structured and unstructured data is part of your role. Ensuring the performance and scalability of ETL processes by identifying bottlenecks and testing large datasets are essential tasks. Additionally, preparing and presenting test execution reports to provide insights on testing progress, bugs, and issues found is crucial for the role. Mandatory Skills: - ETL Testing: Expertise in testing data extraction, transformation, and loading processes across various data pipelines. - Python: Strong proficiency in Python for developing automated test scripts and frameworks. - Data Validation: Proven ability to validate data integrity and accuracy through comprehensive testing strategies. - SQL: Advanced knowledge of SQL to write and execute queries for testing and verifying data at different stages of the ETL process. - Data Sources and Targets: Experience in testing across various data sources (e.g., flat files, APIs, databases) and target systems (e.g., data warehouses, cloud storage).,