Design and execute test plans for ETL processes, ensuring data accuracy, completeness, and integrity.Develop automated test scripts using Python or R for data validation and reconciliation.Perform source-to-target data verification, transformation logic testing, and regression testing.Collaborate with data engineers and analysts to understand business requirements and data flows.Identify data anomalies and work with development teams to resolve issues.Maintain test documentation, including test cases, test results, and defect logs.Participate in performance testing and optimization of data pipelines.
Required Skills & Qualifications:
Strong experience in ETL testing across various data sources and targets.Proficiency in Python or R for scripting and automation.Solid understanding of SQL and relational databases.Familiarity with data warehousing concepts and tools (e.g., Power BI, QlikView, Informatica, Talend, SSIS).Experience with test management tools (e.g., JIRA, TestRail).Knowledge of data profiling, data quality frameworks, and validation techniques.Excellent analytical and communication skills. Qualification
Experience At least 5 years of experience in AWS based projects.Technical skills Proficiency in Python and PySpark for data engineering tasks.Big Data Strong knowledge of Big Data technologies and data warehousing concepts.AWS services Experience with AWS Data Engineering stack, including S3, RDS, Athena, Glue, Lambda, and Step Functions.SQL Strong SQL skills for data manipulation and querying.CI CD Experience with CI CD tools like Terraform and Git Actions.Soft skills Good communication skills and ability to work in a multicultural team.Design and implement data pipelines Develop ETL jobs to ingest and move data within the AWS environment using tools like AWS GlueData storage and processing Build and maintain systems for data collection storage processing and analysis using AWS services such as S3 RDS Athena and RedshiftBig Data technologies Utilize Big Data technologies like Hadoop HDFS and Spark for data processing