Job
Description
Role Overview: As a meticulous and detail-oriented Data Validation and BI Test Engineer at Greenmen Consulting Private Limited, your primary responsibility will be to ensure the accuracy, reliability, and consistency of data within the Databricks environment and the integrity of key performance indicators (KPIs) presented in Power BI. You will play a crucial role in establishing and executing robust testing strategies, identifying data discrepancies, and ensuring the delivery of high-quality data and business intelligence insights. Your role will require a strong understanding of data warehousing concepts, ETL processes, data validation techniques, and experience with both Databricks and Power BI. Key Responsibilities: - Develop and Execute Test Strategies: Design, develop, and implement comprehensive test plans and test cases for data validation, transformation logic within Databricks, and Power BI KPI accuracy. - Data Validation and Testing: Perform rigorous data validation and testing across various stages of the data pipeline, including source systems, data ingestion, data transformation within Databricks (using SQL, PySpark, etc.), and data warehousing. - Reconciliation: Develop and execute reconciliation processes to compare data across different systems and environments, identifying and resolving discrepancies. - Power BI KPI Testing: Design and execute test cases to verify the accuracy, consistency, and performance of KPIs, calculations, and visualizations within Power BI reports and dashboards. - Automation: Identify opportunities for test automation and implement automated test scripts using relevant tools and frameworks to improve efficiency and test coverage. - Defect Management: Identify, document, and track data quality issues and testing defects, working closely with data engineers and BI developers to ensure timely resolution. - Collaboration: Collaborate effectively with data engineers, BI developers, analysts, and other stakeholders to understand data requirements, data flows, and reporting needs. - Documentation: Create and maintain clear and concise documentation for test plans, test cases, test results, and validation procedures. - Performance Testing: Participate in performance testing of Databricks workloads and Power BI reports to ensure optimal performance and scalability. - Continuous Improvement: Stay up-to-date with the latest data validation and testing methodologies, tools, and technologies, and proactively suggest improvements to our testing processes. Qualifications: - Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field. - 7+ years of experience in data validation, data testing, or quality assurance, with a focus on data-centric projects. - Strong proficiency in SQL and experience querying and manipulating large datasets. - Hands-on experience working with Databricks, including data transformation using SQL and/or PySpark. - Solid understanding of data warehousing concepts, ETL processes, and data modeling techniques. - Experience in testing and validating Power BI reports, dashboards, and KPIs. - Familiarity with data quality principles and methodologies. - Experience with test automation tools and frameworks is a plus. - Excellent analytical, problem-solving, and debugging skills. - Strong attention to detail and a commitment to data accuracy and quality. - Excellent communication and collaboration skills. - Ability to work independently and as part of a team in a fast-paced environment.,
