ETL Python

P1, C3, STS Design, build data cleansing and imputation, map to a standard data model, transform to satisfy business rules and statistical computations, and validate data content Develop, modify, and maintain Python and Unix Scripts, and complex SQL Performance tuning of the existing code and avoid bottlenecks and improve performance Build an end-to-end data flow from sources to entirely curated and enhanced data sets Develop automated Python jobs for ingesting data from various source systems Provide technical expertise in areas of architecture, design, and implementation Work with team members to create useful reports and dashboards that provide insight, improve/automate processes, or otherwise add value to the team Write SQL queries for data validation Design, develop and maintain ETL processes to extract, transform and load data from various sources into the data warehouse Collaborate with data architects, analysts and other stakeholders to understand data requirements and ensure quality Optimize and tune ETL processes for performance and scalability Develop and maintain documentation for ETL processes, data flows, and data mappings Monitor and troubleshoot ETL processes to ensure data accuracy and availability Implement data validation and error handling mechanisms Work with large data sets and ensure data integrity and consistency Skills Python ETL Tools like Informatica, Talend, SSIS or similar SQL, MySQL Expertise in Oracle, SQL Server and Teradata DevOps, GIT Lab Experience in AWS Glue or Azure Data Factory Qualification Design, build data cleansing and imputation, map to a standard data model, transform to satisfy business rules and statistical computations, and validate data content Develop, modify, and maintain Python and Unix Scripts, and complex SQL Performance tuning of the existing code and avoid bottlenecks and improve performance Build an end-to-end data flow from sources to entirely curated and enhanced data sets Develop automated Python jobs for ingesting data from various source systems Provide technical expertise in areas of architecture, design, and implementation Work with team members to create useful reports and dashboards that provide insight, improve/automate processes, or otherwise add value to the team Write SQL queries for data validation Design, develop and maintain ETL processes to extract, transform and load data from various sources into the data warehouse Collaborate with data architects, analysts and other stakeholders to understand data requirements and ensure quality Optimize and tune ETL processes for performance and scalability Develop and maintain documentation for ETL processes, data flows, and data mappings Monitor and troubleshoot ETL processes to ensure data accuracy and availability Implement data validation and error handling mechanisms Work with large data sets and ensure data integrity and consistency