Lead- Data Engineer

Job Description

Technical skills: Should have 6+ years of hands-on experience in Data engineering Excellent command over ETL tools covering Snowflake, Azure Data Lake, Databricks and Azure Data Factory Good command over writing Python Scripts for custom build data ingestion Proficient in SQL scripts (Stored procedure, Views, Functions, Joins) Experience with Snowflake Experience with DBT (for ETL transformation logic) Experience with FIVETRAN (for data ingestion) Good understanding of data warehouse fundamentals and principles, database modelling knowledge and experience Experience utilizing data pipelines and connectors across different source systems Experience with cloud data integration tools and services related to Fivetran, DBT or similar data ingestion and data transformation tools Experience working Git version control, preferably GitHub Professional IT related education or equivalent (university) education (e.g., B.E/BTech/MCA/MSC) in a relevant subject, preferably in Computer Science is a preferred Experience working as ETL/ELT developer or a Data engineer Requirements for the role: Analzse and organize raw data, providing datasets and sources for business reporting requirements Implement data quality rules to enhance data reliability Understand business needs and objectives Conduct complex data analysis and data modelling, interpreting trends and patterns Collaborate with data scientists and Reporting engineers to help resolve the reporting and Analytics needs Provide expertise and leadership in the development of new products, services, and processes from a Data Engineering technical perspective. Work independently adhering to high technical standards, taking ownership of projects ensuring they conform to departmental standards before going to QA. Work effectively within an agile, deadline-driven production process to complete project development with rapid, iterative cycles. Liaise with Team Leaders, Product and Project Managers to ensure projects are taken from creation to completion - on time and on budget. Provide accurate estimates for existing and new projects alike. Be an integral part of the code review process, providing constructive feedback against code produced by other team members. Ensure good communication is maintained with other team members/departments throughout project life cycles. Write clear, clean, and concise scripts, SQL and transformation logic Fix bugs and transformation problems rapidly Use lateral thinking and investigative techniques to solve challenges across various technical streams. Participate in the ongoing development, adoption and promotion of Data Engineering standards Investigate and participate in discussions with team members about best practices and help identify optimal technical solutions Actively contribute to departmental learning and standards within the team environment through involvement in presentations/demonstrations and participate in the continual drive for collaboration and knowledge sharing amongst the team Keep abreast of the latest trends and emerging technologies within the Data Engineering community and continually develop skill