Data Engineer Lead

As a detail-oriented and business-savvy Data Engineer Lead at DigiValet, your role involves turning data into actionable insights that drive strategic decisions. You will collaborate with cross-functional teams to deliver impactful dashboards, conduct deep-dive analyses, and support data-driven decision-making. Key Responsibilities: - Collaborate with business stakeholders to understand key performance indicators (KPIs) and their impact on business objectives. - Translate business questions and KPIs into analytical frameworks and data requirements. - Apply advanced analytical techniques to uncover underlying drivers and correlations within KPI data. - Develop insightful visualisations and reports that effectively communicate KPI trends and patterns. - Provide actionable recommendations to improve KPI performance and achieve business goals. - Design, develop, and maintain scalable data pipelines using Databricks and PySpark. - Perform ETL (extraction, transformation, loading) from diverse data sources. - Ensure data quality, consistency, and integrity through rigorous data cleaning and validation. - Optimize data storage and retrieval for efficient analysis and reporting. - Conduct exploratory data analysis (EDA) to identify patterns, trends, and anomalies. - Apply statistical techniques (e.g., hypothesis testing, regression analysis) to validate findings and quantify relationships. - Develop and implement predictive and prescriptive models using machine learning algorithms. - Evaluate model performance and iterate to improve accuracy and robustness. - Build and deploy machine learning models using frameworks like scikit-learn, TensorFlow, or PyTorch. - Implement AI solutions to address specific business challenges and optimize KPI performance. - Monitor and maintain deployed models, ensuring optimal performance and scalability. - Apply MLOps best practices for continuous integration and deployment. - Translate complex data insights into clear, concise, and actionable recommendations. - Create compelling visualisations and dashboards to communicate findings effectively. - Present data-driven insights to both technical and non-technical audiences. - Document all code, models, and analytical processes for transparency and reproducibility. Required Skills: - 5+ years of hands-on experience in data analytics or business intelligence roles. - Proficient in SQL (writing complex queries, joins, subqueries, CTEs, window functions). - Strong expertise in data modeling, and report/dashboard creation. - Experience working with Databricks or other modern data platforms (e.g., Snowflake, BigQuery, Redshift). - Working knowledge of Python for data manipulation and visualisation. - Understanding of data warehousing concepts, lakehouse architectures, and data integration. - Excellent problem-solving, communication, and stakeholder management skills. In this role, you will have the opportunity to work in a mid-size and fast-growing company, engage in exciting projects involving cutting-edge technologies, and benefit from professional development opportunities. The company promotes a friendly working environment without a cliche office hierarchy system, allowing you to contribute new ideas and be appreciated for your innovative thinking.,