Lead Data Architect

As a Lead Data Engineer at Weekday's client, you will be responsible for designing, building, and optimizing scalable data pipelines for large-scale data processing. You should have a strong background in Python, PySpark, SQL, Azure Databricks, Azure DataFactory, Azure Synapse, ETL, and Data Architect. The ideal candidate will leverage their expertise in data processing and cloud platforms, specifically Azure, to handle complex data engineering tasks effectively. Your key responsibilities will include designing and optimizing scalable data pipelines using Python and Apache Spark, utilizing Azure Databricks for big data processing, and ensuring seamless data integration and orchestration across cloud environments with Azure Data Factory. You will also implement solutions using Azure Data Lake Storage, write efficient Python code, work with core Azure data services, design complex SQL queries, and ensure high-quality data management, integrity, and security throughout the data processing and analysis stages. To excel in this role, you should have 7+ years of experience in data engineering or related roles, with a strong focus on cloud technologies and big data. Proficiency in Python, including knowledge of Pythonic principles, data structures, and algorithms, is essential. You should also have in-depth knowledge of Azure services and strong experience with Apache Spark, PySpark, and SQL. Excellent communication and problem-solving skills are crucial for collaborating effectively across teams and resolving complex data challenges. If you are passionate about data engineering, have a knack for problem-solving, and thrive in a collaborative environment, we encourage you to apply for this exciting opportunity and contribute to our growing team.,