Azure with Azure Data Factory & Pyspark - Lead

You will be responsible for designing, implementing, and optimizing end-to-end data pipelines on Azure with a focus on scalability and performance. Your role will involve developing and maintaining ETL workflows to ensure seamless data processing using Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks. It is essential to leverage PySpark for advanced data transformations, including data cleansing, enrichment, and validation processes to deliver high-quality and well-structured output. As a Data Engineer, you will be expected to optimize data pipelines, queries, and PySpark jobs to enhance overall performance and scalability. Your expertise in Azure, particularly Azure Data Factory, and proficiency in PySpark for data processing and analytics are crucial for this role. Experience with data modeling, ETL processes, and data warehousing will be beneficial in successfully delivering on the responsibilities of this position. Key Skills required for this role include proficiency in Azure, Azure Data Factory, and PySpark. About UST: UST is a global digital transformation solutions provider, collaborating with leading companies worldwide for over 20 years to drive real impact through transformation. With a focus on technology, people, and purpose, UST partners with clients from design to operation, embedding innovation and agility into their organizations. With a workforce of over 30,000 employees across 30 countries, UST aims to create boundless impact, touching billions of lives in the process.,