Lead Data Engineer

As a Big Data Engineer at this company, your role involves designing, developing, and maintaining scalable and distributed data architectures capable of processing large volumes of data. Your key responsibilities include: - Designing Big Data Architecture: Develop scalable and distributed data architectures for processing large volumes of data effectively. - Implementing Data Storage Solutions: Utilize technologies such as Hadoop, Spark, and PySpark to implement and optimize data storage solutions. - Developing PySpark ETL Processes: Create and implement efficient ETL processes using PySpark to extract, transform, and load large datasets. - Optimizing Performance: Optimize PySpark applications to enhance performance, scalability, and resource management. To qualify for this role, you should have: - Proven experience as a Big Data Engineer with a strong focus on PySpark. - Deep understanding of Big Data processing frameworks and technologies. - Strong proficiency in PySpark for developing and optimizing ETL processes and data transformations. - Experience with distributed computing and parallel processing. - Ability to collaborate effectively in a fast-paced, innovative environment. About the company: UST is a global digital transformation solutions provider that partners with clients from design to operation, embedding innovation and agility into their organizations. With over 30,000 employees in 30 countries, UST aims to make a real impact through transformation, touching billions of lives in the process. As a Big Data Engineer at this company, your role involves designing, developing, and maintaining scalable and distributed data architectures capable of processing large volumes of data. Your key responsibilities include: - Designing Big Data Architecture: Develop scalable and distributed data architectures for processing large volumes of data effectively. - Implementing Data Storage Solutions: Utilize technologies such as Hadoop, Spark, and PySpark to implement and optimize data storage solutions. - Developing PySpark ETL Processes: Create and implement efficient ETL processes using PySpark to extract, transform, and load large datasets. - Optimizing Performance: Optimize PySpark applications to enhance performance, scalability, and resource management. To qualify for this role, you should have: - Proven experience as a Big Data Engineer with a strong focus on PySpark. - Deep understanding of Big Data processing frameworks and technologies. - Strong proficiency in PySpark for developing and optimizing ETL processes and data transformations. - Experience with distributed computing and parallel processing. - Ability to collaborate effectively in a fast-paced, innovative environment. About the company: UST is a global digital transformation solutions provider that partners with clients from design to operation, embedding innovation and agility into their organizations. With over 30,000 employees in 30 countries, UST aims to make a real impact through transformation, touching billions of lives in the process.