Cloud Data Engineer

Role Overview: As a Data Engineer Offshore at PwC, you will be part of the managed services team focusing on various outsourced solutions to support clients across different functions. Your role involves helping organizations streamline operations, reduce costs, and enhance efficiency by managing key processes and functions on their behalf. You will utilize your skills in project management, technology, and process optimization to deliver high-quality services to clients. Your primary focus will be on transitioning and running services, managing delivery teams, programmes, commercials, performance, and delivery risk. Your work will revolve around continuous improvement and optimizing managed services processes, tools, and services. Key Responsibilities: - Design, develop, and maintain scalable ETL pipelines using DataStage and other ETL tools. - Utilize AWS cloud services for data storage, processing, and analytics. - Analyze, process, and transform data using Databricks to ensure high performance and reliability. - Implement and optimize Delta Live Tables and Delta Tables for efficient data storage and querying. - Process large datasets with Apache Spark to ensure optimal performance and scalability. - Build real-time data processing applications by integrating Kafka and Spark Streaming. - Collaborate with cross-functional teams to gather requirements and deliver data solutions meeting business needs. - Ensure data quality, integrity, and security across all data systems and pipelines. - Monitor and troubleshoot data workflows to ensure smooth operations. - Document data processes, architecture designs, and technical specifications. Qualifications: - Bachelor's degree in Computer Science, Information Technology, or a related field. - Proven experience as a Data Engineer or similar role. - Strong proficiency in SQL and experience with relational databases like Teradata. - Hands-on experience with AWS services such as S3, EMR, Redshift, and Lambda. - Proficiency in using Databricks for data engineering tasks. - Experience with Delta Live Tables and Delta Tables in a data engineering context. - Solid understanding of Apache Spark, Kafka, and Spark Streaming. - Experience with messaging systems like MQ is a plus. - Strong problem-solving skills and attention to detail. - Excellent communication and collaboration skills. Additional Details: At PwC, you will have the opportunity to build meaningful client connections, manage and inspire others, and deepen your technical expertise. You are expected to anticipate the needs of your teams and clients, deliver quality, and embrace increased ambiguity to grow personally and professionally.,