Data Engineer-Analytics aaS Associate - Operate

As a Data Engineer Offshore at PwC, you will be part of the managed services team focusing on outsourced solutions and supporting clients across various functions. Your role will involve streamlining operations, reducing costs, and improving efficiency for organizations by managing key processes and functions on their behalf. Utilizing your skills in project management, technology, and process optimization, you will deliver high-quality services to clients. Your main responsibilities will include transitioning and running services, managing delivery teams, programs, commercials, performance, and delivery risk. Continuous improvement and optimization of managed services processes, tools, and services will be at the core of your work. Key Responsibilities: - Design, develop, and maintain scalable ETL pipelines using DataStage and other ETL tools. - Utilize AWS cloud services for data storage, processing, and analytics. - Implement and optimize Delta Live Tables and Delta Tables for efficient data storage and querying. - Work with Apache Spark to process large datasets, ensuring optimal performance and scalability. - Integrate Kafka and Spark Streaming to build real-time data processing applications. - Collaborate with cross-functional teams to gather requirements and deliver data solutions meeting business needs. - Ensure data quality, integrity, and security across all data systems and pipelines. - Monitor and troubleshoot data workflows for smooth operations. - Document data processes, architecture designs, and technical specifications. Qualifications: - Bachelor's degree in Computer Science, Information Technology, or a related field. - Proven experience as a Data Engineer or in a similar role. - Strong proficiency in SQL and experience with relational databases such as Teradata. - Hands-on experience with AWS services like S3, EMR, Redshift, and Lambda. - Proficiency in using Databricks for data engineering tasks. - Experience with Delta Live Tables and Delta Tables in a data engineering context. - Solid understanding of Apache Spark, Kafka, and Spark Streaming. - Experience with messaging systems like MQ is a plus. - Strong problem-solving skills and attention to detail. - Excellent communication and collaboration skills. (Note: Additional Company Details have been omitted from the provided Job Description),